AJAX Using JSON for data transfer

At its simplest, JSON allows you to transform a set of data represented in a JavaScript object into a string that you can easily pass from one function to another, or -- in the case of asynchronous applications -- from a Web client to a server-side program. The string looks a little odd (you'll see some examples in just a moment), but it's easily interpreted by JavaScript, and JSON allows you to represent structures more complex than name/value pairs. For instance, you can represent arrays and complex objects, rather than just simple lists of keys and values.

Simple JSON examples

At its very simplest, you can represent what is essentially a name/value pair in JSON like this:

{ "firstName": "Tom" } 

This is pretty basic and actually takes up more space than the equivalent name/value pair in clear text:

firstName=Tom

However, JSON illustrates its value when you start to string together multiple name/value pairs. First, you can create what is essentially a record of data, with multiple name/value pairs, like this:

{ "firstName": "Tom", "lastName":"Kirschner", "email": "Tom@mywebmemo.com" }

There's still not much advantage here over name/value pairs in terms of syntax, but in this case JSON is significantly easier to use, and has some readability advantages. For example, it's clear that all three of the values above are part of the same record; the brackets establish that the values have some connection.

Arrays of values

When you need to represent a set of values, JSON starts to be not only more readable, but less verbose. Say, for example, that you want to have a list of people. In XML, you'd be stuck with lots of opening and closing tags, and if you were using typical name/value pairs -- the kind we've looked at in earlier articles in this series -- then you'd have to come up with a proprietary data format, or perhaps modify the key names to something like person1-firstName.

With JSON, you can simply group multiple bracketed records:

{ "people": [
  { "firstName": "Tom", "lastName":"Kirschner", "email": "Tom@mywebmemo.com" },
  { "firstName": "Jason", "lastName":"Smith ", "email": "jason@examples1.com" },
  { "firstName": "David ", "lastName":"Delves ", "email": "davioo@examples2.com" }
]}

This isn't too hard to understand. In this case, there's a single variable, named people, and the value is the array containing three items, each of which is a person record with a first name, a last name, and an e-mail address. The example above illustrates how you can throw records together, and also group the items into a single value with brackets around it. Of course, you could use the same syntax, but have multiple values (each with multiple records):

{ "programmers": [
  { "firstName": "Tom", "lastName":"Kirschner", "email": "Tom@mywebmemo.com" },
  { "firstName": "Jason", "lastName":"Smith ", "email": "jason@examples1.com" },
  { "firstName": "David ", "lastName":"Delves ", "email": "davioo@examples2.com" }
 ],
"authors": [
  { "firstName": "Isaac", "lastName": "Asimov", "genre": "science fiction" },
  { "firstName": "Tad", "lastName": "Williams", "genre": "fantasy" },
  { "firstName": "Frank", "lastName": "Peretti", "genre": "christian fiction" }
 ],
"musicians": [
  { "firstName": "Alejandro ", "lastName": "Clapton", "instrument": "guitar" },
  { "firstName": "Sergei", "lastName": "Rachmaninoff", "instrument": "piano" }
 ]
}

The most obvious thing to note here is your ability to represent multiple values that each in turn have multiple values. What you should also notice, though, is that the actual name/value pairs in the records change across the different main entries (programmers, authors, and musicians). JSON is completely dynamic and lets you change the way you represent data in the middle of your JSON structure.

To put it another way, there is no predefined set of constraints that you need to follow when working with JSON-formatted data. So you can change how you represent things, or even represent the same thing in different ways, all within the same data structure.

Using JSON in JavaScript

After you've got a handle on the JSON format, it's simple to use it within JavaScript. JSON is a native JavaScript format, meaning simply that you don't need any special API or toolkit to work with JSON data within JavaScript.

Assigning JSON data to a variable

For example, you can simply create a new JavaScript variable and then directly assign a string of JSON-formatted data to it:

var people =
  { "programmers": [
    { "firstName": "Tom", "lastName":"Kirschner", "email": "Tom@mywebmemo.com" },
    { "firstName": "Jason", "lastName":"Smith ", "email": "jason@examples1.com" },
    { "firstName": "David ", "lastName":"Delves ", "email": "davioo@examples2.com" }
   ],
  "authors": [
    { "firstName": "Isaac", "lastName": "Asimov", "genre": "science fiction" },
    { "firstName": "Tad", "lastName": "Williams", "genre": "fantasy" },
    { "firstName": "Frank", "lastName": "Peretti", "genre": "christian fiction" }
   ],
  "musicians": [
    { "firstName": "Alejandro ", "lastName": "Clapton", "instrument": "guitar" },
    { "firstName": "Sergei", "lastName": "Rachmaninoff", "instrument": "piano" }
   ]
  }

JSON vs. XML
JSON and XML are basically used for the same purpose—to represent and interchange data. I'll try to show you why you might want to use JSON rather than XML in an AJAX context by showing you an example of how an employee class (actually, a list of employees) might be represented, first in XML. and then in JSON. This side-by-side comparison should let you begin to understand how to represent data in JSON. Here's the XML version:

   <?xml version='1.0' encoding='UTF-8'?>
   <employees>
      <employee>
         <surname>Lacava</surname>
         <name>Alessandro</name>
         <city>Milan</city>
         <state>Italy</state>
         <department>IT</department>
      </employee>
         
      <employee>
         <surname>Brown</surname>
         <name>John</name>
         <city>Rome</city>
         <state>Italy</state>
         <department>IT</department>
      </employee>
   </employees>

   {
      "employees" :
      [
        {
          "surname" : "Lacava",
          "name" : "Alessandro",
          "city" : "Milan",
          "state" : "Italy", 
          "department" : "IT"
        },
        {
          "surname" : "Brown",
          "name" : "John",
          "city" : "Rome",
          "state" : "Italy", 
          "department" : "IT"
        }
      ]
   }

There's nothing complicated going on here; now people contains the JSON formatted data we've been looking at throughout this article. However, this doesn't do much, as the data still isn't in a format that is obviously useful.

Accessing the data

While it might not be obvious, that lengthy string above is just an array, and once you've got that array in a JavaScript variable, you can access it easily. In fact, you can simply separate the array with period delimiters. So, to access the last name of the first entry of the programmers list, you would use code like this in your JavaScript:

people.programmers[0].lastName;

Take note that the indexing is zero-based. So this begins with the data in the people variable; it then moves to the item called programmers and pulls off the first record ([0]); finally, it accesses the value for the lastName key. The result is the string value "Kirschner".

Here are a few more examples using the same variable.

people.authors[1].genre			// Value is "fantasy"

people.musicians[3].lastName		// Undefined. This refers to the fourth entry,
 and there isn't one

people.programmers.[2].firstName	// Value is "David "

With this little bit of syntax, you can work with any variety of JSON-formatted data, all without any extra JavaScript toolkits or APIs.

Modifying JSON data

Just as you can access data with the dot and bracket notation shown above, you can easily modify data in the same way:

people.musicians[1].lastName = "Rachmaninov";

That's all you need to do to change data in a variable once you've converted from a string to JavaScript objects.

Converting back to strings

Of course, all that data modification isn't of much value if you can't easily convert back into the textual format mentioned in this article. That's also trivial in JavaScript:

String newJSONtext = people.toJSONString();

That's all there is to it! Now you've got a string of text that you can use anywhere you like -- you could use it as the request string in an Ajax application, for instance.

Perhaps even more importantly, you can convert any JavaScript object into JSON text. You don't have to work only with variables that were originally assigned a JSON-formatted string. To transform an object named myObject, you would simply issue the same sort of command:

String myObjectInJSON = myObject.toJSONString();

This is the biggest difference between JSON and other data formats this series has explored. With JSON, you call a simple function and get your data, formatted and ready for use. With other data formats, it's your job to handle the conversion between the raw data and the formatted data. Even when using an API like the Document Object Model that provides a function that converts from its own data structure into text, you've got to learn the API and use that API's objects, rather than native JavaScript objects and syntax.

The end result is that when you're already working with lots of JavaScript objects, JSON is almost certainly a good bet for you to easily convert your data into a format that is simple to send in your requests to server-side programs.

Sending JSON to the server isn't particularly difficult, but it is crucial, and you still have a few choices to make. However, once you've already chosen to use JSON, the choices are a lot simpler and quite a bit more limited, so there's not as much to think or worry about. The bottom line is you just need to get your JSON string to the server, preferably as quickly and as simply as possible.

How to Send JSON Data To The Server

Sending JSON in name/value pairs via GET

The easiest way to get your JSON data to the server is to convert it to text, and send it as the value of a name/value pair. Remember, your JSON-formatted data is just one fairly long object, and probably looks something like Listing 1:

var employees = { "accounting" : [   // accounting is an array in employees.
                                    { "firstName" : "John",  // First element
                                      "lastName"  : "Doe",
                                      "age"       : 23 },
                                    
                                    { "firstName" : "Mary",  // Second Element
                                      "lastName"  : "Smith",
                                      "age"       : 32 }
                                  ], // End "accounting" array.                                  
                  "sales"       : [ // Sales is another array in employees.
                                    { "firstName" : "Sally", // First Element
                                      "lastName"  : "Green",
                                      "age"       : 27 },
                                    
                                    { "firstName" : "Jim",   // Second Element
                                      "lastName"  : "Galley",
                                      "age"       : 41 }
                                  ] // End "sales" Array.
                } // End Employees

So you could send this to a server-side script as a name/value pair like this:

var url = "organizeEmployees.php?employees=" + employees.toJSONString();
xmlHttp.open("GET", url, true);
xmlHttp.onreadystatechange = updatePage;
xmlHttp.send(null);

This looks good, but there's one problem: you've got spaces and all sorts of characters in your JSON data string that a Web browser might try and interpret. To ensure that these characters don't mess things up on the server (or in transmission of your data to the server), add in the JavaScript escape() function, like this:

var url = "organizeEmployees.php?employees=" + escape(employees.toJSONString());
request.open("GET", url, true);
request.onreadystatechange = updatePage;
request.send(null);

This function handles whitespace, slashes, and anything else that might trip up browsers and converts them to Web-safe characters (for example, an empty space is converted to %20, which browsers don't treat as a space but instead pass on to a server unchanged). Then servers will (generally automatically) convert these back to what they are supposed to be after transmission.

The downside to this approach is twofold:

• You're sending potentially huge chunks of data using a GET request, which has a length limitation on the URL string. It's a big length, granted, but you never know how long an object's JSON string representation could get, especially if you're using pretty complex objects.
• You're sending all your data across the network as clear text, which is about as unsecure as you can manage to get when it comes to sending data.

To be clear, these are both limitations of GET requests, rather than anything related to JSON data specifically. However, they're very real concerns when you're sending more than just a user's first or last name, or maybe selections on a form. Once you start to deal with anything that might be even remotely confidential, or extremely lengthy, then you should look at using POST requests.

POSTing JSON data

When you decide you want to move to using a POST request for sending your JSON data to the server, you don't have to make a lot of code changes. Here's what you'd want:

var url = "organizeEmployees.php?timeStamp=" + new Date().getTime();
request.open("POST", url, true);
request.onreadystatechange = updatePage;
request.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
request.send(employees.toJSONString());

The request is opened using POST instead of GET, and the Content-Type header is set to let the server know what sort of data it should expect. In this case, that's application/x-www-form-urlencoded, which lets the server know you're just sending across text like it would get from a normal HTML form.

One other quick note is that the URL has the current time appended to it. That ensures that the request isn't cached after it's sent the first time but is recreated and resent each time this method is called; the URL will be subtly different because of the changing time stamp. This is a common trick to ensure that POSTing to a script actually repeatedly generates a new request each time, and the Web browser doesn't try and cache responses from the server.

JSON is just text

Whether you use GET or POST, the big deal here is that JSON is ultimately just text data. You can easily manipulate it and get it to the server because it doesn't require any special encoding, and every server-side script in the world can handle text data. If JSON were a binary format or had some strange textual encoding, that wouldn't be the case; as it is, though, JSON is just normal text data -- like a script would receive from a form submission, as you saw in the POST section and Content-Type header -- so you don't have a lot to worry about when sending it to a server.

Once you've written your client-side JavaScript code, allowed your users to interact with your Web forms and pages, and gathered information that you need to send on to a server-side program for processing, you're to the point where the server becomes the major player in your application (and probably well into what we all think of as "Ajax applications," assuming you've made the call to the server-side program you're using asynchronous). It's here where the choices you made on the client, like using JavaScript objects and then converting them to JSON strings, must be matched by decisions on the server, like which API to use to decode your JSON data.

Working with JSON on the server side

Working with JSON on the server side is essentially a two-step process, no matter what language you're using on the server:

1. Find a JSON parser/toolkit/helper API for the language you're using to write your server-side programs.
2. Use the JSON parser/toolkit/helper API to take the request data from the client and turn it into something your script can understand.

That's really all that there is to it. Let's take each step in a little more detail.

Find a JSON parser

More importantly, JSON libraries almost certainly exist for your language of choice (there are 24 languages represented in the list on JSON.org). Most of them provide two functions—json_encode($data) and json_decode($json) —and that’s all you need.The best resource for finding a JSON parser or toolkit is the JSON Web site In addition to learning a lot about the format itself, this page has links to JSON tools and parsers for everything from ASP to Erlang to Pike to Ruby. Simply find the language your scripts are written in and download a toolkit. Drop that or expand that or install that (there's a lot of variability when you could be using C#, or PHP, or Lisp on the server) so that your scripts and programs on the server can use the toolkit.

For example, if you're using PHP, you could simply upgrade to PHP 5.2 and be done with it; this recent version of PHP includes the JSON extension by default. In fact, that's almost certainly the best way to handle JSON when you're working with PHP. If you're using Java servlets, the org.json package, hosted at json.org, is a simple choice. In this case, you'd download json.zip from the JSON Web site and add the included source files to your project's build directory. Compile these files, and you're ready to go. Similar steps hold true for the other languages supported; your own expertise using the language you prefer is the best guide.

Using your JSON parser

Once you've got the resources available to your program, it's just a matter of finding the right method to call. For example, suppose you were using the JSON-PHP module with PHP:

// This is just a code fragment from a larger PHP server-side script
require_once('JSON.php');

$json = new Services_JSON();

// accept POST data and decode it
$value = $json->decode($GLOBALS['HTTP_RAW_POST_DATA']);

// Now work with value as raw PHP

With that, you've got all the data -- array format, multiple rows, single values, whatever you stuffed into your JSON data structure -- into a native PHP format, in the $value variable.

If you were using the org.json package in a servlet, you'd use something like this:

public void doPost(HttpServletRequest request, HttpServletResponse response)
  throws ServletException, IOException {

  StringBuffer jb = new StringBuffer();
  String line = null;
  try {
    BufferedReader reader = request.getReader();
    while ((line = reader.readLine()) != null)
      jb.append(line);
  } catch (Exception e) { //report an error }

  try {
    JSONObject jsonObject = new JSONObject(jb.toString());
  } catch (ParseException e) {
    // crash and burn
    throw new IOException("Error parsing JSON request string");
  }

  // Work with the data using methods like...
  // int someInt = jsonObject.getInt("intParamName");
  // String someString = jsonObject.getString("stringParamName");
  // JSONObject nestedObj = jsonObject.getJSONObject("nestedObjName");
  // JSONArray arr = jsonObject.getJSONArray("arrayParamName");
  // etc...
}

You can refer to the org.json package documentation for more details. (Note: If you want more detail go on the org.json )

If you have a variety of tools and techniques that will let you send and receive all kinds of data -- and do it in the best, most efficient manner for each data type -- then you're well on your way to being an Ajax pro. Add JSON to what you've already got available in using XML and plain text, and you're equipped for even more complex data structures in JavaScript.