AJAX

OBJECTIVES

In this unit, the student will learn
  • How AJAX works
    • AJAX in native JavaScript
    • Workaround for old browsers
  • How to use AJAX with jQuery
  • AJAX with XML
  • AJAX with JSON
  • AJAX security and debugging

What is AJAX?

  • Asynchronous JavaScript And XML
    • Asynchronous: requests for content happen in response to events
    • JavaScript: client-side scripts interact with server
    • XML: data typically sent in XML format for flexibility
      • Plain text/HTML is also an option
      • JSON is growing in popularity as an alternative
  • Allows for much more interactivity of web page
    • Page acts more like an application
“fearless, smart and of immense strength.” (study.com) Ajax the Great

AJAX overview

  • Synchronous web browsing before AJAX:
    • Client sends request for page
    • Server responds with entire page
    • Client sends new request (via link or form submission)
    • Server responds with another page ...
    • ... and so on
  • Asynchronous communication with AJAX:
    • Client sends request for page
    • Server responds with some content, not necessarily complete page
    • In response to event, JavaScript requests more content
    • Server sends back additional content
      • Content is inserted into page as update, not reload

Uses for AJAX

  • Create form dynamically by querying server
  • Offer completions for form field entries
  • Refresh data in a region of page without page reload
  • Submit form data without new page load
  • Mashups: link site's data with external data (e.g. Google Maps)
  • Web applications: pages that behave like desktop applications

Raw AJAX

  • AJAX is best used from a framework such as jQuery or Prototype
  • We will first look at how AJAX is done in plain JavaScript
    • Cumbersome
    • With browser compatibility issues
  • jQuery makes it simple!

The XMLHttpRequest object

  • Provides the AJAX API via its methods and properties
  • This is the W3C standard
  • Microsoft actually invented AJAX
    • Based on the Microsoft.XMLHTTP object
    • Not supported by non-IE browsers
    • XMLHttpRequest not supported by IE till IE7
    • Requires workarounds if site wants to support IE6
    • Fortunately the methods are the same for both objects

A simple example

  • This example uses raw AJAX, no jQuery.
  • Loads a short biography of the individual selected from a menu.
    • Responds to change event of menu to request data from server.
    • Response data is plain text/HTML.


Example (live)   HTML   JS Code   Sample response file

Same-Origin Policy

  • As a security measure, XMLHttpRequest can only request data from the same origin as the page it runs in:
    • Same protocol
    • Same host
    • Same port
For example, if page is
http://www.dsm.fordham.edu/~janeuser/index.php,
requests may only be to http://www.dsm.fordham.edu.
See the Mozilla definition.
  • Cooperating hosts may allow access using CORS (Cross-Origin Resource Sharing)
  • This is how Google mashups do it

jQuery AJAX methods

jQuery provides a number of methods for AJAX.
  • Work around browser compatibility
  • Simplify request-response machinery
    • Automatically handle many details
  • jQuery or other framework is the way to go!

jQuery load function

  • Simplest way to load HTML content into an element:
     $(selector).load(url)
  • Notes:
    • The URL is subject to same-origin policy.
    • Not to be confused with function of same name that assigns load event handler.
    • If multiple elements match, all get copies of response text.
    • If no elements match, no AJAX call is done.
Here is the previous example done with jQuery. Note how little code is needed.
Example (live)  HTML  JS Code

Other load options

  • Additional parameters may be provided:
     $(selector).load(url,data,callback)
    • data sent to server like form data with request
      • Typically use plain object (JSON) format, e.g.
        { topic: "computers", limit: 25 }
      • Server may tailor response based on data
    • callback function is run after response was successfully inserted into the element.

What's wrong with plain text?

Why isn't plain text/HTML always adequate?
  • It is unstructured. Not easy to:
    • use only part of the content
    • give parts special treatment
  • It can be given structure using homebrew methods, but
    • these are nonstandard
    • require custom coding
  • Better idea: use XML
    • Flexible, versatile, widely used
    • Easy for PHP to generate on demand from database queries
    • Native JS as well as jQuery DOM methods work with it

What is XML?

eXtensible Markup Language
X-ML
  • Close relative of HTML
    • (Old XHTML was actually a subset.)
  • But extensible:
    • Tag names may be freely invented
    • Tag attribute names may be freely invented
  • But also stricter:
    • All tags require closing tag
    • Luckily there are XML validators available, e.g. the one provided by W3Schools.

XML format

  • Must have XML prolog as first line (see example below)
  • Rest of document must be enclosed in one tag (root element)
  • Content enclosed between opening and closing tags
  • Tags nest as usual, comments like in HTML
  • Example (from Stepp et al online slides):
    <?xml version="1.0" encoding="UTF-8"?>   <!-- XML prolog -->
    <note private="true">   <!-- root element -->
    <from>Alice Smith (alice@example.com)</from>
    <to>Robert Jones (roberto@example.com)</to>
    <subject>Tomorrow's "Birthday Bash" event!</subject>
    <message language="english">
    Hey Bob, Don't forget to call me this weekend!
    </message>
    </note>

Designing your XML document

  • Create meaningful tag names
    • Any tag names you like
    • E.g. file of music data might use title, artist, etc.
  • Choose how to best represent your data
    • Use tags for large or complex categories
    • Nest tags within tags for subcategories, e.g.:
      <name>
        <first>Alan</first>
        <last>Turing</last>
      </name>
    • Use attributes for smaller details and metadata

XML DOM traversal

  • XMLHttpRequest responseXML is a document tree like HTML DOM.
  • Some standard JavaScript DOM methods and properties work:
    • xmlDoc.getElementsByTagName("tag")
    • node.nodeName, node.nodeValue,
    • node.childNodes, node.parentNode
    • But no classes, ids, so those methods do not work.
  • However, this gets complicated in native JS.
    • Content of element stored in text node child, not element itself, e.g.:
      txt=xmlDoc.getElementsByTagName("title")[0].childNodes[0].nodeValue
    • See W3Schools tutorial on XML DOM for the gory details.
    • Preferably use jQuery methods

XML DOM methods in jQuery

Give the XML Document object the jQuery upgrade, and:
  • $(xmlDoc).find("selector") works.
    • But no .find(".class"), no .find("#id")
    • Can use CSS relational selectors like
      .find("name > first") — parent-child
      .find("message[language='english']") — element with attribute
  • jQuery get/set and DOM traversal methods work, e.g..
    • .find("selector").text()
    • .find("selector").attr("attr-name")
    • .find("selector").each(callback)

jQuery AJAX methods

  • Cannot use load function with XML response.
    • Response data is XML, not text/HTML.
    • Need to capture XML document, work with it.
  • Methods for this:
    • $.ajax(params) — the most general form
    • $.get(params) — shorthand for request type=GET
    • $.post(params) — shorthand for request type=POST
  • The params argument is a JSON object.
  • Use a callback to process the response.

The jQuery $.get() method

Use this when no need to send data with request, or only a small amount.
  • Usage (with chaining):
    $.get(url)
      .done( successFunction )
      .fail( failureFunction );
  • successFunction(data) receives the response data as argument
    • For text or HTML response, data is a plain string
    • For XML response, data is an XML document object
  • failureFunction(xhr,status,errorText) is called if request fails.
Example (live)  HTML  JS Code  Sample XML file

Sending data with request

  • The $.get() method allows an optional second argument to send data to the server via GET method.
    • But usually preferable to use POST method to send data.
  • Usage (with chaining):
    $.post(url,data)
      .done( successFunction )
      .fail( failureFunction );
  • Provide data as a JavaScript object (JSON):
    { name:value, name:value, ... }

The jQuery $.ajax() method

  • Use this for full functionality. Usage:
    $.ajax(url,settings) or
    $.ajax(settings)
    • In second form, the URL is one of the settings values
  • settings is a JavaScript Object (JSON) with many choices. Some frequently used ones are:
    • url — URL of request
    • data — (JSON) data to be sent to server with request
    • method — request method: "get" or "post"
    • headers — additional request headers to be sent with request
    • success — handler for successful response (or use .done())
    • error — handler for response failure (or use .fail())

What's wrong with XML?

  • Though powerful, XML is
    • verbose, e.g. each tag appears as opener and closer
    • cumbersome to navigate with standard DOM methods
      • more complex structure than necessary
      • though not so bad with jQuery upgrade
  • Many applications now use JSON instead
    • JavaScript object literal
      • But parseable by other languages
    • Very human-readable
    • Gives simpler access to values
      • via standard JavaScript object/array element notation

What is JSON?

Jason, by Bertel Thorvaldsen 1803 (Wikimedia)
  • JavaScript Object Notation
  • Part of JavaScript, defines object literals
    • See definition at json.org.
    • As a data-interchange format, JSON is a subset of object literal notation
      • No functions or expressions as properties
      • No comments
      • All property names and string values must quoted
        • Double quote (") only, not apostrophe (')
    • These rules strictly apply only to JSON sent from server
      • Can relax them in jQuery AJAX calls

jQuery AJAX with JSON

Same example as before, done with JSON response this time.
  • JSON response arrives in responseText
  • Convert to object using JSON.parse()
    • This is done automatically by jQuery


Example (live)  HTML  JS Code  Sample JSON file

AJAX Server

  • The examples so far used static files for response.
  • In real applications, server provides response dynamically
    • PHP or other server-side script receives request
    • Data usually fetched by database query
    • Response formatted as XML or JSON
    • Need to supply explicit content-type header
      • Content-Type: application/xml
        or
      • Content-Type: application/json

An extended example

Now these concepts will be illustrated with a larger example.

The following slides will work through development of the example below, which provides an autocompletion feature for a text box.

Example (live)

First step: the web form

We start with the easy part: creating the web form.
Filename wordsform.html.
  • The text input box and the suggestion area div need ids.
    • Input box: id="word"
    • Suggestion div: id="suggestionBox"
  • The form will simply be submitted to our generic form processor that echoes what was entered.
HTML Code

The AJAX cycle

Next, plan the AJAX request/response cycle:
  • When user types another letter in word box, provide completion
    • Use keyup event as trigger
    • Use jQuery post() to send string typed so far to server
    • Responder on server sends back list of matching words
    • Response handler puts list into suggestion box

AJAX responder

Before writing any JavaScript, write and debug PHP responder offline.
  • Receives POST data containing a string typed by user
  • Looks in word list for words starting with that string
  • Sends back word list to client
    • Use JSON format
  • Details that will be added later in development
    • Limit number of suggestions offered
    • Sort suggestions by frequency, most common first
    • Suggestions match case of word typed in box
      • lowercase, all caps, capitalize first letter

Word list

  • For the word list, use a list available online.
    • Using decent text editor with regular expression replace, process into form of PHP array definition
    • It's long, so store in an include file wordslist.php
      • A real app would use a database
    <?php // Most common 2000 words in normal English => frequency (%)
    $wordslist = array(
    "a" => 2.2996,
    "ability" => 0.0073,
    "able" => 0.0213,
      ...
    "yourself" => 0.0066,
    "youth" => 0.0081
    );
    ?>

Responder: find matches


Name it: wordsmatch.php
  • First part of responder: create a list $suggestions of words matching the prefix, with their frequencies.
  • Don't worry about limit or order yet.
  • Use a simple linear search through the $wordslist array.
    • Inefficient, but list is not that long

Responder: find matches

<?php
include('wordslist.php');       /* Read in the array of words */
        /* Get the prefix that was sent by either GET or POST */
if( $_REQUEST && isset($_REQUEST['prefix']) ) {
  $prefix = $_REQUEST['prefix'];
  $prefix_len = strlen($prefix);
  $suggestions = array(); /* matches will go here */
  foreach( $wordslist as $word => $frequency ) {
    /* Case-insensitive comparison of first $prefix_len chars */
    if( strncasecmp($word, $prefix, $prefix_len) == 0 ) {
      $suggword = $word;
      $suggestions[$suggword] = $frequency; // save matching entry
    }
  }
...

Responder: generate JSON

  • Second part of responder: spit out list of matches in JSON form
  • Don't worry about limit or ordering by frequency yet.
  • Requires Content-type header
  • JSON object is array of objects representing matches
  • Each object has two properties:
    • word = a matching word from list
    • freq = frequency of that word
  • Example, for prefix lea:
    [
    {"word":"least","freq":0.0338},
    {"word":"leave","freq":0.0202},
     ...
    {"word":"leaders","freq":0.0105}
      Note no comma on last line
    ]

Responder: generate JSON

...
  if( count($suggestions) > 0 ) {
    // Spew out the JSON containing the suggestions.
    // Important: header needed to tell response handler this is JSON
    header("Content-Type: application/json");
    print "[\n";      /* start of JSON array */
    $num_words_out = 0;
    foreach( $suggestions as $suggword => $freq ) {
      // avoid printing comma after last element (which may be first)
      if( $num_words_out++ != 0 ) {
        print ",\n";
      }
      print "{\"word\":\"$suggword\",\"freq\":$freq}";
    }
    print "\n]\n";      /* close JSON array */
  }
}

Debugging

  • Debugging AJAX is tricky
    • PHP syntax errors do not appear in web page, even if enabled
    • They do not appear in browser web console
    • AJAX failure handler message usually not helpful
      • E.g. “error: Internal Server Error”
    • Syntax errors do appear in server error_log
    • May also be seen in network debug console, but...
    • Offline debugging is often better in early stages
      • Online debugging will be covered later

Offline Debugging

  • Run PHP CLI (command line interpreter) in a shell
    • Mac or Linux: open a terminal, or ssh to erdos
      • php is installed on erdos and LL812 lab machines
    • Windows: use Run (on Start menu or Accessories menu)
      • Need to set path for executable. For XAMPP:
        path=%path%;c:\xampp\php
    • Change directory to web site, e.g.:
      • cd c:\xampp\htdocs
    • Then execute the script:
      php wordsmatch.php
  • Any syntax errors appear in terminal window.
  • No errors, no output (for our responder) since $_REQUEST not set

Further debugging

  • Once all syntax errors are fixed, need to verify output
  • This requires setting $_REQUEST
  • Easy way: just edit the code temporarily
    <?php
    include('wordslist.php');   /* Read in the array of words */
    $_REQUEST["prefix"]="lea"; // FOR DEBUGGING
    ...
  • Here's a way to do it without touching the code:
    php -B "$_REQUEST['prefix']='lea';" -F wordsmatch.php
    • Then hit Enter, followed by EOF: ctrl-Z (Windows) or ctrl-D (Linux, Mac)
    • This executes the -B parameter before anything else
    • Then executes the -F file once per input line
      • The Enter causes it to execute once
      • Then EOF to quit

Command-line debugging

Example:
php -B '$_REQUEST["prefix"]="lea";' -F wordsmatch.php
(Enter)
[
{"word":"least","freq":0.0338},
{"word":"leave","freq":0.0202},
{"word":"lead","freq":0.0127},
{"word":"learned","freq":0.0115},
{"word":"leaders","freq":0.0105}
]
(ctrl-D)
(Header does not appear — this is normal.)

AJAX code

  • Now we write the jQuery-based AJAX code.
    • Filename wordsform.js
  • Recall the AJAX cycle for autocompletion:
    • Use keyup event on word box as trigger
    • Use jQuery post() to send string typed so far to server
    • Responder on server sends back list of matching words
    • Response handler puts list into suggestion box
  • Don't worry yet about allowing user to select a suggestion:
    • just display the list

Responding to keyup event

  • Attach event handler to word box
    $(document).ready( function() {
      $("#word").keyup( offerCompletions );
    });
  • offerCompletions will initiate the AJAX call.

AJAX initiator

  • If word box nonempty, POST contents to responder
  • Assign success and failure handlers
function offerCompletions()
{
  var prefix = $("#word").val(); // letters typed so far
  if( prefix.length > 0 ) {
    $.post("wordsmatch.php",
            {"prefix":prefix}
          )
    .done( handleCompletions )
    .fail( ajaxFailure );
  }
}

AJAX response handler

  • Remember response (words argument) is a JSON object
function handleCompletions( words )
{
  $("#suggestionBox").html(""); // clear the suggestion box
  for( i in words ) {
    var suggWord = $("<span>").text( words[i].word );
    var suggFreq = $("<span>").text( words[i].freq );

    $("#suggestionBox").append(suggWord)
          .append(" ")   // add a space between word and freq
          .append(suggFreq)
          .append("<br/>");
  }
}

AJAX failure handler

  • Failure handler is straightforward:
function ajaxFailure( xhr, status, exception )
{
  $("#suggestionBox").html( "Ajax failure: " + status + ": " + exception );
}

Online debugging

  • Application is now ready for online debugging.
  • Use browser debugger, e.g. Firefox Developer toolkit.
  • Network tab shows AJAX request and response
    • Examine headers to see they are correct
    • View response data: if OK it appears as parsed JSON object
Example of JSON syntax error in Firefox Network debugger. (Error is a comma after last object.)

Suppressing autocomplete

While testing, we discover that some browsers (e.g. Firefox) want to provide their own autocomplete. A little searching turns up the solution:
  • Attribute autocomplete should be set to "off"
  • We can do this in either of two ways:
    • Add attribute to <input> tag.
    • Do it with jQuery. We choose this method.
    Update wordsform.js:
$(document).ready( function() {
  $("#word").keyup( offerCompletions )
      .attr("autocomplete","off");
});

Responder: sorting results

  • Now add sorting of the words from most to least common
  • There is a function arsort(array) that does this.
    • Sorts associative array by values
    • The r in its name means reverse order
  • Add the highlighted code to wordsmatch.php:
...
  if( count($suggestions) > 0 ) {
    arsort($suggestions); /* sort list in descending order by frequency */
    // Spew out the JSON containing the suggestions.
    // Important: header needed to tell response handler this is JSON
    header("Content-Type: application/json");
...

Limiting the results

  • To prevent list of suggestions from being too long, limit the number.
    • Default limit 5
    • Allow client to change value in request data variable limit
  • Update wordsmatch.php:
    ...
    if( $_REQUEST && isset($_REQUEST['prefix']) ) {
      $prefix = $_REQUEST['prefix'];
      $prefix_len = strlen($prefix);
      if( isset($_REQUEST['limit']) ) {
        $max_words_out = $_REQUEST['limit'];
      }
      else {
        $max_words_out = 5;    /* default */
      }
    ...

Limiting the results

  • Using the limit: update wordsmatch.php:
    ...
        print "[\n";      /* start of JSON array */
        foreach( $suggestions as $suggword => $freq ) {
          if( $num_words_out >= $max_words_out ) {
            break;
          }
          // avoid printing comma after last element (which may be first)
          if( $num_words_out++ != 0 ) {
            print ",\n";
          }
          print "{\"word\":\"$suggword\",\"freq\":$freq}";
        }
        print "\n]\n";     /* close JSON array */
      }
    ...

Limiting the results

  • Add limit variable to data POSTed from jQuery
  • Update wordsform.js:
function offerCompletions()
{
  var prefix = $("#word").val(); // letters typed so far
  if( prefix.length > 0 ) {
    $.post("wordsmatch.php",
            {"prefix":prefix, limit:5}
          )
    .done( handleCompletions )
    .fail( ajaxFailure );
  }
}

Matching case

  • Allow user to type words in upper, lower, or mixed case
  • Match case style in response
  • Function to identify case style on next slide
    • Returns 'lc' for lowercase
    • Returns 'uc' for all UPPERCASE
    • Returns 'cap' for Capitalized
  • Update wordsmatch.php: add the following function at end

Case style function

function casetype($s) {
  if( strtoupper($s) == $s ) {   // all letters are capitals
    $letterCount = strlen(preg_replace('/[^a-zA-Z]/','',$s));
   // no letters: call it lowercase
    if( $letterCount == 0 ) { return 'lc'; }
   // one letter: call it Capitalized
    else if( $letterCount == 1 ) { return 'cap'; }
   // >1 letter: it is ALL CAPS
    else { return 'uc'; }
  }
  else {     // some letters not capitals
    $firstLetter = substr($s,0,1);
    if( strtoupper($firstLetter) == $firstLetter ) { return 'cap'; }
    else { return 'lc'; }
  }
}

Setting case style of matches

  • Get the case style of prefix. Update wordsmatch.php:
...
if( $_REQUEST && isset($_REQUEST['prefix']) ) {
  $prefix = $_REQUEST['prefix'];
  $prefix_len = strlen($prefix);
  $prefix_casetype = casetype($prefix);
...

Setting case style of matches

    /* Case-insensitive comparison of first $prefix_len chars */
    if( strncasecmp($word, $prefix, $prefix_len) == 0 ) {
      // convert word text to match the case type of the prefix
      if( $prefix_casetype == 'uc' ) { /* UPPERCASE */
        $suggword = strtoupper($word);
      }
      else if( $prefix_casetype == 'cap' ) { /* Capitalized */
        $suggword = ucfirst($word);
      }
      else {           /* lowercase */
        $suggword = $word;
      }
      $suggestions[$suggword] = $frequency; // save matching entry

Selecting completions

  • Almost done. Final step is to let user click on suggested words
    • Selected word replaces contents of word box.
    • Include highlighting of suggestion on hover.
    Update wordsform.js:
    function handleCompletions( words )
    {
      $("#suggestionBox").html(""); // clear the suggestion box
      for( i in words ) {
        var suggWord = $("<span>").text( words[i].word );
        var suggFreq = $("<span>").text( words[i].freq );
        suggWord.addClass( "offerCompletion" ); // adds hover effects
        suggWord.click( acceptCompletion ); // replace word in box by choice
    ...

Accepting completions

  • Add function to wordsform.js
    • Gets text of clicked word and copies it to word box
function acceptCompletion()
{
  $("#word").val( $(this).text() );
}

Selection styling

  • Provides visual feedback when selecting completion from list
    • Style the words to change color on hover
    • Provide hand pointer instead of default I-beam
    • Uses offerCompletion class added to word by jQuery
  • Stylesheet wordsform.css:
.offerCompletion:hover {
  background-color : #aaffff;
  color : red;
  cursor : pointer;
}

PHP json_encode function

PHP provides a function json_encode(array)
  • Spares you the trouble of printing JSON.
  • array is sequential or associative array.
  • Returns string containing its JSON representation.
    • sequential array as JSON array [ ... ]
    • associative array as JSON object { ... }
Here is the PHP responder rewritten to use it:  PHP Code