Web application data communication¶

Sync vs. Asynchronous model¶

By default, the XMLHttpRequest https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest uses asynchronous operation, there is a “async” argument where you can set to “false” if you want synchronous operation.

Same Origin and Cross origin¶

The same origin policy https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy is a retriction for web security.

Here are some examples of resources which may be embedded cross-origin:

1. JavaScript with <script src=”...”></script>. Error messages for syntax errors are only available for same-origin scripts.
2. CSS with <link rel=”stylesheet” href=”...”>. Due to the relaxed syntax rules of CSS, cross-origin CSS requires a correct Content-Type header. Restrictions vary by browser: IE, Firefox, Chrome, Safari (scroll down to CVE-2010-0051) and Opera.
3. Images with <img>. Supported image formats include PNG, JPEG, GIF, BMP, SVG, ...
4. Media files with <video> and <audio>.
5. Plug-ins with <object>, <embed> and <applet>.
6. Fonts with @font-face. Some browsers allow cross-origin fonts, others require same-origin fonts.
7. Anything with <frame> and <iframe>. A site can use the X-Frame-Options header to prevent this form of cross-origin interaction.

So, to trick data loading from a cross-origin site, you could load an external script that defines your data as a global variable. (JSONP is a common example of this.)

By default, most browsers do not allow cross-domain requests. To enable cross-domain requests, have the server set the header Access-Control-Allow-Origin: *. For more details, see the W3C recommendation on Cross-Origin Resource Sharing. Or, setup a server-side proxy – see this article https://developer.yahoo.com/javascript/howto-proxy.html

Open or Force download a data file from server use html(client) and php(server)¶

1. Open data in browser window:

html:

<a href="files/csv/example/example.csv">
   Click here to download an example of the "CSV" file
</a>

1. Force download:

html::

<a href=”files/csv/example/csv.php”>

Click here to download an example of the “CSV” file

</a>

csv.php:

header('Content-Type: application/csv');
header('Content-Disposition: attachment; filename=example.csv');
header('Pragma: no-cache');
readfile("/path/to/yourfile.csv");

Client: Retrieve data from server use javascript¶

Here we use d3.json() and $.ajax() function as example:

1. request a csv file ::

   d3.json(“/path/to/file.csv”, function(error, data){...});

2. request a server url ::

   d3.json(“/myweb/totalQtyService.do”, callback); d3.json(“php/data2.php”, function(error, data) {...}); $.ajax({url: “http://www.derkholm.net:8080/das/hg19comp/sequence?segment=22:30000000,30000300”,

   success: function(result, status, xhr) {

   console.log(“ajax call success”, result, status, xhr); $(“#content”).val(result);

   }

   });

3. request jsonp

   $.getJSON("http://api.flickr.com/services/feeds/photos_public.gne?jsoncallback=?",
         {
             tags: "jquery",
             tagmode: "any",
             format: "json"
         },
         function(data) {...});
   

4. post data to server url

   jquery:

   $.ajax({
      type: "POST",
      url: url,
   
      // parameter here
      data : {year: "2012", customer: "type1“},
      success: function(json){...},
      error:function (request, err, ex) {...}
      });
   

   d3:

   var my_request = d3.xhr(url);
   my_request.post({year: "2012", customer: "type1"}, function(d){
      console.log("got response, now parse it")
   });
   

Server: RESTful architecture¶

REST http://rest.elkstein.org/ is an architecture style for designing networked applications. The idea is that, rather than using complex mechanisms such as CORBA, RPC or SOAP to connect between machines, simple HTTP is used to make calls between machines.

In many ways, the World Wide Web itself, based on HTTP, can be viewed as a REST-based architecture.

RESTful applications use HTTP requests to post data (create and/or update), read data (e.g., make queries), and delete data. Thus, REST uses HTTP for all four CRUD (Create/Read/Update/Delete) operations.

Server side programming : connect with database¶

1. PHP ::

   <?php $username = “homedbuser”; $password = “homedbuser”; $host = “localhost”; $database=”homedb”;

   $server = mysql_connect($host, $username, $password); $connection = mysql_select_db($database, $server);

   $myquery = ” SELECT date, close FROM data2 ”; $query = mysql_query($myquery);

   if ( ! $query ) { echo mysql_error(); die; }

   $data = array();

   for ($x = 0; $x < mysql_num_rows($query); $x++) { $data[] = mysql_fetch_assoc($query); }

   echo json_encode($data);

   mysql_close($server); ?>

2. Node : Use Node, Express and MongoDB to build your server side appliation is very popular, search online you can find many examples and tutorials, such as this one http://scotch.io/tutorials/javascript/build-a-restful-api-using-node-and-express-4 and this one http://blog.ijasoneverett.com/2013/03/a-sample-app-with-node-js-express-and-mongodb-part-1/

Example data service apis¶

1. Twitter dev api
2. Google map api
3. Flickr api

For more api info, check this list : http://www.programmableweb.com/apis/directory

To call the api functions, mostly you need authentication, which requires an apikey and secret pair. But there are also a few public api functions you can call without the apikey authentication. Check the api documentation for details. For example the Twitter api documentation is here https://dev.twitter.com/docs Flickr documentation https://www.flickr.com/services/api/auth.howto.web.html Tutorials : http://mashupguide.net/1.0/html/ch06s05.xhtml Some library wrappers: https://github.com/Pomax/node-flickrapi

You can apply multiple apikeys for multiple applications you are developing.

A Mashup Web Application¶

There is this buzzword ‘mashup’ which is popular a few years ago. Come across this online book http://mashupguide.net/1.0/html/ch01s02.xhtml help you understand what it counts for. Bacically it means somehow mix/combine part of data services from multiple data sources to create a new form of data perception and user experience. We don’t use this word too often these days anymore, because it has became a normal web application design pattern. For example, Google map displays lots of information on top of its map service, it became a big mashup itself.

There is another example called GMiF http://mashupguide.net/1.0/html/ch01s03.xhtml in the book, which the author break down to explain.

Housingmaps.com use server-side code to do data integration (retrieve and organize data from craiglist for googlemap client to display). The GMiF script is client-side (inject javascript to flickr web page to collect geo-encode data and save as KML format for googlmap to display). The LibraryLookUp bookmarklet example http://mashupguide.net/1.0/html/ch01s04.xhtml is also a client-side “hackish” example. When user invoke the bookmark, it executes a small piece of javascript, which send a search reqeust to your library’s online search api, and open a new window to display the response from the search.

A web service which calls web APIs for you¶

There are soooo many web apis out there, each provides interesting data, that’s so much fun – until you need to deal with 100 of them in your development work! :)

So you’d like a system, such as a database system, to ease the data access, validation, joining, filtering, paging, etc. This is the design principle of YQL.

https://developer.yahoo.com/yql/guide/feature_list.html

http://code.tutsplus.com/tutorials/an-api-for-the-web-learning-yql–net-8769

YQL