Apache Thrift with Java quickstart

Apache Thrift is a RPC framework founded by facebook and now it is an Apache project. Thrift lets you define data types and service interfaces in a language neutral definition file. That definition file is used as the input for the compiler to generate code for building RPC clients and servers that communicate over different programming languages. You can refer Thrift white paper also.

According to the official web site Apache Thrift is a,

software framework, for scalable cross-language services development, combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.

Image courtesy wikipedia

Installing Apache Thrift in Windows

Installation Thrift can be a tiresome process. But for windows the compiler is available as a prebuilt exe. Download thrift.exe and add it into your environment variables.

Writing Thrift definition file (.thrift file)

Writing the Thrift definition file becomes really easy once you get used to it. I found this tutorial quite useful to begin with.

Example definition file (add.thrift)

namespace java com.eviac.blog.samples.thrift.server  // defines the namespace 

typedef i32 int  //typedefs to get convenient names for your types

service AdditionService {  // defines the service to add two numbers
        int add(1:int n1, 2:int n2), //defines a method
}

Compiling Thrift definition file

To compile the .thrift file use the following command.

 
thrift --gen <language> <Thrift filename>

For my example the command is,

 
thrift --gen java add.thrift

After performing the command, inside gen-java directory you'll find the source codes which is useful for building RPC clients and server. In my example it will create a java code called AdditionService.java

Writing a service handler

Service handler class is required to implement the AdditionService.Iface interface.

Example service handler (AdditionServiceHandler.java)

 
package com.eviac.blog.samples.thrift.server;

import org.apache.thrift.TException;

public class AdditionServiceHandler implements AdditionService.Iface {

 @Override
 public int add(int n1, int n2) throws TException {
  return n1 + n2;
 }

}

Writing a simple server

Following is an example code to initiate a simple thrift server. To enable the multithreaded server uncomment the commented parts of the example code.

Example server (MyServer.java)

package com.eviac.blog.samples.thrift.server;

import org.apache.thrift.transport.TServerSocket;
import org.apache.thrift.transport.TServerTransport;
import org.apache.thrift.server.TServer;
import org.apache.thrift.server.TServer.Args;
import org.apache.thrift.server.TSimpleServer;

public class MyServer {

 public static void StartsimpleServer(AdditionService.Processor<AdditionServiceHandler> processor) {
  try {
   TServerTransport serverTransport = new TServerSocket(9090);
   TServer server = new TSimpleServer(
     new Args(serverTransport).processor(processor));

   // Use this for a multithreaded server
   // TServer server = new TThreadPoolServer(new
   // TThreadPoolServer.Args(serverTransport).processor(processor));

   System.out.println("Starting the simple server...");
   server.serve();
  } catch (Exception e) {
   e.printStackTrace();
  }
 }
 
 public static void main(String[] args) {
  StartsimpleServer(new AdditionService.Processor<AdditionServiceHandler>(new AdditionServiceHandler()));
 }

}

Writing the client

Following is an example java client code which consumes the service provided by AdditionService.

Example client code (AdditionClient.java)

package com.eviac.blog.samples.thrift.client;

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;

public class AdditionClient {

 public static void main(String[] args) {

  try {
   TTransport transport;

   transport = new TSocket("localhost", 9090);
   transport.open();

   TProtocol protocol = new TBinaryProtocol(transport);
   AdditionService.Client client = new AdditionService.Client(protocol);

   System.out.println(client.add(100, 200));

   transport.close();
  } catch (TTransportException e) {
   e.printStackTrace();
  } catch (TException x) {
   x.printStackTrace();
  }
 }

}

Run the server code(MyServer.java). It should output following and will listen to the requests.

Starting the simple server...

Then run the client code(AdditionClient.java). It should output following.

300

MongoDB with Java

NoSQL databases due to their scalability are becoming increasingly popular. When used appropriately NoSQL databases can offer real benefits. MongoDB is such a highly scalable opensource NoSQL database written in C++.

1. Installing MongoDB

Without much of a trouble you can install MongoDB using the instructions given in the official MongoDB site, according to whatever the OS you are using.

2. Starting the MongoDB server

This is quite simple. Run the mongod.exe file inside bin folder(I am using windows OS here) to start the MongoDB server.

By default the server will start on port 27017 and the data will be stored at /data/db directory which you'll have to create during the installing process.

3. Starting MongoDB shell

You can start the MongoBD shell by running the mongo.exe file.

4. Creating a database with MongoDB

To create a database named "company" using MongoDB type the following on MongoDB shell

 use company 

Mind that MangoDB will not create a database until you save something inside it.

Use following command to view the available databases and that will show you that "company" database hasn't been created yet.

 show dbs; 

5. Saving data in MongoDB

Use following commands to save employee data to a collection called employees

 
employee = {name : "A", no : 1} 
db.employees.save(employee) 

To view the data inside the collection use following command,

 
db.users.find(); 

Do it with Java :)

Following is a simple Java code which is doing the same thing we did above. You can get the mongo-java driver from here.

Just go through the code, it's very simple, hopefully you'll get the idea.

 
package com.eviac.blog.mongo;

import java.net.UnknownHostException;

import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBCursor;
import com.mongodb.Mongo;
import com.mongodb.MongoException;

public class MongoDBClient {

 public static void main(String[] args) {

  try {

   Mongo mongo = new Mongo("localhost", 27017);

   DB db = mongo.getDB("company");

   DBCollection collection = db.getCollection("employees");

   BasicDBObject employee = new BasicDBObject();
   employee.put("name", "Hannah");
   employee.put("no", 2);

   collection.insert(employee);

   BasicDBObject searchEmployee = new BasicDBObject();
   searchEmployee.put("no", 2);

   DBCursor cursor = collection.find(searchEmployee);

   while (cursor.hasNext()) {
    System.out.println(cursor.next());
   }

   System.out.println("The Search Query has Executed!");

  } catch (UnknownHostException e) {
   e.printStackTrace();
  } catch (MongoException e) {
   e.printStackTrace();
  }

 }

}
 

Result

 
{ "_id" : { "$oid" : "4fec74dc907cbe9445fd2d70"} , "name" : "Hannah" , "no" : 2}
The Search Query has Executed! 

MPI - Message Passing Interface

MPI is a message-passing library specification. Interface specifications have been defined for C/C++ and Fortran programs[1]. Provided examples use language C and LAM/MPI. LAM/MPI is a high quality implementation of the Message Passing Interface (MPI) Standard.

Example1: demo.c

#include "mpi.h"
#include <stdio.h>

   int main(int argc,char *argv[])
   {
   int  numtasks, rank, rc; 

   MPI_Init(&argc,&argv);
   
   MPI_Comm_size(MPI_COMM_WORLD,&numtasks);
   MPI_Comm_rank(MPI_COMM_WORLD,&rank);
   printf ("Number of tasks= %d My rank= %d\n", numtasks,rank);  

   MPI_Finalize();
   }

Commands

lamboot
mpicc -o demo demo.c
mpirun -np <number of processes> demo

Result

The next example is the in class assignment we got for the course module 'High performance computing', to design matrix multiplication using MPI.

Matrix size N is divisible by the number of workers (eg. if matrix size is 4 and the number of workers is 4, each worker will get 1 row from matrix A). Master sends each worker equal number of rows from matrix A, full matrix B and an offset to track down the position of rows. Each worker receives the information master sends and completes the matrix multiplication for the relevant rows and creates the relevant rows of result matrix C and sends it along with the offset to track down the position of the rows. Master receives all the rows of the result matrix C from every worker and the result matrix is completed.

Example2

/**********************************************************************
 * MPI-based matrix multiplication AxB=C  
 *********************************************************************/


#include <stdio.h>
#include "mpi.h"
#define N    4        /* number of rows and columns in matrix */

MPI_Status status;

double a[N][N],b[N][N],c[N][N];  
       
main(int argc, char **argv) 
{
  int numtasks,taskid,numworkers,source,dest,rows,offset,i,j,k;

  struct timeval start, stop;
  
  MPI_Init(&argc, &argv);
  MPI_Comm_rank(MPI_COMM_WORLD, &taskid);
  MPI_Comm_size(MPI_COMM_WORLD, &numtasks);

  numworkers = numtasks-1;

  /*---------------------------- master ----------------------------*/
  if (taskid == 0) {
    for (i=0; i<N; i++) {
      for (j=0; j<N; j++) {   
 a[i][j]= 1.0;
 b[i][j]= 2.0;
      }
    }

    gettimeofday(&start, 0);

    /* send matrix data to the worker tasks */
    rows = N/numworkers;
    offset = 0;
    
    for (dest=1; dest<=numworkers; dest++) 
    {       
      MPI_Send(&offset, 1, MPI_INT, dest, 1, MPI_COMM_WORLD);
      MPI_Send(&rows, 1, MPI_INT, dest, 1, MPI_COMM_WORLD);
      MPI_Send(&a[offset][0], rows*N, MPI_DOUBLE,dest,1, MPI_COMM_WORLD);
      MPI_Send(&b, N*N, MPI_DOUBLE, dest, 1, MPI_COMM_WORLD);
      offset = offset + rows;
    }

    /* wait for results from all worker tasks */
    for (i=1; i<=numworkers; i++) 
    {   
      source = i;
      MPI_Recv(&offset, 1, MPI_INT, source, 2, MPI_COMM_WORLD, &status);
      MPI_Recv(&rows, 1, MPI_INT, source, 2, MPI_COMM_WORLD, &status);
      MPI_Recv(&c[offset][0], rows*N, MPI_DOUBLE, source, 2, MPI_COMM_WORLD, &status);
    }

    gettimeofday(&stop, 0);

    printf("Here is the result matrix:\n");
    for (i=0; i<N; i++) { 
      for (j=0; j<N; j++) 
 printf("%6.2f   ", c[i][j]);
      printf ("\n");
    }
  
    fprintf(stdout,"Time = %.6f\n\n",
         (stop.tv_sec+stop.tv_usec*1e-6)-(start.tv_sec+start.tv_usec*1e-6));

  } 

  /*---------------------------- worker----------------------------*/
  if (taskid > 0) {
    source = 0;
    MPI_Recv(&offset, 1, MPI_INT, source, 1, MPI_COMM_WORLD, &status);
    MPI_Recv(&rows, 1, MPI_INT, source, 1, MPI_COMM_WORLD, &status);
    MPI_Recv(&a, rows*N, MPI_DOUBLE, source, 1, MPI_COMM_WORLD, &status);
    MPI_Recv(&b, N*N, MPI_DOUBLE, source, 1, MPI_COMM_WORLD, &status);
  
    /* Matrix multiplication */
    for (k=0; k<N; k++)
      for (i=0; i<rows; i++) {
        c[i][k] = 0.0;
        for (j=0; j<N; j++)
   c[i][k] = c[i][k] + a[i][j] * b[j][k];
      }


    MPI_Send(&offset, 1, MPI_INT, 0, 2, MPI_COMM_WORLD);
    MPI_Send(&rows, 1, MPI_INT, 0, 2, MPI_COMM_WORLD);
    MPI_Send(&c, rows*N, MPI_DOUBLE, 0, 2, MPI_COMM_WORLD);
  }  
    
  MPI_Finalize();
} 

Result

As the part 2 of the assignment we were suppose to optimize the code so it will fulfill following requirements,

• The matrix size should be defined using #define N
• The type of elements in the matrix should be ``float''
• The matrices can be square (e.g., NXN) but N may not be exactly divisible by the number of processors
• Should check the correctness of computations by printing the result matrix. May use #define PRINT to enclose the code segments for printing
• Should find the speedup of the code (e.g., for N=100, 200, etc.) compared with a sequential matrix multiplication code.

Example 3: Solution

/**********************************************************************
 * MPI-based matrix multiplication AxB=C  
 *********************************************************************/


#include <stdio.h>
#include <sys/time.h>
#include "mpi.h"
#define N    500        /* number of rows and columns in matrix */

MPI_Status status;

float a[N][N],b[N][N],c[N][N];  
       
main(int argc, char **argv) 
{
  int numtasks,taskid,numworkers,source,dest,rows,offset,remain,i,j,k;
  struct timeval start, stop;
  MPI_Init(&argc, &argv);
  MPI_Comm_rank(MPI_COMM_WORLD, &taskid);
  MPI_Comm_size(MPI_COMM_WORLD, &numtasks);

  numworkers = numtasks-1;

  /*---------------------------- master ----------------------------*/
  if (taskid == 0) {
    for (i=0; i<N; i++) {
      for (j=0; j<N; j++) {   
        a[i][j]= 1.0;
        b[i][j]= 2.0;
      }
    }

#ifdef PRINT
        /* print matrices */
        printf("Matrix A:\n");           
        for (i=0; i<N; i++){      
           for (j=0; j<N; j++)  
              printf("%.3f\t",a[i][j]); 
           printf("\n");  
        }        

        printf("Matrix B:\n");       
        for (i=0; i<N; i++){  
           for (j=0; j<N; j++)
              printf("%.3f\t",b[i][j]);
           printf("\n");
        }   

#endif

    gettimeofday(&start, 0);

    /* send matrix data to the worker tasks */
    if (N <= numworkers)
    {
 rows = 1;  
    }
    else
    {
      if (N%numworkers!=0) // Not divisible by numworkers
      {        
 rows = N/numworkers+1;
        remain = N%numworkers;
      }
      else
      {
        rows = N/numworkers;  
      }
    }
    offset = 0;
    
    for (dest=1; dest<=numworkers; dest++, remain--) 
    {       
      MPI_Send(&offset, 1, MPI_INT, dest, 1, MPI_COMM_WORLD);
      MPI_Send(&rows, 1, MPI_INT, dest, 1, MPI_COMM_WORLD);
      MPI_Send(&a[offset][0], rows*N, MPI_FLOAT,dest,1, MPI_COMM_WORLD);
      MPI_Send(&b, N*N, MPI_FLOAT, dest, 1, MPI_COMM_WORLD);
      offset = offset + rows;
      if(remain==1) rows-=1;
    }

    /* wait for results from all worker tasks */
    for (i=1; i<=numworkers; i++) 
    {   
      source = i;
      MPI_Recv(&offset, 1, MPI_INT, source, 2, MPI_COMM_WORLD, &status);
      MPI_Recv(&rows, 1, MPI_INT, source, 2, MPI_COMM_WORLD, &status);
      MPI_Recv(&c[offset][0], rows*N, MPI_FLOAT, source, 2, MPI_COMM_WORLD, &status);
    }

    gettimeofday(&stop, 0);
#ifdef PRINT
    printf("Here is the result matrix:\n");
    for (i=0; i<N; i++) { 
      for (j=0; j<N; j++) 
 printf("%6.2f   ", c[i][j]);
      printf ("\n");
    }
#endif  
    fprintf(stdout,"Time = %.6f\n\n",
         (stop.tv_sec+stop.tv_usec*1e-6)-(start.tv_sec+start.tv_usec*1e-6));

  } 

  /*---------------------------- worker----------------------------*/
  if (taskid > 0) {
    source = 0;
    MPI_Recv(&offset, 1, MPI_INT, source, 1, MPI_COMM_WORLD, &status);
    MPI_Recv(&rows, 1, MPI_INT, source, 1, MPI_COMM_WORLD, &status);
    MPI_Recv(&a, rows*N, MPI_FLOAT, source, 1, MPI_COMM_WORLD, &status);
    MPI_Recv(&b, N*N, MPI_FLOAT, source, 1, MPI_COMM_WORLD, &status);
  
    /* Matrix multiplication */
    for (k=0; k<N; k++)
      for (i=0; i<rows; i++) {
        c[i][k] = 0.0;
        for (j=0; j<N; j++)
   c[i][k] = c[i][k] + a[i][j] * b[j][k];
      }


    MPI_Send(&offset, 1, MPI_INT, 0, 2, MPI_COMM_WORLD);
    MPI_Send(&rows, 1, MPI_INT, 0, 2, MPI_COMM_WORLD);
    MPI_Send(&c, rows*N, MPI_FLOAT, 0, 2, MPI_COMM_WORLD);
  }  
    
  MPI_Finalize();
}
 

Example 4: Sequential code

/* Matrix Multiplication */

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <sys/time.h>
#include <assert.h>

#define RANDLIMIT 5 /* Magnitude limit of generated randno.*/
#define N  500   /* Matrix Size */
#define NUMLIMIT 70.0

float a[N][N];
float b[N][N];
float c[N][N];

int main(int argc, char *argv[])
{
    struct timeval start, stop; 
    int i,j,k;

 
    /* generate mxs  */
    for (i=0; i<N; i++)
        for (j=0; j<N; j++) {
            a[i][j] = 1+(int) (NUMLIMIT*rand()/(RAND_MAX+1.0)); 
            /*a[i][j] = 1.0; 
            b[i][j] = 2.0;*/
            b[i][j] = (double) (rand() % RANDLIMIT);
            /*c[i][j] = 0.0;*/
        }

#ifdef PRINT
        /* print matrices */
        printf("Matrix A:\n");           
        for (i=0; i<N; i++){      
           for (j=0; j<N; j++)  
              printf("%.3f\t",a[i][j]); 
           printf("\n");  
        }        

        printf("Matrix B:\n");       
        for (i=0; i<N; i++){  
           for (j=0; j<N; j++)
              printf("%.3f\t",b[i][j]);
           printf("\n");
        }   

        printf("Matrix C:\n");       
        for (i=0; i<N; i++){  
           for (j=0; j<N; j++)
              printf("%.3f\t",c[i][j]);
           printf("\n");
        }   
#endif

    gettimeofday(&start, 0);

    for (i=0; i<N; i++) {
 for (j=0; j<N; j++) {
           c[i][j] = 0.0;
    for (k=0; k<N; k++)
              c[i][j] = c[i][j] + a[i][k]*b[k][j]; /* Working;standard way */
            /*c[j][i] = c[j][i] + a[j][k]*b[k][i];*/ /* Working; Makes C column by col */
        } /* end j loop */
    }

    gettimeofday(&stop, 0);

#ifdef PRINT
    /* print results*/
    printf("Answer c:\n");
    for (i=0; i<N; i++){
        for (j=0; j<N; j++) 
           printf("%.3f\t",c[i][j]);
        printf("\n");
    }
#endif

    fprintf(stdout,"Time = %.6f\n\n",
         (stop.tv_sec+stop.tv_usec*1e-6)-(start.tv_sec+start.tv_usec*1e-6));
    return(0);
}

Result

Examining the results we can clearly see that sequential program takes more time for matrix multiplication than the parallel program when the size of matrix is large.

[1] http://en.wikipedia.org/wiki/Message_Passing_Interface

Using Drupal menu system callback mapping to retrieve JSON or XML data

In my previous post I talked about Drupal menu system callback mapping. Today I'm going to show you how Drupal menu system callback mapping can be used to retrive JSON or XML data.

The way to accomplish the task is to register a path for a menu item with an optional argument. If you pass that optional argument along with the request URL callback function will return JSON data else it will return XML data. In that way callback function will serve both data formats.

Following is a coding example of above mentioned scenario.

<?php
//hook_perm implementation
function example_perm() {
    return array(
        'access city'
    );
}

//hook_menu implementation
function example_menu() {
    $items = array();
 
    //menu item 
    $items['full_city_list'] = array(
        'title' => t('City List JSON'
        ), 'page callback' => 'full_city_list', 'page arguments' => array(
            1
        ), 'access arguments' => array(
            'access city'
        ), 'type' => MENU_CALLBACK);   

    return $items;
}

function full_city_list() {
 $jsonParam=arg(1);//optional argument
 $result = db_query("SELECT  tid,name FROM term_data  WHERE  vid=2 order by name ASC");
 
 if(strcmp($jsonParam, "json") == 0){
    $response = array();
    
    while ($city_data = db_fetch_object($result)) {
        $response[$city_data->name]['city_name'] = $city_data->name;
        $response[$city_data->name]['tid'] = $city_data->tid;
    }

    if (!empty($response)) {
        //encodes city data as json  
        $jsonstrCityList = json_encode($response);
        echo $jsonstrCityList;
    } else {
        return false;
    }
 }elseif(strcmp($jsonParam, "") == 0){
 //sending city data as xml
 $dom = new DOMDocument("1.0");
    $root_elm = $dom->createElement("markers");
    $root = $dom->appendChild($root_elm);    

    while ($city_data = db_fetch_object($result)) {

        $marker_elm = $dom->createElement("marker");
        $marker_node = $root->appendChild($marker_elm);

        $marker_node->setAttribute("name", $city_data->name);
        $marker_node->setAttribute("tid", $city_data->tid);
    }
    header("Content-type: text/xml");
    echo $dom->saveXML();
 }
}
?>

output for http://example.com/full_city_list/json

{"Agalawatta":{"city_name":"Agalawatta","tid":"156"},"Ahungalla":{"city_name":"Ahungalla","tid":"214"},"Akurana":{"city_name":"Akurana","tid":"164"},"Akuressa":{"city_name":"Akuressa","tid":"162"},"Aluthgama":{"city_name":"Aluthgama","tid":"120"},"Ambalangoda":{"city_name":"Ambalangoda","tid":"125"},"Ampara":{"city_name":"Ampara","tid":"211"},"Anuradhapura":{"city_name":"Anuradhapura","tid":"46"},"Arugambay":{"city_name":"Arugambay","tid":"114"},"Avissawella":{"city_name":"Avissawella","tid":"212"},"Badulla":{"city_name":"Badulla","tid":"108"},"Balangoda":{"city_name":"Balangoda","tid":"149"},"Balapitiya":{"city_name":"Balapitiya","tid":"486"},"Bandarawela":{"city_name":"Bandarawela","tid":"197"}}

output for http://example.com/full_city_list

<markers>
<marker name="Agalawatta" tid="156"/>
<marker name="Ahungalla" tid="214"/>
<marker name="Akurana" tid="164"/>
<marker name="Akuressa" tid="162"/>
<marker name="Aluthgama" tid="120"/>
<marker name="Ambalangoda" tid="125"/>
<marker name="Ampara" tid="211"/>
<marker name="Anuradhapura" tid="46"/>
<marker name="Arugambay" tid="114"/>
<marker name="Avissawella" tid="212"/>
<marker name="Badulla" tid="108"/>
<marker name="Balangoda" tid="149"/>
<marker name="Balapitiya" tid="486"/>
<marker name="Bandarawela" tid="197"/>
</markers>

Drupal menu system callback mapping

How does drupal menu system callback mapping happen?

When a user makes a request to drupal through a browser, using the request URL drupal menu system decides which callback function to invoke.

what is a callback function?

Callback function in this case is a php function which gets called when a user trying to access a path that has already been registered in a menu item.

In the given example-1 if a browser makes a request using the URL "http://example.com/hello", menu system will look for a menu item which contains that path and invokes the relevant callback function in this case, "hello()".

example-1

<?php

//hook_perm implementation  
function example_perm() {
    return array(
        'access hello'
    );
}

//hook_menu implementation  
function example_menu() {
    $items = array();
    //menu item   
    $items['hello'] = array(
    'title' =--> t('hello'
    ), 'page callback' => 'hello', 'page arguments' => array(
    0
    ), 'access arguments' => array(
    'access hello'
    ), 'type' => MENU_CALLBACK);

    return $items;
}

//callback function  
function hello() {
    echo "hello!";
}
?>  

Output for example-1

hello!

example-2

This example shows how to get city data in json or xml using a request URL.

<?php

//hook_perm implementation
function example_perm() {
    return array(
        'access hello'
    );
}

//hook_menu implementation
function example_menu() {
    $items = array();
    //menu item 
    $items['hello'] = array(
        'title' => t('hello'
        ), 'page callback' => 'hello', 'page arguments' => array(
            0
        ), 'access arguments' => array(
            'access hello'
        ), 'type' => MENU_CALLBACK);

    $items['full_city_list_json'] = array(
        'title' => t('City List JSON'
        ), 'page callback' => 'full_city_list_json', 'page arguments' => array(
            0
        ), 'access arguments' => array(
            'access hello'
        ), 'type' => MENU_CALLBACK);

    $items['full_city_list_xml'] = array(
        'title' => t('City List XML'
        ), 'page callback' => 'full_city_list_xml', 'page arguments' => array(
            0
        ), 'access arguments' => array(
            'access hello'
        ), 'type' => MENU_CALLBACK);

    return $items;
}

function hello() {
    echo "hello!";
}

function full_city_list_json() {
    $response = array();
    $result = db_query("SELECT  tid,name FROM term_data  WHERE  vid=2 order by name ASC");
    while ($city_data = db_fetch_object($result)) {
        $response[$city_data->name]['city_name'] = $city_data->name;
        $response[$city_data->name]['tid'] = $city_data->tid;
    }

    if (!empty($response)) {
        //encodes city data as json  
        $jsonstrCityList = json_encode($response);
        echo $jsonstrCityList;
    } else {
        return false;
    }
}

function full_city_list_xml() {
//sending city data as xml
    $dom = new DOMDocument("1.0");
    $root_elm = $dom->createElement("markers");
    $root = $dom->appendChild($root_elm);

    $result = db_query("SELECT  tid,name FROM term_data  WHERE  vid=2 order by name ASC");

    while ($city_data = db_fetch_object($result)) {

        $marker_elm = $dom->createElement("marker");
        $marker_node = $root->appendChild($marker_elm);

        $marker_node->setAttribute("name", $city_data->name);
        $marker_node->setAttribute("tid", $city_data->tid);
    }
    header("Content-type: text/xml");
    echo $dom->saveXML();
}

?>

Outputs for example-2

for http://example.com/full_city_list_json

{"Agalawatta":{"city_name":"Agalawatta","tid":"156"},"Ahungalla":{"city_name":"Ahungalla","tid":"214"},"Akurana":{"city_name":"Akurana","tid":"164"},"Akuressa":{"city_name":"Akuressa","tid":"162"},"Aluthgama":{"city_name":"Aluthgama","tid":"120"},"Ambalangoda":{"city_name":"Ambalangoda","tid":"125"},"Ampara":{"city_name":"Ampara","tid":"211"},"Anuradhapura":{"city_name":"Anuradhapura","tid":"46"},"Arugambay":{"city_name":"Arugambay","tid":"114"},"Avissawella":{"city_name":"Avissawella","tid":"212"},"Badulla":{"city_name":"Badulla","tid":"108"},"Balangoda":{"city_name":"Balangoda","tid":"149"},"Balapitiya":{"city_name":"Balapitiya","tid":"486"},"Bandarawela":{"city_name":"Bandarawela","tid":"197"}}

for http://example.com/full_city_list_xml

<markers>
<marker name="Agalawatta" tid="156"/>
<marker name="Ahungalla" tid="214"/>
<marker name="Akurana" tid="164"/>
<marker name="Akuressa" tid="162"/>
<marker name="Aluthgama" tid="120"/>
<marker name="Ambalangoda" tid="125"/>
<marker name="Ampara" tid="211"/>
<marker name="Anuradhapura" tid="46"/>
<marker name="Arugambay" tid="114"/>
<marker name="Avissawella" tid="212"/>
<marker name="Badulla" tid="108"/>
<marker name="Balangoda" tid="149"/>
<marker name="Balapitiya" tid="486"/>
<marker name="Bandarawela" tid="197"/>
</markers>

As in example-2 after you grabbed the city data in json or in xml you can parse them and use in any of your applications.

Parsing JSON (comes as a response to a web request) using java

JSON (JavaScript Object Notation) as in definition is a light weight data exchange format, a fat free alternative to XML. In today's post I will show you how to parse JSON (comes as a response to a web request) using java.

First download json-lib and add it to your classpath. You'll also have to add following libraries to your classpath

1. commons-lang.jar
2. commons-beanutils.jar
3. commons-collections.jar
4. commons-logging.jar
5. ezmorph.jar

What you need next is a request url and it's response should be JSON data. You will see mine in the example code I've provided, if you know how to use firebug the task will be much more simpler. In your browser run that request you'll get the JSON data which I parse, it's lengthy that's why I didn't add it in here. If you get untidy JSON data, format it using jasonlint.

Now let's move on to the example code, it's not hard to understand, simply go through it. Refer comments for clarifications.

java code

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import net.sf.json.JSONArray;
import net.sf.json.JSONObject;
import net.sf.json.JSONSerializer;

public class JsonParse { 

 public static void main(String[] args) {
  String jsonString = "";

  String city = "London";
  String country = "United+Kingdom";

  //This is the request url  
  String requestUrl = "http://www.bing.com/travel/hotel/hotelReferenceData.do?q="
    + city
    + "%2C+"
    + country
    + "+leave+07%2F08%2F2011+return+07%2F10%2F2011+adults%3A2+rooms%3A1&distanceFrom=22&HotelPriceRange=1%2C2%2C3%2C4&pageOffset=0&pageCount=240&cim=2011-07-08&com=2011-07-10&a=2&c=0&r=1&sortBy=PopularityDesc";
  try {
   URL url = new URL(requestUrl.toString());
   BufferedReader in = new BufferedReader(new InputStreamReader(
     url.openStream()));
   String inputLine;

   while ((inputLine = in.readLine()) != null) {
    //JSON data get stored as a string
    jsonString = inputLine;

   }
   in.close();
   //the way to parse JSON array
   JSONArray json = (JSONArray) JSONSerializer.toJSON(jsonString);
   //getting another JSON string to parse
   String parse1 = json.getString(0);
   
   //the way to parse JSON object
   JSONObject json2 = (JSONObject) JSONSerializer.toJSON(parse1);
   //getting a json array inside JSON object
   JSONArray array1 = json2.getJSONArray("hotels");

   for (int i = 0; i < array1.size(); i++) {
    //getting a JSON object inside JSON array
    JSONObject jhotelrec = array1.getJSONObject(i);    
    System.out.println("Hotel Name= "+jhotelrec.getString("name")+"\n"+"Geo Codes= "+jhotelrec.getJSONArray("geoPoint").getString(0)+","+jhotelrec.getJSONArray("geoPoint").getString(1));
    }

  } catch (IOException e) {
   e.printStackTrace();
  }
 }
 
 

}

xml parsing with java

In certain situations we java programmers may need to access information in XML documents using a java xml parser. This post will guide you to parse a xml document with java, using DOM parser and Xpath.

xmlfile.xml

Given XML document is quite lengthy, but do not worry because parsing a lengthy XML document is as easy as parsing a short XML document :) .

<a>
<e>
<hotels class="array">
   <e class="object">
    <addressLine1 type="string">4 Rue du Mont-Thabor</addressLine1>
    <amenities class="array">
     <e type="string">24</e>
     <e type="string">31</e>
     <e type="string">42</e>
     <e type="string">52</e>
     <e type="string">9</e>
    </amenities>
    <brandCode type="string">69</brandCode>
    <cachedPrice type="number">935</cachedPrice>
    <city type="string">Paris</city>
    <country type="string">US</country>
    <geoPoint class="array">
     <e type="number">48.86536</e>
     <e type="number">2.329584</e>
    </geoPoint>
    <hotelRateIndicator type="string">2</hotelRateIndicator>
    <id type="number">56263</id>
    <name type="string">Renaissance Paris Vendome Hotel</name>
    <neighborhood type="string" />
    <popularity type="number">837</popularity>
    <starRating type="string">5</starRating>
    <state type="string">IdF</state>
    <telephoneNumbers class="array">
     <e type="string" />
    </telephoneNumbers>
    <thumbnailUrl type="string">http://www.orbitz.com//public/hotelthumbnails/53/97/85397/85397_TBNL_1246535840051.jpg
    </thumbnailUrl>
    <total type="number">250</total>
    <ypid type="string">YN10001x300073304</ypid>
   </e>
   <e class="object">
    <addressLine1 type="string">39 Avenue de Wagram</addressLine1>
    <amenities class="array">
     <e type="string">24</e>
     <e type="string">31</e>
     <e type="string">42</e>
     <e type="string">9</e>
    </amenities>
    <brandCode type="string">69</brandCode>
    <cachedPrice type="number">633</cachedPrice>
    <city type="string">Paris</city>
    <country type="string">US</country>
    <geoPoint class="array">
     <e type="number">48.877106</e>
     <e type="number">2.297451</e>
    </geoPoint>
    <hotelRateIndicator type="string">3</hotelRateIndicator>
    <id type="number">112341</id>
    <name type="string">Renaissance Paris Arc de Triomphe Hotel</name>
    <neighborhood type="string" />
    <popularity type="number">796</popularity>
    <starRating type="string">5</starRating>
    <state type="string">IdF</state>
    <telephoneNumbers class="array">
     <e type="string" />
    </telephoneNumbers>
    <thumbnailUrl type="string">http://www.orbitz.com//public/hotelthumbnails/21/72/302172/302172_TBNL_1246535872514.jpg
    </thumbnailUrl>
    <total type="number">250</total>
    <ypid type="string">YN10001x300073331</ypid>
   </e>
   <e class="object">
    <addressLine1 type="string">35 Rue de Berri</addressLine1>
    <amenities class="array">
     <e type="string">24</e>
     <e type="string">31</e>
     <e type="string">42</e>
     <e type="string">9</e>
    </amenities>
    <brandCode type="string">82</brandCode>
    <cachedPrice type="number">706</cachedPrice>
    <city type="string">Paris</city>
    <country type="string">US</country>
    <geoPoint class="array">
     <e type="number">48.873684</e>
     <e type="number">2.306411</e>
    </geoPoint>
    <hotelRateIndicator type="string">3</hotelRateIndicator>
    <id type="number">108606</id>
    <name type="string">Crowne Plaza Hotel PARIS-CHAMPS ELYSÉES</name>
    <neighborhood type="string" />
    <popularity type="number">796</popularity>
    <starRating type="string">5</starRating>
    <state type="string">IdF</state>
    <telephoneNumbers class="array">
     <e type="string" />
    </telephoneNumbers>
    <thumbnailUrl type="string">http://www.orbitz.com//public/pegsimages/CP/thumb_PARAT.jpg
    </thumbnailUrl>
    <total type="number">250</total>
    <ypid type="string">YN10001x300161106</ypid>
   </e>
   </hotels>
 </e>
</a>   

XMLsample.java

In this code I have used DOM parser and xpath java library to query xml document. Here I am trying to extract hotel information[Hotel name, geo codes, star rating] from xmlfile.xml document

package com.eviac.blog;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpression;
import javax.xml.xpath.XPathFactory;

import org.w3c.dom.Document;
import org.w3c.dom.NodeList;

public class XMLsample {

 public static void main(String[] args) {

  try {
   // loading the xml document into DOM Document object
   DocumentBuilderFactory domFactory = DocumentBuilderFactory.newInstance();
   domFactory.setNamespaceAware(true);
   DocumentBuilder builder = domFactory.newDocumentBuilder();
   Document doc = builder.parse("xmlfile.xml");

   // XPath object using XPathFactory
   XPath xpath = XPathFactory.newInstance().newXPath();
   
   // XPath Query, compiling the path using the compile() method
   XPathExpression expr = xpath.compile("//hotels/e/name | //hotels/e/starRating | //hotels/e/geoPoint/e/text()");
   Object result = expr.evaluate(doc, XPathConstants.NODESET);
   NodeList nodes = (NodeList) result;
   for (int i = 0; i < nodes.getLength(); i++) {
    System.out.println(nodes.item(i).getTextContent());
   }
  } catch (Exception e) {
   e.printStackTrace();
  }
 }
}

Output

48.86536
2.329584
Renaissance Paris Vendome Hotel
5
48.877106
2.297451
Renaissance Paris Arc de Triomphe Hotel
5
48.873684
2.306411
Crowne Plaza Hotel PARIS-CHAMPS ELYSÉES
5