How to Create Peer-to-Peer Web Servers, Servlets, JSPs, and XML-RPC Clients and Servers

by Brad Neuberg
11/23/2004

The Peer-to-Peer (P2P) Sockets Project reimplements Java's standard Socket, ServerSocket, and InetAddress classes to work on the JXTA peer-to-peer network, rather than on the standard TCP/IP network. It also includes ports of many popular web packages, such as the Jetty web server, the Apache XML-RPC client and server libraries, and the Apache Jasper JSP engine, to run on the Peer-to-Peer Sockets framework.

By the end of this article you will know how to create and access peer-to-peer web servers, servlets, JSPs, and XML-RPC clients and servers. Why would you want to do this? First, P2P web servers can be run on machines that would ordinarily not be resolvable or reachable, due to the issues discussed above. Second, you can now leverage your existing servlet and JSP skills to create peer-to-peer networks without learning new APIs. Peers in your network can assume the roles of both HTTP client and server, interacting with other peers who are themselves both HTTP clients and servers. The servlets and JSPs exposed could be related to instant messaging, blogging, WikiWiki groupware functionality, or more. Now every peer in the network, regardless of whether their cable ISP treats them as second-class citizens, can easily be both a writeable and readable node, opening the possibility of a Two Way Web. Even better, everything is built above a modified version of Jetty, an open-source web server that is specifically designed to be small and embeddable in other applications, making it easy for you to roll P2P functionality into your own applications.

For more information on using the raw P2P sockets and server sockets in your own code, and for details information on how the peer network itself is created, see the tutorial Introduction to Peer-to-Peer Sockets.

Motivation

The P2P Sockets project is designed for developers interested in:

The P2P Sockets project reimplements the standard java.net classes on top of the JXTA peer-to-peer network. "Aren't standard TCP/IP sockets and server sockets already peer-to-peer?" some might ask. Standard TCP/IP sockets and server sockets are theoretically peer-to-peer, but in practice are not, due to firewalls, NAT devices, and political and technical issues with the Domain Name System (DNS). First, many of the peers on the Internet are given dynamic IP addresses through DHCP, shared or filtered IP addresses through NAT devices, or IP addresses that are very hard to reach due to firewalls. Creating server sockets under such conditions is either impossible or requires elaborate application-level ruses to bypass these restrictions. Second, TCP/IP sockets and server sockets depend on DNS to resolve human-friendly host names into IP addresses. DNS is theoretically semi-decentralized, but on an administrative level, it is centralized under ICANN, an unresponsive, non-innovative organization. Further, standard DNS does not deal well with edge-peers that have filtered or dynamic IP addresses, and updates take too long to propagate and have no presence information. Developers who wish to create exciting, new applications that extend DNS into new directions, such as storing instant messaging usernames and presence info in DNS, are stymied by the DNS system's technical and political issues.

P2P Sockets is based on JXTA, an open source project that creates a peer-to-peer overlay network that sits on top of TCP/IP. Every peer on the network is given an IP-address-like number, even if they are behind a firewall or don't have a stable IP address. Super-peers on the JXTA network run application-level routers that store special information such as how to reach peers, how to join subgroups of peers, and what content peers are making available. JXTA application-level relays can proxy requests between peers that would not normally be able to communicate due to firewalls or NAT devices. Peers organize themselves into peer groups, which scope all search requests and act as natural security containers. Any peer can publish and create a peer group in a decentralized way, and other peers can search for and discover these peer groups using other super-peers. Peers communicate using pipes, which are very similar to Unix pipes. Pipes abstract the exact way in which two peers communicate, allowing peers to communicate using other peers as intermediaries if they normally would not be able to communicate due to network partitioning.

JXTA is an extremely powerful framework. However, it is not an easy framework to learn, and porting existing software to work on JXTA is not for the faint-of-heart. P2P Sockets effectively hides JXTA by creating a thin illusion that the peer-to-peer network is actually a standard TCP/IP network. If peers wish to become servers, they simply create a P2P server socket with the domain name they want, and the port other peers should use to contact them. P2P clients open socket connections to hosts that are running services on given ports. Hosts can be resolved either by domain name, such as www.nike.laborpolicy, or by IP address, such as 44.22.33.22. Behind the scenes, these resolve to JXTA primitives, rather than being resolved through DNS or TCP/IP. P2P sockets and server sockets work exactly the same as normal TCP/IP sockets and server sockets. For the technically inclined and those who already understand JXTA, a table exists illustrating how standard TCP/IP concepts such as host name, IP address, etc., map to their JXTA equivalents.

The P2P Sockets project already includes a large amount of software ported to use the peer-to-peer network, including a web server (Jetty) that can receive requests and serve content over the peer-to-peer network; a servlet and JSP engine (Jetty and Jasper) that allows existing servlets and JSPs to serve P2P clients; an XML-RPC client and server (Apache XML-RPC) for accessing and exposing P2P XML-RPC endpoints; an HTTP/1.1 client (Apache Commons HTTP-Client) that can access P2P web servers; a gateway (Smart Cache) to make it possible for existing browsers to access P2P web sites; and a WikiWiki (JSPWiki) that can be used to host WikiWikis on your local machine that other peers can access and edit through the P2P network. Even better, all of this software works and looks exactly as it did before being ported. Everything included in the P2P sockets project is open source, mostly under BSD-type licenses, and cross-platform due to being written in Java.

The 1.1.4 release of P2P Sockets includes powerful autoconfiguration technology through the JXTA Profiler. The JXTA Profiler is an addition to the JXTA project spun out of P2P Sockets that has now been refactored and reintegrated back into the core JXTA. On startup, a P2P Sockets application automatically scans a peers local environment to determine if they are NATed, firewalled, and more, and autoconfigures the JXTA network stack appropriately. Programmers and end-users now do not need to manually configure their peers based on their network environment, drastically simplifying deployment and configuration.

Requirements and Configuration

You must download and install the following software to develop and work with P2P Sockets.

Install and configure the JDK and Ant, and make sure both are in your path so they can be run from the command line. Unzip p2psockets-1_1_4.zip into the top level of your hard drive; spaces are not allowed in the directory names, or the P2P Sockets build files will not work correctly.

When testing the examples in this tutorial, you must be connected to the Internet. This is for two reasons: first, the examples use a public JXTA server Sun(tm) has set up that helps bootstrap peers into the JXTA network and profile their network condition; and second, on some operating systems (such as on Windows XP, by default), the network subsystem shuts down if you are not connected to the Internet, preventing client peers and server peers that are running on the same machine from finding or communicating with each other.

Starting a P2P Web Server from the Command-Line

Starting a simple P2P web server to serve static content and run servlets and JSPs is easy. Open a shell window and enter the directory you installed the P2P Sockets package into. Then, define the following shell variables, modifying them as appropriate for your platform and where you have installed your software:

set p2psockets_home=c:\p2psockets
set JAVA_HOME=c:\j2sdk1.4.1

To start your P2P web server, type:

ant webserver-run -D--host=www.nike.laborpolicy -D--network=TestNetwork -D--port=80

This will start up a P2P web server. You should see something like the following printed out:

Signing into the P2P network...
Using the application peer group name TestNetwork
Waiting for RendezVous Connection....
Finished connecting to RendezVous.
02:58:42.730 EVENT Checking Resource aliases
02:58:48.889 EVENT Starting Jetty/4.2.10pre0
02:58:48.909 EVENT Started org.mortbay.http.NCSARequestLog@33f0de
02:58:50.602 EVENT setStatsOn true for
WebApplicationContext[/jetty,Jetty Demo]
02:58:50.862 EVENT Started WebApplicationContext[/jetty,Jetty
Demo]
02:58:51.874 EVENT Started ServletHttpContext[/cgi-bin]
02:58:52.174 EVENT Started WebApplicationContext[/,Jetty Demo
Root]
02:58:52.214 EVENT Started ServletHttpContext[/demo]
02:58:52.705 EVENT Started
WebApplicationContext[/examples,c:\p2psockets/support-packages/jetty/demo/webapps/examples]
02:58:52.735 EVENT ContextListener: contextInitialized()
02:58:52.735 EVENT SessionListener: contextInitialized()
02:58:52.835 EVENT ContextListener:
attributeAdded('javax.servlet.context.tempdir',
'C:\DOCUME~1\BRADNE~1\LOCALS~1\Temp\Jetty_238_39_22_221_80__examples')
02:58:52.855 EVENT Started ServletHttpContext[/javadoc]
02:59:15.327 EVENT Started P2PSocketListener on 238.39.22.221:80
02:59:15.337 EVENT Started AJP13Listener on 0.0.0.0:8009
02:59:15.337 EVENT NOTICE: AJP13 is not a secure protocol. Please
protect the port 0.0.0.0:8009
02:59:15.337 EVENT Started
org.mortbay.p2psockets.jetty.Server@2acc65

By changing the values after -D--port and -D--host you can gain greater control over the port and host name clients will use to access your web server.

The -D--network=TestNetwork require mores explanation. TestNetwork, in the above example, is a unique name that will be given to your peer-to-peer network. Different peer-to-peer networks that are based on P2P Sockets can co-exist without knowing about each other. Clients and servers create and resolve their server sockets and sockets in a specific peer-to-peer network. This value is the name of your own private, application-specific peer-to-peer network4. If you create your P2P web server in the peer-to-peer network named TestNetwork, while a client signs into another peer-to-peer network named InstantMessagingNetwork, then they will not be able to find each other or communicate. Choose your network name to go along with the name of your application or the type of network you are creating, such as MyApplicationsNetwork or AcmeCompanyInformationNetwork.

By default, the web server started above will look for static content files in the directory p2psockets/docroot. To make your own content accessible to P2P clients simply drop it into this folder. To add dynamic content, you must add your servlet, JSP file, or WAR file to p2psockets/webapps. Your web application will then be served from /webapps on the URL; i.e. if you put myapp.war into p2psockets/webapps then you would access it as a URL you would access it as /webapps/myapps if you web.xml file in myapp.war mapped it to myapps

To test your P2P web server, open another window and change into the p2psockets directory. After doing so, type the following:

ant httpclient-run -D--network=TestNetwork -D--url=p2phttp://www.nike.laborpolicy/webapps/examples/jsp/index.html

This will start a modified version of the Apache Commons HTTP-Client, using the clientpeer JXTA configuration files in test/clientpeer Apache HTTP-Client is a client-side implementation of the HTTP/1.1 protocol; it has been modified to work on the JXTA P2P network.

You should see HTML code fill the page in the window you started the HTTP client in.

Accessing your P2P Web Server from a Normal Browser

If P2P web servers could only be accessed from command-line HTTP clients then the system would have limited utility. The P2P Sockets project, however, includes a special system to make it possible for ordinary web browsers to access P2P web servers. This is done by running a special proxy on the local machine and modifying the web browser to use this proxy for all domain name resolutions5. This locally-running proxy takes normal HTTP requests from a web browser, proxies them over the peer-to-peer network to a P2P web server, and then returns the P2P results as an ordinary HTTP response to the browser.

To start this proxy, type the following:

ant p2pproxy-run -D--network=TestNetwork -D--port=8080

This will start the P2P Proxy on port 8080, using the JXTA configuration file located in test/clientpeer. The peer-to-peer network that will be used is the one named TestNetwork. You should see something similar to the following printed out in the window:


Using the application peer group name TestNetwork
Waiting for RendezVous Connection................
Finished connecting to RendezVous.
Creating proxy...
Smart Cache 0.88 - full featured caching proxy server and web forwarder.
Copyright (c) Radim Kolar 1998-2003. Opensource software; There is NO
warranty.
See the GNU General Public License version 2 or later for copying
conditions.

[WARNING] No config file (.\scache.cnf) found, using defaults.
Checking filesystem (store)
 Filesystem allows ending dot in filename: false
 Filesystem allows backslash in filename : false
 Filesystem reports real directory size : false

Fri Sep 05 03:36:28 PDT 2003 Smart Cache 0.88 ready on 8080/*

You now need to configure your web browser to use the proxy that is running on your machine.

If you are using Mozilla, do the following. From the pull-down menus, select Edit > Preferences... A pop-up window should appear. On the left of the window, select Advanced > Proxies. The proxy configuration should appear on the right as follows:

Screenshot of Mozilla proxy before changing

Select the radio button titled Manual proxy configuration, and in the field labeled HTTP Proxy enter the value 127.0.0.1; in the field label Port to the right of the HTTP Proxy field enter the value 8080. After finishing, the dialog should look as follows:

Screenshot of Mozilla proxy after changing

Press the OK button.

If you are using Firefox, do the following. From the pull-down menus, select Tools > Options... A popup window should appear; select the icon on the lower left titled Connection. You should see the following:

FireBird proxy before changing

Select the radio button titled Manual proxy configuration, and in the field labeled HTTP Proxy enter the value 127.0.0.1; in the field label Port to the right of the HTTP Proxy field enter the value 8080. After finishing, the dialog should look as follows:

FireBird proxy after changing

Press the OK button.

It is also possible to configure Internet Explorer to use your local proxy, though this tutorial does not detail the steps necessary to do so.

After configuring your browser, enter the address www.nike.laborpolicy in the location bar in your browser, just as you would for an ordinary web-site. You should see a web page load in the browser.

Creating P2P Servlets, JSPs, Web Application Archives (WARs)

There is absolutely nothing you need to do to make your servlets, JSPs, and WARs P2P enabled. Your servlets will receive their HTTP requests and responses as normal, but instead of these HTTP requests comming over a normal HTTP TCP/IP connection, they are tunneled over a Jxta Pipe connection. Your servlets are hidden from the specifics of the P2P connection, and clients simply appear as normal clients. JSPs and WARs also work the same way, and are 'tricked' into believing that they are communicating with normal HTTP clients.

Starting a P2P Web Server from your Java Software

If you want more control over how Jetty is started and where it places its resources and servlets, such as if you are embedding Jetty into a larger piece of software, then you will need to start Jetty programatically. The example below shows how this is done:

import java.io.*;
import java.net.*;

import org.p2psockets.*;

import org.mortbay.util.*;
import org.mortbay.http.*;
import org.mortbay.p2psockets.util.*;
import org.mortbay.p2psockets.http.*;
import org.mortbay.jetty.*;
import org.mortbay.jetty.servlet.*;
import org.mortbay.http.handler.*;
import org.mortbay.servlet.*;

/** This class tests setting up a peer-to-peer web server. */
public class ExampleP2PWebServer {
  public static void main(String args[]) throws Exception {
    // initialize and sign into the peer-to-peer network
    P2PNetwork.signin("serverpeer", "serverpeerpassword", "TestNetwork", true);
    
    // Create the server
    Server server = new Server();
 
    // Create a port listener
    System.out.println("Starting web server for www.boobah.cat...");
    SocketListener listener=new P2PSocketListener();

    listener.setInetAddress(P2PInetAddress.getByAddress("www.boobah.cat", null));
    listener.setPort(80);
    server.addListener(listener);

    // Create a context
    HttpContext context = new HttpContext();
    context.setContextPath("/");
    server.addContext(context);
 
    // Create a servlet container
    ServletHandler servlets = new ServletHandler();
    context.addHandler(servlets);

    // Map a servlet onto the container
    servlets.addServlet("Dump","/Dump/*","org.mortbay.servlet.Dump");
    server.addWebApplication("/", System.getProperty("jetty.home") +  
                             "/demo/webapps/jetty/JSPWiki.war");
                             
    // Serve static content from the context
    context.setResourceBase(System.getProperty("jetty.home") +  
                            "/demo/docroot");
    context.addHandler(new ResourceHandler());

    // Start the http server
    server.start();
 }
}

In the example above, we first sign in and initialize our P2P network:

// initialize and sign into the peer-to-peer network
P2PNetwork.signin("serverpeer", "serverpeerpassword", "TestNetwork", true);

To sign into the network, you must provide a username and password. The signin method looks for a special hidden directory named .jxta that holds various important JXTA P2P information. The directory is always created in the same directory that the application is run in. If the directory does not exist then the peer is profiled and a configuration is transparently created (the fourth argument to signin, true, controls whether to create a JXTA configuration directory if one does not exist for the given username).

The first two arguments to the signin method are a secure username and password. If you are using the already configured .jxta directory in test/clientpeer, then the username is clientpeer and the password is clientpeerpassword. The user name and password for the JXTA peer configured in test/serverpeer is serverpeer and serverpeerpassword.

The third value, TestNetwork in the above example, is a unique name that will be given to your peer-to-peer network. Different peer-to-peer networks that are based on P2P Sockets can co-exist without knowing about each other. Clients and servers create and resolve their server sockets and sockets in a specific peer-to-peer network. The final value is the name of your own private, application-specific peer-to-peer network. If you create your server socket in the peer-to-peer network named TestNetwork, while a client signs into another peer-to-peer network named InstantMessagingNetwork, then they will not be able to find each other or communicate4. Choose your network name to go along with the name of your application or the type of network you are creating, such as MyApplicationsNetwork or AcmeCompanyInformationNetwork.

After signing into the network, you then create a server and a peer-to-peer socket listener that will listen for requests from peer listeners, rather than on a normal TCP/IP socket:

  // Create the server
 Server server = new Server();
 
 // Create a port listener
 System.out.println("Starting web server for www.boobah.cat...");
 SocketListener listener = new P2PSocketListener();

 listener.setInetAddress(P2PInetAddress.getByAddress("www.boobah.cat", null));
 listener.setPort(80);
 server.addListener(listener);

The P2PInetAddress class is a subclass of java.net.InetAddress with knowledge of how to deal with P2P hostnames and IP addresses. It is fully detailed in the tutorial Introduction to the Peer-to-Peer Sockets Project.

The rest of the example uses the normal Jetty API to add and configure a servlet, a resource path to retrieve static content, and a Web Application Archive. See the Jetty documentation for more information on how to use the Jetty API.

To run the example web server, open a window and enter the following:

ant example-webserver-run

Accessing a P2P Web Server from your Java Software

To access P2P web servers programatically from your applications as clients, you can use the Apache Commons HTTP-Client library. This library has been modified to work on a P2P network. An example is shown below:

import java.io.*;
import java.net.*;

import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.methods.*;
import org.apache.commons.httpclient.protocol.*;

/**
*
* This is a simple text mode application that demonstrates
* how to use the Jakarta HttpClient API. All requests are
* made over JXTA rather than through normal IP sockets.
*/
public class ExampleP2PHttpClient {

 public static void main(String[] args) {
  // initialize JXTA
  try {
   // sign in and initialize the JXTA network
   Network.signin("clientpeer", "clientpeerpassword", "TestNetwork");
  }
  catch (Exception e) {
   e.printStackTrace();
   System.exit(1);
  }

  // register the P2P socket protocol factory
  Protocol jxtaHttp = new Protocol("p2phttp", new P2PProtocolSocketFactory(), 80 );
  Protocol.registerProtocol("p2phttp", jxtaHttp );

  //create a singular object
  HttpClient client = new HttpClient();

  //establish a connection within 50 seconds
  client.setConnectionTimeout(50000);

  String url = args[2];
  HttpMethod method = null;

  //create a method object
  method = new GetMethod(url);
  method.setFollowRedirects(true);
  method.setStrictMode(false);
 
  //execute the method
  String responseBody = null;
  try{
    client.executeMethod(method);
    responseBody = method.getResponseBodyAsString();
  } catch (HttpException he) {
    System.err.println("Http error connecting to '" + url + "'");
    System.err.println(he.getMessage());
    System.exit(-4);
  } catch (IOException ioe) { 
    System.err.println("Unable to connect to '" + url + "'");
    System.exit(-3);
  }

  //write out the request headers
  System.out.println("*** Request ***");
  System.out.println("Request Path: " + method.getPath());
  System.out.println("Request Query: " + method.getQueryString());
  Header[] requestHeaders = method.getRequestHeaders();
  for (int i=0; i<requestHeaders.length; i++){
    System.out.print(requestHeaders[i]);
  } 

  //write out the response headers
  System.out.println("*** Response ***");
  System.out.println("Status Line: " + method.getStatusLine());
  Header[] responseHeaders = method.getResponseHeaders();
  for (int i=0; i<responseHeaders.length; i++) {
    System.out.print(responseHeaders[i]);
  }

  //write out the response body
  System.out.println("*** Response Body ***");
  System.out.println(responseBody); 

  //clean up the connection resources
  method.releaseConnection();
  method.recycle();

  System.exit(0);
 }
}

The first part of this example initializes and signs into the P2P network, just as in the P2P web server example before. You must always take this step before interacting with the network:

  // initialize JXTA
  try {
    // sign in and initialize the JXTA network
    Network.signin("clientpeer", "clientpeerpassword", "TestNetwork");
  }
  catch (Exception e) {
    e.printStackTrace();
    System.exit(1);
  }

Next, we must establish the existence of a new protocol handler to the HTTP-Client library:

  // register the P2P socket protocol factory
  Protocol jxtaHttp = new Protocol("p2phttp", new P2PProtocolSocketFactory(), 80 );
  Protocol.registerProtocol("p2phttp", jxtaHttp );

This informs the client library that whenever it sees URL that starts with p2phttp://, that is should use our modified P2P factory to create P2P sockets rather than the normal one that creates TCP/IP sockets. You could also override all http:// URLs as well with the following:

  // register the P2P socket protocol factory
  Protocol jxtaHttp = new Protocol("http", new P2PProtocolSocketFactory(), 80 );
  Protocol.registerProtocol("http", jxtaHttp );

We avoid this in the example above so that the client library can be used to resolve both normal, TCP/IP HTTP servers as well as web servers resolved through the JXTA, P2P network.

To test out this example, open a shell window and enter the following:

ant example-webclient-run -D--network=TestNetwork -D--url=p2phttp://www.boobah.cat

You should see the window fill with HTML.

Creating P2P XML-RPC Clients and Servers

The Apache XML-RPC Client and Server libraries have also been ported to work on a P2P network. Creating these is exactly like creating normal Apache XML-RPC clients and servers, but at the beginning of your code you must call the P2PNetwork.signin method, just as in the examples above.

Using these modified libraries, you can create XML-RPC servers that expose their functions over a peer-to-peer network and XML-RPC clients that access this functionality over the peer network. Peers in your network can assume the roles of both XML-RPC client and server, interacting with other peers who are themselves both XML-RPC clients and servers. The XML-RPC functions exposed could be related to instant messaging, blogging using the MetaWeblog XML-RPC API, editing WikiWikis using the WikiWiki XML-RPC API, and more. Even peers that are normally unreachable can now be first-class citizens in the network. Easily create P2P applications using your existing XML-RPC skills.

Limitations and Security Concerns

The P2P Sockets project currently has the following limitations and security issues:

License Information

P2P Sockets, including the source code in this article, is under the Sun Project JXTA Software License.

Resources

Questions? Comments?

See the P2P Sockets Homepage or contact Brad Neuberg at . Feel free to call him at 1 (510) 938-3263 (Pacific Standard Time, San Francisco) Monday through Friday. Also see his weblog, www.codinginparadise.org, for Mozilla, Java, JXTA, and P2P news.

Brad Neuberg is an open source developer living in San Francisco. He is a Senior Software Engineer for Rojo Networks Inc., a weblog and reputation aggregator.