The Python standard library comes with two HTTP clients: urllib2, which has a file- like interface like Ruby’s open-uri; and httplib, which works more like Ruby’s Net::HTTP. Both offer transparent support for HTTPS, assuming your copy of Python was compiled with SSL support. There’s also an excellent third-party library, Joe Gre- gorio’s httplib2 (http://bitworking.org/projects/httplib2/), which is the one I recom- mend in general. httplib2 is an excellent piece of software, supporting nearly every feature on my wish list—most notably, transparent caching. Table 2-2 lists the features available in each library.
Table 2-2. HTTP feature matrix for Python HTTP client libraries
urllib2 httplib httplib2
HTTPS Yes (assuming Python was compiled with SSL support) " " HTTP verbs GET, POST All All Custom data Yes " " Custom headers Yes " "
Proxies Yes No No
Compression No No Yes
Caching No No Yes
Auth methods Basic, Digest None Basic, Digest, WSSE, Goo- gle
Cookies Yes (Use urllib2.build_opener(HTTPCookieProces
sor)) No No
Redirects Yes No Yes
Example 2-5 is a del.icio.us client that uses httplib2. It uses the ElementTree library to parse the del.icio.us XML.
Example 2-5. A del.icio.us client in Python #!/usr/bin/python2.5
# delicious-httplib2.py import sys
from xml.etree import ElementTree import httplib2
# Fetches a del.icio.us user's recent bookmarks, and prints each one. def print_my_recent_bookmarks(username, password):
client = httplib2.Http(".cache")
client.add_credentials(username, password)
# Make the HTTP request, and fetch the response and the entity-body. response, xml = client.request('https://api.del.icio.us/v1/posts/recent') # Turn the XML entity-body into a data structure.
doc = ElementTree.fromstring(xml) # Print information about every bookmark. for post in doc.findall('post'):
print "%s: %s" % (post.attrib['description'], post.attrib['href']) # Main program
if len(sys.argv) != 3:
print "Usage: %s [username] [password]" % sys.argv[0] sys.exit()
username, password = sys.argv[1:]
print_my_recent_bookmarks(username, password)
Java: HttpClient
The Java standard library comes with an HTTP client, java.net.HttpURLConnection. You can get an instance by calling open on a java.net.URL object. Though it supports most of the basic features of HTTP, programming to its API is very difficult. The Apache Jakarta project has a competing client called HttpClient (http://jakarta.apache.org/com mons/httpclient/), which has a better design. There’s also Restlet (http://www.rest let.org/). I cover Restlet as a server library in Chapter 12, but it’s also an HTTP client library. The class org.restlet.Client makes it easy to make simple HTTP requests, and the class org.restlet.data.Request hides the HttpURLConnection programming necessary to make more complex requests. Table 2-3 lists the features available in each library.
Table 2-3. HTTP feature matrix for Java HTTP client libraries.
HttpURLConnection HttpClient Restlet
HTTPS Yes " " HTTP verbs All " " Custom data Yes " " Custom headers Yes " " Proxies Yes " " Compression No No Yes Caching Yes No Yes Auth methods Basic, Digest, NTLM " Basic, Amazon Cookies Yes " "
Redirects Yes " "
Example 2-6 is a Java client for del.icio.us that uses HttpClient. It works in Java 1.5 and up, and it’ll work in previous versions if you install the Xerces parser (see “Java: javax.xml, Xerces, or XMLPull” later in this chapter).
Example 2-6. A del.icio.us client in Java // DeliciousApp.java import java.io.*; import org.apache.commons.httpclient.*; import org.apache.commons.httpclient.auth.AuthScope; import org.apache.commons.httpclient.methods.GetMethod; import org.w3c.dom.*; import org.xml.sax.SAXException; import javax.xml.parsers.*; import javax.xml.xpath.*; /**
* A command-line application that fetches bookmarks from del.icio.us * and prints them to standard output.
*/
public class DeliciousApp {
public static void main(String[] args)
throws HttpException, IOException, ParserConfigurationException, SAXException, XPathExpressionException
{
if (args.length != 2) {
System.out.println("Usage: java -classpath [CLASSPATH] " + "DeliciousApp [USERNAME] [PASSWORD]"); System.out.println("[CLASSPATH] - Must contain commons-codec, " + "commons-logging, and commons-httpclient"); System.out.println("[USERNAME] - Your del.icio.us username"); System.out.println("[PASSWORD] - Your del.icio.us password"); System.out.println();
System.exit(-1); }
// Set the authentication credentials.
Credentials creds = new UsernamePasswordCredentials(args[0], args[1]); HttpClient client = new HttpClient();
client.getState().setCredentials(AuthScope.ANY, creds); // Make the HTTP request.
String url = "https://api.del.icio.us/v1/posts/recent"; GetMethod method = new GetMethod(url);
client.executeMethod(method);
InputStream responseBody = method.getResponseBodyAsStream(); // Turn the response entity-body into an XML document. DocumentBuilderFactory docBuilderFactory =
DocumentBuilderFactory.newInstance(); DocumentBuilder docBuilder =
docBuilderFactory.newDocumentBuilder(); Document doc = docBuilder.parse(responseBody); method.releaseConnection();
// Hit the XML document with an XPath expression to get the list // of bookmarks.
XPath xpath = XPathFactory.newInstance().newXPath(); NodeList bookmarks = (NodeList)xpath.evaluate("/posts/post", doc, XPathConstants.NODESET); // Iterate over the bookmarks and print out each one.
for (int i = 0; i < bookmarks.getLength(); i++) {
NamedNodeMap bookmark = bookmarks.item(i).getAttributes(); String description = bookmark.getNamedItem("description") .getNodeValue();
String uri = bookmark.getNamedItem("href").getNodeValue(); System.out.println(description + ": " + uri); } System.exit(0); } }
C#: System.Web.HTTPWebRequest
The .NET Common Language Runtime (CLR) defines HTTPWebRequest for making HTTP requests, and NetworkCredential for authenticating the client to the server. The HTTPWebRequest constructor takes a URI. The NetworkCredential constructor takes a username and password (see Example 2-7).
Example 2-7. A del.icio.us client in C# using System;
using System.IO; using System.Net; using System.Xml.XPath; public class DeliciousApp {
static string user = "username"; static string password = "password";
static Uri uri = new Uri("https://api.del.icio.us/v1/posts/recent"); static void Main(string[] args) {
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(uri); request.Credentials = new NetworkCredential(user, password); HttpWebResponse response = (HttpWebResponse) request.GetResponse(); XPathDocument xml = new
XPathDocument(response.GetResponseStream()); XPathNavigator navigator = xml.CreateNavigator();
foreach (XPathNavigator node in navigator.Select("/posts/post")) { string description = node.GetAttribute("description",""); string href = node.GetAttribute("href","");
Console.WriteLine(description + ": " + href); }
} }