A RESTful Web Service Interface to the ATLAS COOL Database

(1)

A RESTful Web Service Interface

to the ATLAS COOL Database

(2)

COOL

(see talk by A. Valassi, this conference)

2 A RESTful Web Service... (Shaun Roe, Atlas)

http://lcga pp .cern.ch/do xygen/COOL/COOL_2_6_0/do xygen/html/ CHEP'09 Prague

COOL: A database schema and API designed to be technology neutral

(Oracle/MySQL/SQLite)

Data structure is hierarchical:

Originally in C++, its methods are exposed in Python,

in the ‘PyCool’ module. High Voltage values from the

Detector Control System of Atlas SemiConductor Tracker

(3)

REST

REpresentational State Transfer noun verb

e.g. sending a form

e.g. A browser normally GETs a web page

‘404 Not Found…’ ‘Accept: text/html’ Ro y Fielding’ s disser tation: http:// www .ics.uci.edu/~fielding/pubs/disser tation/top .htm RESTful

(4)

Motivation

CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 4

Rewind to 2006…

People want data from Cool on the web!

Solutions:

• Use their own SQL from PHP/Python…

• Use a shell script to set up environment, then

run the Cool api behind some custom code

• Use a cron job to generate local html files

(5)

Aim: Universal Web Service

Identify resources with url e.g.

http://server/database/schema/dbname/some/folder/path/timespan/0-100/channels/23

Use xml as the data transport format.

‘GET’ retrieves the values

‘POST’ creates the channel in a folder

‘PUT’ updates the channel

‘DELETE’ deletes the channel from the folder

Multi-channel uploads/downloads should be possible. DB Modification should be protected.

Additional resources to be matched to a url:

Database nodelist, folder description, folder payload definition,

list of channels in a folder, list of tags; it should be possible to create and delete databases, folders.

(6)

Data format: XML

What about JSON ? (see

later)

</channel>

</channels>

Definition of an XSD schema allows validation of data format on upload, or type-ahead and

documentation tooltips in an editor:

Namespaces?

(oXygen editor)

XML Schema Description

Ja

(7)

Implementation: CherryPy

_{www.cherrypy.org}

CherryPy is a Python application server. It is very easy

to install and use.

The ‘Routes’ module (from Ruby) allows automatic or

manual mapping of Python classes and methods to

URLs:

d.connect('folder_payload','/:server/:schema/:dbname/*folderPath/ payload',controller=restController,action='folderPayload’)

def folderPayload(self, server, schema, dbname, folderPath):

http://myserver.cern.ch/cooldb/ATLAS_COOLPROD/ATLAS_COOL_SCT/DCSP200/SCT/DCS/HV/payload

maps this to this

The http headers and the methods are also available and can be used to route the URL to the appropriate Python method; other modules for security, caching etc are available.

(8)

Examples, curl

Command line examples

http://

curl.haxx.se

(9)

‘In the wild’ examples

/**

* @author formica */

public class CherryPyClient {

private String name = "user.name"; private String psswd = "mypass";

private String readnodeUrl = "http://server/ATLAS_SCT_COMMCOND_DEV/ROETEST/Branch/Trial1/payload";

private String writenodeUrl = http://server/ATLAS_SCT_COMMCOND_DEV/ROETEST/Branch/Trial1/timespan/0-100; public String coolGetCherryPy() {

try {

URL url = new URL(this.readnodeUrl);

java.net.URLConnection conn = url.openConnection(); InputStream is = (InputStream) conn.getInputStream();

BufferedReader in = new BufferedReader(new InputStreamReader(is)); String inputLine = in.readLine();

is.close(); return inputLine; } catch (Exception e) { e.printStackTrace(); } return "none"; } $header=array('Accept: text/xml'); // $ch = curl_init(); curl_setopt($ch, CURLOPT_HTTPHEADER,$header); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); $format=requested_format(); $http_result = curl_exec($ch); if ($format != 'text/xml'){ $dom=domxml_open_mem($http_result); } PHP httpHeaders={'Accept':'text/xml'} textData=None request=urllib2.Request(url,textData, httpHeaders) u=urllib2.urlopen(request) xmlDocString = u.read() return xmlDocString

(10)

eXtensible Stylesheet Language for Transformations

Example: Run List

Search for runs matching criteria, display results;

Uses an ‘Ajax’ request to get the CherryPy XML.

Client-side Local server side

XML is inserted directly in the XHTML page, and associated with an XSLT stylesheet to format it as a table

(11)

Plans, conclusion

Use of CherryPy allowed rapid development of a web service interface to Cool which greatly simplified conditions access and adheres to ‘RESTful’ principles.

It was used during 2008 as the engine for the ‘run list’ query page, for various dcs retrieval services, and for insertion of detector status and calibration data; its adoption is growing as a part of web information collations in the detector subsystems.

Future extensions include extra methods for tag viewing, and more flexible output format (e.g. JSON, zipped format…)