iPlant Atmosphere: A Gateway to Cloud Infrastructure for
the Plant Sciences
Edwin Skidmore
iPlant Collaborative
edwin@iplantcollaborative.org
Sriramu Singaram
iPlant Collaborative
sriramus@iplantcollaborative.org
Seung-jin Kim
iPlant Collaborative
seungjin@iplantcollaborative.org
Nirav Merchant
University of Arizona
nirav@email.arizona.edu
Sangeeta Kuchimanchi
iPlant Collaborative
sangeeta@iplantcollaborative.org
Dan Stanzione
Texas Advanced Computing Center
dan@tacc.utexas.edu
ABSTRACT
The cloud platform complements traditional compute and storage infrastructures by introducing capabilities for efficiently provisioning resources in a self-service, on-demand manner. The new provisioning model promises to accelerate scientific discovery by improving access to customizable and task-specific computing resources. This paradigm is well-suited, especially for those applications tailored to leverage cloud-style of infrastructure capabilities.
Adoption of the cloud model has been challenging for many domain scientists and scientific software developers due to the technical expertise required to effectively utilize this infrastructure. Some of the key limitations of cloud infrastructure are: limited integration with institutional authentication and authorization frameworks, lack of frameworks to enable domain-specific configurations for instances, and integration with scientific data repositories alongside existing computational clusters and grid deployments. Specifically designed to address some of these operational barriers towards adoptions by the plant sciences community, the iPlant Collaborative cloud platform, aptly named Atmosphere, is an open-source, robust, configurable gateway that extends established cloud infrastructure to meet the diverse computing needs for the plant science.
Atmosphere manages the Virtual Machine (VM) lifecycle while maximizing the utilization of cloud resources for scientific workflows. Thus, Atmosphere allows researchers developing novel analytical tools to deploy them with ease while abstracting the underlying computing infrastructure, at the same time making it relatively easy for the users to access these tools via web browser. Atmosphere also provides a rich extensible Application Programming Interface (APIs) for integration and automation with other services. Since its launch, Atmosphere has seen a wide adoption by the plant sciences community for a broad array of applications that range from image processing to next generation sequence (NGS) analysis and can serve as a template for providing similar capabilities to other domains.
Categories and Subject Descriptors
C.2.4 [Distributed Systems]: Distributed Applications – cloud computing and storage.
General Terms
Management, Experimentation, Performance
Keywords
cloud, cloud computing, cloud storage, virtualization, plant sciences, cyberinfrastructure
1.
INTRODUCTION
The iPlant Collaborative was established with a very broad mandate from the National Science Foundation (NSF) to support the cyberinfrastructure (CI) needs of researchers addressing the “grand challenges” of plant biology [1]. Plant biology itself is a very diverse field. The rapid expansion of the –omics era (genomics, proteomics, metabolomics, transcriptomics, etc.) is transforming what has traditionally been a bench- and field-based research discipline into a computationally intense, data-driven area of science.
The research problems of plant biology range from those requiring intense computation (genome assembly, genome wide association studies) to those requiring little computation but intensive data integration (genome and functional annotations). The data sets range in scale and type from molecular phenotype to satellite maps of species’ range, with most scales falling somewhere in between. In most areas of inquiry around plants, the algorithms and workflows are still rapidly evolving, as are the types of questions being explored. The entire cyberinfrastructure foundation, therefore, must support the needs of a diverse group of plant science researchers, ranging from biologists to computer scientists. The iPlant Collaborative cyberinfrastructure comprises of high performance computing (HPC) cluster and data storage resources to the support large-scale data and computational needs of plant science research. While the HPC cluster and storage resources are suitable for many applications, not all scientific computations require the use of supercomputer-scale resources [2]. Some computational workflows require a dedicated server to provide their own interfaces, some of which are web based or native operating system based graphical user interfaces (GUI) with associated local databases, or compute environments. Furthermore, the nature of algorithm development and data analyses as applied to plant biology domain requires customized software development environments with significant software version dependencies, while requiring simultaneous access to large-scale compute and storage infrastructures.
Many existing plant science-related computational tools are a natural fit for the cloud based infrastructure, many of which are highly serial or data parallel. Traditionally, such tools have been designed for single-threaded, small-scale computations, easily executed on a single workstation. Large-scale compute environments typically discourage the execution of these types of tools, as they do not make efficient use of cluster resources. Additionally, large shared clusters are not always amenable to customized execution environments for the domain-specific tasks,
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
Figure 1. High-level illustration of Atmosphere’s components.
such as configuring a wide range of system library versions to support legacy versions of applications, or the latest “bleeding edge” versions of bioinformatics software [3]. The deployment of algorithms and tools thus becomes a challenge in shared computing environments.
Atmosphere addresses these issues by providing preconfigured, domain-specific virtual instances of common bioinformatic tools that are integrated within other parts of iPlant's cyberinfrastructure. In this manner, Atmosphere users are able to quickly develop algorithms and deploy workflows, thereby reducing the extensive time, resources, and overhead needed to set up analyses. Users can access the subset of data required by a specific analysis from iPlant’s large-scale storage by staging the data into their virtual machine (VM). In addition, users are able to preserve the state of their VM instances, saving not only a specific workflow and analysis but an entire system state, introducing new opportunities for algorithm development and experimental reproducibility.
A fundamental practice within biosciences research, particularly in the plant sciences community, is the sharing of data sets, analysis tools, and computational workflows. Tool developers have their own community of collaborators and often seek convenient access to computational resources in order to provide early and reliable access to the analytical tools for their community. This iterative step is often a tedious limitation for biologists, as the data and resource provisioning to validate the tools is a time consuming process. While many cloud solutions focus on the Infrastructure-as-a-Service (IAAS) or Platform-as-a-Service (PAAS) model, Atmosphere reinforces the Software-as-a-Service (SAAS) approach by allowing users to collaborate, share their images, and conveniently launch applications using one-click buttons. SAAS allows the tool developers to provide web-based, secure single click access to their collaborators to precisely configured versions of the application and their dependencies, allowing them to provide quick assessment and feedback before publishing the tool for public consumption. This capability also allows the biologists to adjust the underlying computational infrastructure and modify items such as the number of cores and RAM depending on the size of the data set being analyzed, without having to encumber the developer for additional resources.
In addition to the various cloud service models that Atmosphere supports, the iPlant cloud services platform exposes its functionality by providing HTTP-based application programming interfaces (APIs) through its middleware. The APIs enable functionality available through the web front end as well as mechanisms to customize notifications, resource management, and metadata management. The purpose of the Atmosphere APIs is to encourage deeper integration with other applications and services while making Atmosphere a first-class citizen of the institutional infrastructure through the integration capabilities.
2.
RELATED WORK
Before the inception of Atmosphere in 2010, there were few mature open-source cloud-centric middleware or portal projects focused toward a biological sciences community. Below are some of the projects that iPlant evaluated before developing a targeted cloud infrastructure for the plant science community.
Many IAAS clouds utilize virtualization software, such asXen [4], VMWare [5], or Kernel Virtual Machine (KVM) [6].
Virtualization software typically exists at a low level of most infrastructures and requires proficient systems administrators to provision resources for end-users. The nature of virtualization and the provisioning of virtual resources often demand an intimate knowledge of underlying physical resources and the low level access controls. Additionally, the user interfaces for virtualization software are typically command-line tools or, at best, desktop applications requiring direct access to the systems and hardware. Many academic and open-source cloud projects began in an attempt to fulfill an unmet need to provide the dynamic, user-centric provisioning of virtual resources. Some projects began as research projects and have continued as open-source projects, such as OpenNebula [7]. Other projects have attempted to model themselves after successful private industry services. One such project is Eucalyptus Cloud [8], which provides API-compatible web services to Amazon Elastic Compute Cloud (EC2) [9] and a very basic web interface to managing VMs and storage resources. Other projects aim to be more of a toolkit or API to service, such as Nimbus [10] and OpenStack [11]. Most of these projects cater more toward service providers who wish to build a cloud rather than delivering the cloud more directly to the end-users themselves. A major distinction between Atmosphere and these other projects is that Atmosphere attempts to close the usability gap between a cloud provider and that of cloud users, particularly for biologists and plant science researchers.
3.
ARCHITECTURE
Conceptually, Atmosphere can be separated into three logical layers accompanied by a set of toolkits which reside within the virtual machine (VMs) (see Figure 1). The three layers of Atmosphere are the cloud engine, the middleware, and the web frontend. The toolkits facilitate the configuration of the virtual machines, communication with the middleware tier, and the interfacing with other parts of iPlant’s cyberinfrastructure.
Figure 2. Screenshot of Atmosphere home screen. VMs organized as "Applications".
Figure 3. Dashboard view to customize a virtual machine. The dashboard view provides fine-grained configuration options for expert users.
3.1
Cloud Engine
Atmosphere’s design planned for the interchange of multiple cloud types and cloud engines. Currently, Atmosphere supports Eucalyptus Cloud, an open-source project with API compatibility with Amazon Elastic Compute Cloud and other Amazon web services. At the time of Atmosphere’s inception, Eucalyptus Cloud was determined to be the most stable, feature rich, and extensibles. Integration with OpenStack is planned by early 2012.
3.2
Middleware
Atmosphere’s middleware is built on top of Django [12], a high-level extensible python web framework designed for rapid development. While having originated as a content management system, the Django provides an excellent foundation for web applications. Other python web frameworks were considered, but Django was considered the most mature and offered the most features when Atmosphere development began. Django’s core components include an object relational mapper and user interface facilities. Atmosphere’s middleware adds additional services for auditing and monitoring, resource and job scheduling, authentication, and metadata management.
Atmosphere’s authentication middleware service, named CloudAuth, is designed to accept different authentication schemes. Currently, it supports LDAP and an internal database. Once a user authenticates, tokens are used for subsequent API calls within the middleware. Authentication is discussed in more detail in the security section of the paper.
Auditing and monitoring is provided through extensive logging of most activities in Atmosphere and stored within Atmosphere’s internal database. Detailed activities include VM launches, VM terminations, storage requests, user logins, and user logouts. In addition, Atmosphere performs periodic polling of the cloud resources to perform resource consistency checks or to monitor for known failure states of the underlying cloud. For instance, a known failure state of Eucalyptus occurs when new VM cannot obtain an IP address. For failure states that cannot be automatically remedied, notifications are sent to cloud administrators.
Atmosphere requires internal scheduler to perform periodic tasks, such as monitoring and polling. This internal schedule was built using python celery [13]. Another important function of the scheduler is to throttle simultaneous requests to the cloud engine at times of high utilization, without which the cloud engine could be paralyzed with too many requests. Future plans for the scheduler include providing a means to launch VMs to backfill resources during periods of low utilization, maximizing the overall cluster efficiency.
Metadata management is facilitated through tags, descriptions, and derived elements provided by users or cloud administrators. Future versions of Atmosphere will include more intuitive means to manage and search metadata, and to link metadata information generated by other components of iPlant’s infrastructure.
3.3
Web Frontend
Atmosphere’s web frontend is a rich web 2.0 application, which extensively uses AJAX [14] and JavaScript [15]. Ext JS [16] provides much of the look and feel. Bidirectional communication between middleware and the client browser, such as event notifications, uses Faye [17], which implements the Bayeux protocol [18].
The primary motivation for the user interface design is usability for both inexperienced and expert cloud users. The home screen for Atmosphere allows users to review and launch virtual machines in an application store model, resembling modern mobile and touch devices (see Figure 2). Convenient access to VM and resource metadata is also provided using sidebars in the interface. For expert cloud users, detailed information and fine-grain access to the underlying cloud resources are available through the “Dashboard” (see Figure 3).
3.4
Virtual Machine Toolkits
Atmosphere bundles several tools, in the form of scripts and applications, through the virtual machine toolkits. Some tools are essential to the operation of Atmosphere, assisting with the communication between the Atmosphere middleware and the VM. Other tools enhance the usability of a VM within iPlant’s infrastructure or provide integration with iPlant’s compute and storage resources. The following are descriptions of the tools in the iPlant VM toolkits:
atmoCL – This program provides convenient access to select
Atmosphere web service functions, such as querying Atmosphere metadata and managing cloud-specific storage. A typical use for atmoCL is mounting EBS volumes onto a VM without using the web frontend.
Chart 1. This chart illustrates the cumulative growth in the number of users since Atmosphere’s public launch in January 2011.
atmo_init – atmo_init executes as part of the boot-time process
and facilitates the configuration of a VM before it is available to a user. Users will never need to access this tool directly.
condor tools – iPlant provides access to a small compute grid
based on Condor [19]. The condor tools are essentially the condor executables necessary to submit jobs to iPlant’s grid for some types of computations.
configuration management – Traditional configuration
management enables systems administrators to control and automate the configuration of system software and services. Atmosphere uses Puppet [20], a configuration management tool, to dynamically configure systems on virtual machines. Future versions of Atmosphere will enable users to dynamically configure their own VMs.
image_request – This command-line tool collects the necessary metadata from a VM, which will ultimately be displayed within Atmosphere’s graphical catalogue.
iPlant Data Store utilities – iPlant utilitizes iRODS Data Grid
[21] for large scale storage. Users have access to both command-line and GUI-based clients to manage data within iPlant’s Data Store.
4.
SECURITY AND AUTHENTICATION
A function of the Atmosphere middleware is to mediate both the cloud engine’s authentication and iPlant’s central authentication services. An authentication service, called CloudAuth, provides a pluggable framework to integrate with different authentication mechanisms via modules. Currently, CloudAuth supports an internal database or LDAP [22]. Planned authentication modules include CAS [23] and Shibboleth [24].
In a typical authentication use case, users authenticate to the web frontend. Secure sockets are used for every layer of network communication. When a user authenticates using the web frontend, a user’s iPlant credentials are mapped to the corresponding credential provided by the cloud engine. The cloud credentials allow the user to provision resources within their namespace, allowing Atmosphere to leverage the cloud engine’s mechanism for resource isolation.
Atmosphere web services APIs employ a simplified version of a token-based authentication system. After authentication, external services use a token to call methods on behalf of a user. Tokens have a finite lifetime, configurable by the cloud administrator. Users authenticate to their VMs primarily using their iPlant credentials. SSH access is automatically configured to allow ssh access to the specific user and by cloud administrators. Secure VNC [25] access is enabled using a RealVNC [26] Enterprise Server embedded on the VM.
5.
DATA
Plant science is a data intensive science [27]. To address their extensive data needs within the cloud, Atmosphere users have access to two types of storage. The first type is provisioned through the underlying cloud engine and is exclusive to a user’s virtual machines. The cloud engine’s storage is recognized by the VM as a native block device and uses typical system utilities for managing devices, such as fdisk, parted and mount.
The second type of storage is the iPlant Data Store. Users can pull, push, or synchronize data in parallel using iRODS command-line utilities or graphical clients. Another method of managing data is using a Filesystem in Userspace (FUSE) [28] client, which translates the iRODS API calls into filesystem calls. One important distinction to using the iPlant Data Store is that users can readily access the data across the entirety of iPlant’s cyberinfrastructure, including the HPC resources and the Discovery Environment.
6.
CLOUD UTILIZATION
When Atmosphere was launched as a “preview” to the public in January 2011, access was limited. Shortly after its initial launch, iPlant Atmosphere opened access to researchers. The diversity of current users represents 16 countries and 87 institutions. Within the United States, where 89.6% of the total users reside, 30 states are represented (see Chart 1).
Atmosphere has been utilized in three workshops and one graduate-level bioinformatics course. Atmosphere is in active use by five research laboratories to share image data using the iPlant data store and analyze it using custom command line and GUI based applications developed in MATLAB, these tools and dependencies were deployed as compiled binaries as a bundled VM; this custom VM is made available to the community to use with their own data sets through Atmosphere (See Figure 4).
7.
USE CASES
iPlant’s cyberinfrastructure provides multiple entry points to its storage and compute resources, where Atmosphere fills specific needs unmet by the other infrastructure services. The iPlant Discovery Environment provides a structured way for users to integrate tools and perform analysis via a web portal. Web service APIs, through the Foundational and Semantic APIs, programmatically expose this functionality to developers to integrate with their existing science portals and tools. Direct access to complex, large scale compute and storage is generally available to plant scientists through XSEDE providers, such as Texas Advanced Computing Center (TACC). Given these modes
Figure 4. A remote VNC connection to a Atmosphere instance showing the image analysis toolkit and iPlant data store windows.
of access, there was a need for highly configurable environments for algorithm exploration, tool development, small- to medium-sized analyses, or analyses that might not be traditionally suited for HPC environments.
The following provides a glimpse of the typical use cases that Atmosphere has used over the past several months since Atmosphere has been released to the public:
• Algorithm and Tool Development: Many of Atmosphere’s users do not have convenient access to a UNIX environment to design their own tools, whether via shell access or a graphic desktop. The cloud, with its self-service, on-demand model, is an obvious fit for these users. Typically, the period of algorithm and tool development is finite between releases, after which the environment may be saved or terminated until the next stage of development begins. In some cases, algorithm and tool developers publish their virtual machine images for other community users. One example is the Phytomorph VM, developed by Nathan Miller from the University of Wisconsin, which provides machine vision tools to correlate seed morphology to seed development.
• On-Demand, Standalone Analysis: As mentioned previously, some users utilize virtual machines, configured by tool developers, as part of their data analysis pipeline. In other cases, users have developed their own analysis pipeline want to deploy it virtually. A common problem mentioned by some users is that their analysis pipeline exists on a desktop or laptop, lacking the reliability or performance they need to scale their analysis. In many cases, users choose Atmosphere as a facility to share their analysis pipeline with other lab members or collaborators. In these cases, providing wholly contained, reproducible computational environments is the most attractive features of Atmosphere.
• On-Demand, Integrated Analysis: Integrated analysis refers to analysis that may be partially performed using other parts of iPlant’s infrastructure. For example, some users may have part of their analysis performed using the Discovery Environment and wish to further process the data using pre-installed tools on a virtual machine in Atmosphere. In other cases, Atmosphere VMs have been used to prepare an analysis to be later targeted for HPC resources. For integrated analyses, the iPlant DataStore is used for sharing data across iPlant’s various services.
• Education, workshops, and training: Another large category of users utilize Atmosphere for workshops and training events. Oftentimes, workshop organizers need pre-configured VMs, loading with sample data, for their participants. The iPlant staff works closely with workshop organizers to structure their environments and sample data.
8.
CONCLUSION AND FUTURE WORK
The goal for Atmosphere is to provide ease of access to highly customizable computational infrastructure, functioning as a gateway that integrates and augments cloud resources with capabilities such as HPC and data grids. Atmosphere also serves as a platform to allow computational tool designers and developers the ability to collaborate, and rapidly deploy their analysis pipelines for broad use by the community. Atmosphere plays a key role in providing customized computational resources for pre and post analysis; e.g. Atmosphere has custom VM of eight popular genome visualizers which are used to visualize a large genome assembly, the compute intensive tasks were performed on a HPC system where these resources intensive GUI applications are not typically installed. Atmosphere’s ability to provide easy web interface to preserve data, tools, and workflows with minimal effort and skill overcomes the limitations and technological barriers that prevent adoption of the cloud.
Deploying a cloud for the plant sciences hasn’t always been smooth. One of the most salient challenges initially faced when the project began was selecting the best cloud technology to use at a time when cloud technologies were emergent. Selecting a dominant cloud technology was more comparable to hedging a bet than making a thorough competitive analysis. To move forward with the project, we selected the best technologies at the time with a design philosophy accounting for the fact that that the underlying technologies would rapidly change and most likely be replaced in the future.
Current and ongoing development module / features include:
• OpenStack and public cloud integration; replacing the use of euca2ools python library with a more generic, flexible library, such as Apache’s Libcloud [29].
• Utilization-based scheduling of resources, including backfilling instances during low utilization periods
• Multi-cloud support; the scheduling and provisioning of resources across multiple, geographically dispersed clouds
• Tighter integration with iPlant Discovery Environment, Grid, and HPC resources
• Automated, user-initiated VM image bundling
• Expansion of support for mobile devices. Currently, an Android application is available to view, launch, and terminate instances.
• Authentication with common academic institutional authentication standards, such as Jasig Central Authentication Service (CAS) or Shibboleth; Integration with InCommon [30].
• More intelligent metadata management and search capabilities; integration with semantic approaches to metadata search.
9.
ACKNOWLEDGMENTS
The iPlant Collaborative is funded by a grant from the National Science Foundation Plant Cyberinfrastructure Program (#DBI-0735191).
10.
REFERENCES
[1] PSCIC. (2006) Plant Science Cyberinfrastructure
Collaborative. [Online]. http://www.nsf.gov/publications/pub_summ.jsp?ods_key=n
sf06594
[2] Armando Fox, "Cloud Computing - What's in it for Me as a Scientist," Science, vol. 331, no. 6016, pp. 406-407, 28 January 2011.
[3] P. Khatri and S. Drăghici, "Ontological analysis of gene expression data: current tools, limitations, and open problems," Bioinformatics, vol. 21, no. 18, pp. 3587-3595, June 2005.
[4] Xen. [Online]. http://xen.org
[5] VMware. [Online]. http://www.vmware.com
[6] Kernel-based Virtual Machine. [Online]. http://www.linux-kvm.org/
[7] J. Fontan, T. Vazquez, L. Gonzalez, R. S. Montero, and I. M. Llorente, "OpenNebula: The Open Source Virtual Machine Manager for Clouster Computing," in Open Source Grid and Cluster Software Conference, San Francisco, CA, May 2008.
[8] D. Nurmi et al., "The Eucalyptus Open-source Cloud-computing System," in CCA08: Cloud Computing and Its Applications, 2008.
[9] Amazon. Amazon Elastic Compute Cloud. [Online]. http://aws.amazon.com/ec2/
[10] K. Keahey and T. Freeman, Nimbus or an Open Source Cloud Platform or the Best Open Source EC2 No Money Can Buy, Supercomputing 2008; invited talk.
[11] OpenStack. [Online]. http://www.openstack.org/
[12] Django Software Foundation. Django. [Online]. https://www.djangoproject.com/
[13] Python Celery. [Online]. http://celeryproject.org/
[14] AJAX. [Online].
http://en.wikipedia.org/wiki/Ajax_(programming)
[15] JavaScript. [Online].
http://en.wikipedia.org/wiki/JavaScript
[16] Sencha Inc. Ext JS. [Online].
http://www.sencha.com/products/extjs/
[17] James Coglan. Faye. [Online]. faye.jcoglan.com
[18] Alex Russell, Greg Wilkins, David Davis, and Mark Nesbitt. (2007) Bayeux Protocol -- Bayeux 1.0.0. [Online]. http://svn.cometd.com/trunk/bayeux/bayeux.html
[19] M. Livny, M. Litzkow, and M. Mutka, "Condor - a hunter of idle workstations," in Proceedings of the 8th International Conference of Distributed Computing Systems, IEEE Computer Society Press, Madison, Wisconsin, 1988, pp. 104-111.
[20] J. Lope, Managing Infrastructure with Puppet, 1st ed. Sebastopol, CA, USA: O'reilly Media, Inc., 2011.
[21] A. Rajasekar et al., iRODS Primer: Synthesis Lectures on Information Concepts, Retrieval, and Services.: Morgan & Claypool Publishers, 2010, vol. 2.
[22] J. Sermersheim et al., "Lightweight Directory Access Protocol (LDAP): The Protocol," Internet Engineering Task Force, Network Working Group, RFC 4511, June 2006. [Online]. http://tools.ietf.org/html/rfc4511
[23] Jasig. Central Authentication Service. [Online]. http://www.jasig.org/cas
[24] Internet2. Shibboleth. [Online]. http://shibboleth.internet2.edu/
[25] T. Richardson, Q. Stafford-Fraser, K. Wood, and A. Hooper, "Virtual Network Computing," Internet Computing, IEEE, vol. 2, no. 1, pp. 33-38, Jan/Feb 1998.
[26] RealVNC Ltd. RealVNC. [Online]. http://http://www.realvnc.com
[27] A. Smith et al., "Biology and Data-Intensive Scientific Discovery in the Beginning of the 21st Century," OMICS, vol. 15, no. 4, pp. 209-211, 2011.
[28] Filesystem in Userspace. [Online]. http://fuse.sourceforge.net/
[29] Apache Libcloud. [Online]. http://libcloud.apache.org/ [30] InCommon. [Online]. http://www.incommon.org [31] boto. [Online]. http://code.google.com/p/boto/
