OCCI at the OpenStack Design Summit

Hot on the heels of the OCCI OpenStack implementation (wiki, code review) a number of our community members (big thanks to Eugene from R2AD) will be organising an OpenStack Design Summit unconference OCCI session. All are welcome to it from the inquisitive to the sceptical!

Topics to be discussed include:

  • What is OCCI and its goals?
  • Where does OCCI fit the OpenStack picture?
  • How should OpenStack address “extra” APIs?
  • Q&As

The session will be on Wednesday at 4.30pm.in the Golden Gate room.

rOCCI – A Ruby OCCI Framework

rOCCI is an OCCI 1.1 compliant server implementation written in Ruby. Through its modular architecture it can be easily extended to support arbitrary Cloud Frameworks.

rOCCI as mentioned in our last blog post, is the OCCI implementation that powers the work which enables OCCI interoperability with Amazon EC2.

Work on rOCCI started as part of the EU FP7 project SLA@SOI under the working name OGF-OCCI in early 2011. The goal was to develop an OCCI 1.1 compliant server which initially supports OpenNebula and can later be easily extended to support other Cloud Frameworks. As OpenNebula offered a robust Ruby API binding, it was decided to develop the project in Ruby as well. Soon after work on the implementation started it was accepted as an official OpenNebula Ecosystem project.

Interoperability with other implementations of the OCCI standard was ensured by taking part in testing sessions during the SNIA Cloud Plugfests and using OCCI compliance tools such as the OCCI ANTLR Grammar and the DoYouSpeakOCCI tool. For the last Plugfest taking place at the end of February 2012, the project was renamed to rOCCI in order to distinguish it from the name of the standard (OGF-OCCI).

The current architecture of rOCCI is shown in the following overview:

Detailed architecture showing the components of rOCCI

Current features include:

rOCCI is currently mainly developed by employees of GWDG. New contributors are very welcome. For more information visit:

Home of rOCCI

rOCCI on GitHub


OCCI’fied Amazon EC2

OCCI has been successfully mapped and implemented upon the Amazon EC2 API. The work has been carried out by TU Dortmund University in cooperation with the compute and research center GWDG. The implementation uses the rOCCI framework – more on that in a later post!

Screencast EC2 in rOCCI from Max Günther on Vimeo.

 

German Government Recognises OCCI as Leading the Cloud Standards Arena

Today, we are happy and proud to find the German Federal Minstry of Economics and Technology (BMWi) fully endorses OCCI in its freshly published analysis on “The Standardisation Environment for Cloud Computing”.

This is a big vote of confidence in our work which was echoed by the UK G-Cloud report. Being picked up by the German government is obviously kudos to the community’s work, especially as it shows OCCI as a leader of current Cloud standardisation activities.

Lately, OCCI has received a lot of attention: the leading open-source cloud computing software stack implements it, a commercially focused project and related companies builds its whole ecosystem around it, the largest eInfrastructure provider in Europe endorses it, and many other things are ongoing around it.

The study identifies 19 standardisation organisations as “leading”, among which well-known international ones such as ISO, NIST, SNIA, and DMTF are listed next to OGF, the home of OCCI, and other European ones, such as ETSI and BITKOM.

Regarding standards, the report features “20 prototypical cloud standards [that] serve as models, […] and are greatly respected by experts”. Looking at this in detail, it is hugely encouraging to see that OCCI is considered to be the one with the greatest importance (together with OpenStack, which is on the way of speaking OCCI as well, and OAuth, which is orthogonal): the matrix above shows the classification by Booz and Partners on behalf of the BMWi. If you take a look on the upper right, you’ll find that OCCI is not only the one with the greatest maturity and quality, but also has the highest dissemination potential!

The report makes an analysis from the European and German point of view, and discusses the current field of standardisation in the Cloud arena. Stating that “the standardisation environment for cloud computing is only just starting to develop”, the report identifies OCCI (next to OVF, OpenStack, and CDMI) as “proving attractive”. It features a taxonomy of standards in cloud computing along challenges they address (“why?”) and the basis of their approach (“how?”), and identifies nine challenges, with data privacy the most prevalent, next to three fields (technology, management, and legal).

Well done to all and thanks to Alexander for the article and translations!

Do You Speak OCCI?

The doyouspeakOCCI Compliance Testing Facility is a Google App Engine (GAE)-based checking tool for the Open Cloud Computing Interface (OCCI) family of specifications. More specifically, it provides a full compliance test suite for the OCCI Core (GFD.183)OCCI Infrastructure (GFD.184), and OCCI RESTful HTTP Rendering (GFD.185)specifications.

doyouspeakOCCI is written in Python and heavily building on the GAE services, mainly Task QueueURL Fetch, and the webapp Framework.

Note that doyouspeakOCCI is not to be considered the “official” testing suite for OCCI endorsed by the Open Grid Forum, but rather than that, a third-party contribution which aims to be as close as possible. For a more thorough explanation, please take a look at the wiki pages.

How to Use

doyouspeakOCCI was hard to implement, but is simple to use. Just point your browser tohttp://doyouspeakocci.appspot.com, enter the base URL of your OCCI implementation, and press “Go!”.

Optionally, you can provide credentials for HTTP basic auth, if your service is secured. We strongly recommend to use a one-time test account; although we promise to use the credentials only for the compliance test, we cannot guarantee what others on the way (especially GAE) will do with them. In the near future, doyouspeakOCCI will support OAuth to ameliorate this issue.

Please note that doyouspeakOCCI records data on every test run in the GAE DataStore. This is done solely for the sake of displaying usage statistics. Within the limitations of applicable jurisdiction and the GAE Terms of Service, we will not disclose this data to anyone beyond what is being displayed on the doyouspeakOCCI web presence.

For other questions, please also take a look at the FAQ.

Where to Get

doyouspeakOCCI is available as a source code release only, which can be obtained by two ways:

Alternatively, you might want to pick one of the advertised downloads (click on the “Downloads” button in the upper right of the doyouspeakOCCI development home page at GitHub.

If you wish to run the service on your local system for testing purposes, please take a look at the doyouspeakOCCI Installation Guide for a detailed explanation on how to setup the environment.

Contributing

doyouspeakOCCI aims to be a community effort, and help is always welcome. Please contact us on the mailing list to learn more.

License

We think that doyouspeakOCCI should be available to everyone with the utmost amount of freedom. To make sure that contributions to doyouspeakOCCI itself remain perpetually free, the code has been developed under the GNU General Public License, Version 3. The documentation coming with doyouspeakocci is available under a Creative Commons Attribution Share-Alike 3.0 License.

Grokking OCCI Syntax: OCCI ANTLR Grammar

Interoperability is key belief in OCCI. The OCCI specifications set out in text how to implement the API and formats, however it is still up to our community and adopters to implement this. It is often the case when implementing a specification certain implementation decisions made can have an impact on an implementations interoperability. It is for this reason why this ANTLR grammar has been created (thanks to SLA@SOI) to aid developers in creating their parser. The ANTLR grammar specifies the text rendering format in an abstract language known as a grammar.

What?

The grammar defined, when used with the ANTLR tools, will generate a lexer and parser that will validate any valid OCCI text format. The grammar itself does not currently check whether a value associated with an attribute is valid. Rather it primarily ensures that the structure of the request is valid. It is still up to the implementer that service behaviours (e.g. correctly responding to HTTP GETs) are implemented correctly.

Why?

One of the key strengths of ANTLR, outside of its lexer/parser technology, is it has a wide range of target languages (18), so your favourite language has a good chance!  This is an important advantage as this enables multiple implementations of OCCI (client or server) to all share a the same rules that parse the OCCI text format, regardless of language.

The Goodies

There are two grammars currently defined, one based on the other:

  1. occi-antlr-grammar: this solely defines the grammar and does not contain any target language specifics. It can be used as the foundation of target language specific OCCI grammars. An example of this extension is the occi-antlr-java. This grammar can generate lexer and parsers, however they will only validate the input and not extract values from the supplied input. If extracted values are required, a very common case, then the occi-antlr-java grammar gives an example of this.
  2. occi-antlr-java: this extends occi-antlr-grammar to include target language specifics. The target language used is Java. In this grammar file there are Java-specific ANTLR actions that extract the values from a valid OCCI text request/responses.
  3. occi-antlr-ruby: this just like occi-antlr-java extends the occi-antlr-grammar to include ruby language specifics. This was contributed by GWDG.

There are a number of rules that are present within the OCCI grammar that can be reused in order to validate certain supplied values. This would describe a second pass parsing phase. In the case that an implementer would like to validate a URI value then they can do so by using this URI grammar.

The following presentation also goes through some details related to ANTLR and the OCCI grammar.

We hope you find this useful! If there are any problems, issues or suggestions open a ticket.

Performance testing

One of the more recent questions was: how many overhead does OCCI add with regards to performance. The following diagram shows the ‘Requests per Second’ as well as the ‘Time per request’ from one of the python based OCCI implementations. Measured where only the times is took for an OCCI framework to parse and deal with a request to show that OCCI barely adds to the performance of the overall system. As the concurrency level rises the peak number of request can easily reach 4000 request per second whereas the time per request varies from 0.2 to 0.3 milliseconds.

An Open, Interoperable Cloud

A number of OCCI members have got together and have written this piece in InfoQ.com on how an open and interoperable can be presented to end-users using today’s contemporary cloud standards, including not only OCCI but also CDMI and OVF. The article is the result of the excellent meetings had at the DMTF APTS back in May.

OCCI Presentation – A Walkthrough of the Specification

The following slide-set gives an overview (incl. examples) of the OCCI specification:

Please see the ‘speaker notes’ for more information on each slide. The presentation of course can be shared upon request in several formats. Feel free to contact the chairs of the OCCI working group.