OW2Con2011 Videos are online

The videos of all the OW2Con2011 have been published to the OW2 Youtube channel. My talk about Petals BPM and The Cloud is also available.

You are right, I need to smile more, be less tired and have a demo of the BPM editor working on low resolution displays… BTW, the demo of the DSB Monitoring & Management console used to deploy and monitor BPEL process works.

Playing in Brussels

Last week was the first annual review meeting of the Play project I work on since one year. I am involved at several levels in this project: from the architecture point of view, to the software integration and quality ones. On my side, my goal is to provide the efficient software infrastructure for events actors, or how to build an Event Driven Architecture based on Petals Distributed Service Bus. There are others points which have to be developed, especially all the platform governance stuff and Service Level Agreement for events, what we call Event Level Agreement.

We showed several things to the European Commission reviewers and we were also able to show an early prototype (this one was originally planned to be show at mid project ie in 6 months…). I made a video capture of the demo, which really needs to be explained…

  • There is the idea of a market place for events : The event marketplace. From there users are able to subscribe to things they are interested in. For now it is just a simple Web application which subscribe on behalf of the user to events through topics. This subscription is sent to the Play paltform using Web standards. Here we use OASIS Web service Notification to create this communication link.
  • Events are collected from several sources by events adapters. In the video above, we can see the user setting this FB status which is collected by the Play system and transformed into a notification which is published to the platform. There are also some Twitter adapters and Pachube ones which collects data in real time and publish them to the platform. This time again, we use Web standards for adapters communication.
  • Now, what happens in the video? The user logs in to event marketplace and subscribes to FB events. The user then publishes something to its FB wall (note that it is not mandatory to have the same user in the event marketplace and in FB. A user can subscribe to FB events and receive events from all the FB statuses collected from the FB application users). After the event propagation delay, the event marketplace display the event to the user.

So do we need such machinery to do things like that? No, if you want to do some other simple mashup portal. There are several components which are under active development: Storage and processing. Yes we store all events in the Play platform. This storage will be huge, but it is one of the project goal: Providing efficient and elastic storage in some P2P way. The need for this storage comes with the other important aspect of the project: Complex Event Processing. We will soon be able to create complex rules on top of events and be able to generate notifications based on past and real time notifications because we have efficient storage and real time stuff inside the platform. I am not an expert of this domain, so I can not give more details about that point but capabilities are huge! For example, we can express something like “Hey Play, can you send me a SMS me when there is my favorite punk rock band playing just around and I am not on a business trip and X and !Y or Z”. All of this intelligence coming from the processing of various sources I push since months in the platform coming from Twitter, FB, last.fm and other data providers.

Now let’s take some time to work on my OW2Con talk. The session name is pretty cool : Open Cloud Summit Session.

Service Bus Live Monitoring

I explained in the last articles how I tested the Play Framework, Web sockets and how I integrated all this nice stuff with a real example based on a Service Bus, Web services Notifications, etc…

This time, let’s go one step further. We have a Service Bus which is Web service notification enabled like last time. We can bind services to the bus, expose service endpoints as Web services, blahblahblah… But, this time, I am interested on having some real time monitoring of service invocations. It means that each time a message goes through the service bus (a service invocation in fact), I want to know (almmost) immediatly the service response time.
Hopefuly, the PetalsLink Distributed Service Bus I develop and use provides many extension points. One is the capability to add modules to the routing engine ie the software module each message must be able to go through on service request and response. So adding some router module which catch all the messages, timestamp them and then send this monitoring data to someone is quite easy. At the implementation level, this monitoring router module publishes monitoring reports to the service bus notification engine topic dedicated to monitoring stuff.

So, a client interested in monitoring data just has to register itself as subscriber to the monitoring notification topic. Every time a message is published in the topic, it will be delivered to all the subscribers. Up to the subscriber to display data as soon as it can. This is where Play, Web sockets and some cool javascript library came in. Since I never developed javascript stuff, I tried to find an easy to integrate solution to create some moving plots, asking twitter. I finally found the Smoothie Chart library which is really easy to use and updates graph in real time.

The high level architecture of the system can be defined as

The following video shows the result of the complete stack: Each time a message a service is invoked with SOAPUI, a Web service notification is sent to a Play application which subscribed to the monitoring topic, the Play application then pushes the data to the client by using a Web socket. Finally, the javascript code on the client side feeds the Smoothie chart which updates automatically. At the end, it is quite simple and efficient.

Oh, I forgot to say something: This took me 2 or 3 hours to create all this stuff… The code has been published on github in the dsbmanager-webapp project.

Create a management application in 3 hours with Play

Story, code, compare

Yet another ‘nightly project’ (thanks to current house build project and the lack of sleep it brings). This time I needed to be able to manage the so-famous Service Bus from some Web enabled tooling. I already developed such tool in a research project but the fact is that the licence of some libraries are not compatible with the petalslink open source approach. The second thing is that it is GWT based (which bores me, has almost 100 libraries dependencies and takes 10 minutes to compile). So this application is a good candidate to compare development productivity between GWT and the Play Framework. Here is a summary of the application creation:

  • I bootstraped the application yesterday during lunch between 1PM and 2PM. I was already able to invoke most of the interesting service bus actions with the help of service bus SOAP API.
  • I added some pages and actions last night, let say that since it was between 11PM and 1AM I was not very productive…
  • I fixed some bugs this morning

As a result, I think I worked around 3 hours on this application. I think I spent one hour to resolve a dependency conflict between Play and a CXF dependency but as a result I have a good result which is almost equivalent in functionality to the GWT based application. I still miss some operations but I do not need them for now… I feel ashamed to say how long it tooks me to create the GWT version…

Deploy

Let’s talk about deployment… I did not have time to play with heroku to push this application in the cloud. Tis will be a future step but since I use Play and git, I am able to push the code to github and then pull it on an OVH server I rent. I am able to provide this instance for some project partners if they want to manage the service bus. The time it took? 5 minutes (wget play, git clone and play start…).

Code

The Play enabled application is available on github at chamerling/dsbmanager-webapp.

Some Play Framework, Service Bus, WS Notification and Web Sockets…

The Context

In the previous post I was introducing some tests I did with Play Framework and Web sockets. To summarize, it was just ‘about’ receiving messages on the Play! application and pushing them to the browser. This time, let’s go one step forward: Let’s add some infrastructure stuff to do something more real…

In the current case, I want to introduce a service bus which allows to create a real integration between service consumers and producers (I need to write another article in response to this nice videocast about integration from Zenexity guys ‘EAI & ESB n’apportent rien si les applications ne sont pas intégrables et interopérables’, but I really need more time to explain my thoughs…).
Using a service bus in the current case (and not for this case in fact…) must bring some added value. Here I choose to show that a service bus, even if it is not so lightweight at all, can provide some real cool features that you can have out of the box. Now, let’s add some event stuff to switch to an event-based world where we can have tons of event producers and let’s say thousands of event consumers. Since I can not setup such hude amount of actors, let’s say that we have one event producer and two event consumers:

  • The event producer wants to publish some stuff somewhere. To illustrate, let’s say that we have a weather sensor connected to our platform.
  • The platform provides a list of topics which producers can use to publish data. One is the weather topic which will be used by the producer above.
  • The event consumers want to be notified on new weather data i.e. as soon as the weather sensor publish new data. To keep it simple, they need to subscribe to the weather topic provided by the platform.

To recap, in the event context, the event producer only knows the service he has to push weather data to, the event consumers just have to subscribe to a topic they are interested in. All the knowledge stuff about producers, consumers, topics and all the mapping is delgated at the service bus level. Yes, true it is exactly like in some topic-based messaging stuff because at the end of the day, it is topic-based messaging stuff…

The Stuff

Now we can speak about the stuff we are going to use in the software point of view for notificaiton actors…

  • The event producer will not be a sensor but a Play! application. The application sends Web service notifications message to the service bus on a given topic.
  • The service bus is (of course) the Petals Distributed Service Bus with some Web service notification modules inside.
  • The event consumers are 1/A Play! application exposing a Web service to receive notification it subscribes to and 2/ A local java application displaying OS X notifications using Growl (let’s use JavaGrowl I published some days ago…). Note that the Play! application which have subscribed to notifications pushes them to their clients (browsers) using the funky Websocket stuff.

Let’s look at what really happens in a short video:

  • I use the Play! powered application play-soap-wsnclient to subscribe to notification on behalf of the play-soap-websocket Web application.
  • In Eclipse, I start a Web service notification powered Web service which subscribes to the same notification topic. Its listener is configured to use JavaGrowl to display incoming notifications.
  • I use play-soap-wsnclient to send notification messages to the notification service hosted on the service bus.
  • Once received, the service bus forwards the notification to all the subscribers using internal routing and WSN stuff.
  • The play-soap-websocket Web application receives a notification and push it to the client browser using Websocket.
  • At the same time, the Java application also receives a notification and display it using Growl.

One (or more) step(s) further…

And what if we have something which is not a Web service which subscribes to notifications? With the help of a service bus like Petals ESB/DSB, we just have to add a component which knows how to speak with the subscriber. For example, let’s say that SOAP is bad and that REST is the best thing ever. Can we have REST services receiving notifications? Yes, we can! Let’s just add the REST connector to the service bus. Another protocol/transport/format? Develop and add the new one. This is where the service bus can also help you. Hopefully, there are also other things which are possible with a service bus, we will see it later in other posts if I have time (as usual): Let’s think about business processes, service orchestration, transformation…
Next time we will have a look on what we can do if we use the distributed feature of the service bus, for example, receiving some notifications on one node and be able to notify subscribers which are bound to other nodes…

Source Code

Petals BPM

Petals BPM (previously Geasy BPMN Editor) is an open source, web-based BPMN 2.0 editor developped with GWT.

Petals BPM

Petals BPM

Tout est dit en une seule phrase… Petals BPM est un des produits que nous sommes en train de développer activement au Labs. Cette Web application open source, développée en GWT permet de créer et d’éditer des process BPMN 2.0, d’importer et d’exporter en BPMN 2.0, XPDL, mais aussi exporter vers un fichier BPEL 2.0.

Beaucoup de choses à venir autour de ce produit, j’espère assez rapidement. Ca sera bien sur décrit ici.

PetalsLink DSB expliqué

Dédramatiser le Bus de Service que je développe depuis 2 ou 3 ans, tel est le but de ces quelques slides que j’ai présenté à l’équipe R&D la semaine dernière…

 

PLAY Project Video

Le projet PLAY ne déroge pas à la règle de la vidéo explicative et voici donc la première expliquant un scénario d’utilisation avec le fameux Paul…

Marre du Google Summer of Code? Essayez le OW2 Programming Contest

Il est temps de changer un peu de jeu, c’est l’heure de l’OW2 Programming Contest… Bien que le Google Summer of Code soit plein d’avantages que ne peut pas fournir un consortium comme OW2, les aspects techniques pour les candidats sont tout aussi intéressants. Rien à gagner, pas de primes, de cadeaux, ou de voyage payé pour les States (quoi que, pourquoi ne pas payer un trip à Paris aux gagnants pour présenter leur travail lors de la prochaine conférence annuelle à Paris?), mais ne devrait-on pas dire que être gratifié sur un sujet pour une communauté open-source en vaut déjà la peine?

The purpose of the competition is to develop awareness for the OW2 code base among students and technicians, and to provide an opportunity for contestants to demonstrate their talent in computer programming. What’s more, the contest aims at promoting teamwork and the ability of college students in science and technology to study and use OW2 open source projects. Based on principles of “freedom, sharing and creativity”, and aiming at improving international communication and cooperation, the competition is overseen by IT professionals and academics from around the world.

Bref, j’ai soumis deux sujets à l’OW2 programming contest :

  1. Petals ESB et IaaS, ou comment interfacer un bus de services avec une couche IaaS pour qu’il en tire les avantages de la couche infrastructure (je pense principalement à l’élasticité).
  2. Console Web de Monitoring & Management pour Bus de Services, le but étant de développer une Web console un minimum dynamique avec GWT et ses extensions. Comme base de travail, une version utilisant GWT-ext qui n’est pas conforme à nos attentes au niveau licence sera fournie.

Il y a une tonne d’autres sujets proposés par les partenaires (dont beaucoup sur Jonas, Jasmine et compagnie), jetez-y un coup d’oeil, ça vaut le coup de voir les idées qui trottent dans la tête des gens.

Pensées sur le Cloud Hybride

Dans la galaxie de termes et de technologies que contient le Cloud Computing aujourd’hui, attardons nous un peu sur le Cloud Hybride. Un Cloud Hybride est grosso-modo un mix entre un Cloud public (du type EC2) et un Cloud privé (managé par un Framework à la OpenNebula).

Je pense vraiment que ce genre d’approche est la plus intéressante à mettre en oeuvre. Il est clair que les entreprises ne sont pas prêtes (et ne le seront sûrement jamais) à basculer tout leur système dans le Cloud public. On pense principalement au problème qu’il peut y avoir au niveau de la sécurité des données. Par contre, on peut imaginer qu’une partie des ressources (au sens large) peut potentiellement être déportée dans un Cloud public. Dès lors que l’on fait cohabiter du Cloud public et du Cloud privée, on passe donc dans le Cloud hybride. Simple? Pas tout a fait. Bien que le concept soit à mon avis le plus intéressant, il est aussi le plus dur à mettre en oeuvre.
Heureusement, des frameworks Cloud, tels que OpenNebula (brièvement présenté dans un précédent article) permettent de créer et de gérer des Clouds hybrides de façon transparente. Dans un de leur récent article, l’équipe qui est dérriere OpenNebula présente quelques cas d’usages qui répondent au besoin du Cloud hybride. Dans ‘Scaling out Web Servers to Amazon EC2‘, ils présentent comment on peut load-balancer une application Web en utilisant des ressources privées et publiques sur Amazon EC2. L’intérêt est ici de pouvoir fournir une qualité de service ‘constante’ en utilisant des ressources privées et d’avoir la possibilité de répondre à des pics de charge momentanés en utilisant des ressources publiques louées ponctuellement.

Cette approche est un bon compromis. Je ne crois vraiment pas au ‘tout public’. En me plaçant au niveau purement technique, je n’y vois aucun intérêt et si c’était le cas, Petals* serait déjà dans le Cloud depuis un bon moment. Par contre au niveau business, on pourrait alors dire que l’on fait du Cloud comme certains de nos concurrents…

Une approche SOA dans le Cloud hybride est donc plus dans la mouvance de ce que je veux pousser et développer cette année. Construire un bus de services pouvant s’adresser et utiliser les fonctionnalités du Cloud hybride me parait vraiment un bon challenge, et qui plus est, qui peut le plus peut le moins…

Ca a l'air compliqué, mais je vais quand même y aller

Ca a l'air compliqué, mais je vais quand même y aller

Mises à jour Twitter