Simple HeartBeat Manager with Play

One more time, one more tiny (and maybe useful in some cases…) application with Play!.

WTF, my server is dead again!?

This app is a simple heartbeat manager looking at remote HTTP services and notifying you by email when something becomes unreachable. It uses the background Job feature of the Play framework and just does HTTP GETs on the specified list. That’s all, that’s simple, but that’s can be useful sometimes…

The code is located at https://github.com/chamerling/heartbeat-manager

Pushing your Web services in the Cloud in 5 minutes…

… or less! Heroku is defined as a “Cloud application platform”. I just want to redefine it to “Awesome Cloud application Platform”. So, this awesome platform provides a way to host and scale your application in the Cloud really easily with 3 or 4 commands…

Since I am currently working on my talk at #OW2Con 2011 (coming later this week) dealing with BPM, Services and the Cloud, I wanted to host some Web services on several places. I never had time to test Heroku but I just took this precious time today. After looking some examples, I created a Maven project template (no I do not have time to create an archetype, maybe there is one somewhere) which uses Jetty and Apache CXF to expose JAXWS annotated classes as Web services. So now, using heroku to freely expose your services is easy as:

  1.  Sign up to heroku
  2. Download the heroku client for your platform
  3. Clone/Fork the repository at https://github.com/chamerling/heroku-cxf-jaxws
  4. Add your own services
  5. Login to heroku ‘heroku auth:login
  6. Create the app on heroku ‘heroku create -s cedar
  7. Push your services to heroku ‘git push heroku master‘. There is a git hook somewhere which just automatically compile and start your application after you pushed it.
  8. Open your CXF services summary page ‘heroku open’
The default application name is some random one, you can rename it by using the ‘heroku rename yournewname‘ but in the current case I had an issue on the generated Web service endpoint name. So I suggest restarting your app after renaming (have a look to the ‘heroku ps‘ command).
That’s all, that’s quick!

Open Source Tweet Wall

I am a big fan of Twitter and last week devoxx live tweet wall (http://wall.devoxx.com/) just made me want to create an open source live tweet wall. So last saturday night, I took some time to code it using the now well known Play! framework, some Web sockets and Twitter4J which supports the Twitter Stream API. This streaming API is really nice and provide a way to be notified almost instantly according to what you decided to listen to. After less than one hour (most of the time was consumed to read to Twitter4J documentation), the result is really fun. In the video below, I configured the wall to catch all the tweets containing ‘Apple’ and ‘Microsoft’ terms (I am not a MS supporter, but OMG, there are lot of tweets about it… probably only jokes and bugs…), and tweets are displayed in live on the wall (no I am not moving anything, the page is populated through a Web socket)

The resulting prototype is quite simple but works, and since I am not a front-end developer at all, it uses Twitter Bootstrap as CSS… The code is available on GitHub at https://github.com/chamerling/play-twall and is not perfect at all but just works: Twitter configuration is available in the Play! application.conf file and the Twitter connection is created in the Bootstrap…

Looks like I should start creation sort of “Coding a nice thing in one hour” collection…

Service Bus Live Monitoring

I explained in the last articles how I tested the Play Framework, Web sockets and how I integrated all this nice stuff with a real example based on a Service Bus, Web services Notifications, etc…

This time, let’s go one step further. We have a Service Bus which is Web service notification enabled like last time. We can bind services to the bus, expose service endpoints as Web services, blahblahblah… But, this time, I am interested on having some real time monitoring of service invocations. It means that each time a message goes through the service bus (a service invocation in fact), I want to know (almmost) immediatly the service response time.
Hopefuly, the PetalsLink Distributed Service Bus I develop and use provides many extension points. One is the capability to add modules to the routing engine ie the software module each message must be able to go through on service request and response. So adding some router module which catch all the messages, timestamp them and then send this monitoring data to someone is quite easy. At the implementation level, this monitoring router module publishes monitoring reports to the service bus notification engine topic dedicated to monitoring stuff.

So, a client interested in monitoring data just has to register itself as subscriber to the monitoring notification topic. Every time a message is published in the topic, it will be delivered to all the subscribers. Up to the subscriber to display data as soon as it can. This is where Play, Web sockets and some cool javascript library came in. Since I never developed javascript stuff, I tried to find an easy to integrate solution to create some moving plots, asking twitter. I finally found the Smoothie Chart library which is really easy to use and updates graph in real time.

The high level architecture of the system can be defined as

The following video shows the result of the complete stack: Each time a message a service is invoked with SOAPUI, a Web service notification is sent to a Play application which subscribed to the monitoring topic, the Play application then pushes the data to the client by using a Web socket. Finally, the javascript code on the client side feeds the Smoothie chart which updates automatically. At the end, it is quite simple and efficient.

Oh, I forgot to say something: This took me 2 or 3 hours to create all this stuff… The code has been published on github in the dsbmanager-webapp project.

Create a management application in 3 hours with Play

Story, code, compare

Yet another ‘nightly project’ (thanks to current house build project and the lack of sleep it brings). This time I needed to be able to manage the so-famous Service Bus from some Web enabled tooling. I already developed such tool in a research project but the fact is that the licence of some libraries are not compatible with the petalslink open source approach. The second thing is that it is GWT based (which bores me, has almost 100 libraries dependencies and takes 10 minutes to compile). So this application is a good candidate to compare development productivity between GWT and the Play Framework. Here is a summary of the application creation:

  • I bootstraped the application yesterday during lunch between 1PM and 2PM. I was already able to invoke most of the interesting service bus actions with the help of service bus SOAP API.
  • I added some pages and actions last night, let say that since it was between 11PM and 1AM I was not very productive…
  • I fixed some bugs this morning

As a result, I think I worked around 3 hours on this application. I think I spent one hour to resolve a dependency conflict between Play and a CXF dependency but as a result I have a good result which is almost equivalent in functionality to the GWT based application. I still miss some operations but I do not need them for now… I feel ashamed to say how long it tooks me to create the GWT version…

Deploy

Let’s talk about deployment… I did not have time to play with heroku to push this application in the cloud. Tis will be a future step but since I use Play and git, I am able to push the code to github and then pull it on an OVH server I rent. I am able to provide this instance for some project partners if they want to manage the service bus. The time it took? 5 minutes (wget play, git clone and play start…).

Code

The Play enabled application is available on github at chamerling/dsbmanager-webapp.

Some Play Framework, Service Bus, WS Notification and Web Sockets…

The Context

In the previous post I was introducing some tests I did with Play Framework and Web sockets. To summarize, it was just ‘about’ receiving messages on the Play! application and pushing them to the browser. This time, let’s go one step forward: Let’s add some infrastructure stuff to do something more real…

In the current case, I want to introduce a service bus which allows to create a real integration between service consumers and producers (I need to write another article in response to this nice videocast about integration from Zenexity guys ‘EAI & ESB n’apportent rien si les applications ne sont pas intégrables et interopérables’, but I really need more time to explain my thoughs…).
Using a service bus in the current case (and not for this case in fact…) must bring some added value. Here I choose to show that a service bus, even if it is not so lightweight at all, can provide some real cool features that you can have out of the box. Now, let’s add some event stuff to switch to an event-based world where we can have tons of event producers and let’s say thousands of event consumers. Since I can not setup such hude amount of actors, let’s say that we have one event producer and two event consumers:

  • The event producer wants to publish some stuff somewhere. To illustrate, let’s say that we have a weather sensor connected to our platform.
  • The platform provides a list of topics which producers can use to publish data. One is the weather topic which will be used by the producer above.
  • The event consumers want to be notified on new weather data i.e. as soon as the weather sensor publish new data. To keep it simple, they need to subscribe to the weather topic provided by the platform.

To recap, in the event context, the event producer only knows the service he has to push weather data to, the event consumers just have to subscribe to a topic they are interested in. All the knowledge stuff about producers, consumers, topics and all the mapping is delgated at the service bus level. Yes, true it is exactly like in some topic-based messaging stuff because at the end of the day, it is topic-based messaging stuff…

The Stuff

Now we can speak about the stuff we are going to use in the software point of view for notificaiton actors…

  • The event producer will not be a sensor but a Play! application. The application sends Web service notifications message to the service bus on a given topic.
  • The service bus is (of course) the Petals Distributed Service Bus with some Web service notification modules inside.
  • The event consumers are 1/A Play! application exposing a Web service to receive notification it subscribes to and 2/ A local java application displaying OS X notifications using Growl (let’s use JavaGrowl I published some days ago…). Note that the Play! application which have subscribed to notifications pushes them to their clients (browsers) using the funky Websocket stuff.

Let’s look at what really happens in a short video:

  • I use the Play! powered application play-soap-wsnclient to subscribe to notification on behalf of the play-soap-websocket Web application.
  • In Eclipse, I start a Web service notification powered Web service which subscribes to the same notification topic. Its listener is configured to use JavaGrowl to display incoming notifications.
  • I use play-soap-wsnclient to send notification messages to the notification service hosted on the service bus.
  • Once received, the service bus forwards the notification to all the subscribers using internal routing and WSN stuff.
  • The play-soap-websocket Web application receives a notification and push it to the client browser using Websocket.
  • At the same time, the Java application also receives a notification and display it using Growl.

One (or more) step(s) further…

And what if we have something which is not a Web service which subscribes to notifications? With the help of a service bus like Petals ESB/DSB, we just have to add a component which knows how to speak with the subscriber. For example, let’s say that SOAP is bad and that REST is the best thing ever. Can we have REST services receiving notifications? Yes, we can! Let’s just add the REST connector to the service bus. Another protocol/transport/format? Develop and add the new one. This is where the service bus can also help you. Hopefully, there are also other things which are possible with a service bus, we will see it later in other posts if I have time (as usual): Let’s think about business processes, service orchestration, transformation…
Next time we will have a look on what we can do if we use the distributed feature of the service bus, for example, receiving some notifications on one node and be able to notify subscribers which are bound to other nodes…

Source Code

Testing Play! and WebSockets

I spent one hour playing with the [Play Framework](http://playframework.org) and WebSockets in order to push some (SOAP) messages received on some Web services hosted by the Play application to the clients browser.

The result is really amazing: We can simply push these SOAP messages to clients in less than 100 lines of code. There are some problems with some messages lost due to some conception problems but which are not Play ones. In fact, the current prototype just send the messages to all the clients but what should be done is creating streams per client with some ID to identify them…

The source code is available on GitHub at [https://github.com/chamerling/play-soap-websockets](https://github.com/chamerling/play-soap-websockets)

A quick screen capture with SOAPUI sending messages to the Play! SOAP service. The Play application pushes the SOAP message to the clients (Two browsers).

Invoke business services from service bus kernel

Let’s introduce what it means…

While Petals ESB does not provide any solution to invoke services directly from the kernel/component Java code, the DSB now provides a solution for that… By using this DSB feature, we can really use the power of service oriented architecture directly in our code. No more direct calls to services, you just need to call the client and say which service you want to invoke. The service bus will resolve the endpoint and route the message to the right place by using the right way/path. By using this way, you do not have to care if you need to speak with a SOAP service, a JMS queue or a REST service: Just call the service, the DSB will do the rest.

Now some code…

So, let’s have a look on the client code and first, let’s imagine that you have a web service which is bound to the service bus. Once bound, a DSB endpoint is available inside the DSB, and all messages which are sent to this endpoint will be forwarded to the final web service. One basic approach is to create a web service hosted by the service bus which ‘consumes’ the DSB service. As a result, a SOAP message sent to the DSB web service will be forwarded to the DSB endpoint which will forward it again to the final web service. Allright, but this is just for external SOAP clients and it is impossible to invoke anything firectly from the service bus kernel.
Now, let’s call the DSB endpoint from Java code… To put ourselves in a **real** case, let say that you are DSB kernel developer and that you want to say hello to the world (oh s*** really?) by using the *HelloWorldService* which is bound to the bus. Let’s code this client:

https://gist.github.com/1186183

So, just get a *org.petalslink.dsb.service.client.Client* from the *org.petalslink.dsb.service.client.ClientFactory* factory, create your *org.petalslink.dsb.service.client.Message* and send it by using the *sendReceive* method. That’s all… Let’s try to clarify what it means with a simple schema:

Hacker inside!

It looks so simple… I just deeply hacked the petals service bus kernel to provide such a thing. Why? Because the service bus is also deeply based on the JBI specification: Creating a client means that the client need to be able to receive responses and acts itself as a service on the service bus point of view. It is also built using all the component context stuff. No component context means no way to send/receive messages (component contexts are just created when JBI components are installed).
So it is really important to release the client when you do not use it anymore: There are tons of listeners and threads in the different layers which are just used to receive messages. If you do not release the client, there will be unused resources for a long time.

Jouons avec WebServiceProvider…

Il y a des jours où exposer ses classes annotés avec @WebService n’est pas satisfaisant…
Pour moi ce jour c’est aujourd’hui: Il m’est impossible de marshaller mes beans en document DOM à cause d’un contexte JAXB tordu et d’une API qui ne prend que du DOM en entrée (OK bon ça c’est nul mais c’est pas de moi…). Qu’a cela ne tienne, il est temps d’utiliser les WebServiceProvider puisque ils vont me permettre de directement récupérer mon message SOAP sous forme de document… Un peu moins straight forward comme approche mais qui peut convenir dans certains cas. Comme les exemples ne courent pas le Web, regardons ce que l’on peut faire pour s’en sortir…

(Tout le code source de cet article est disponible sur Github : http://github.com/chamerling/chamerling.org-samples/tree/master/cxf-serviceprovider-072011)

Une première solution simple est d’implémenter javax.xml.ws.Provider, de rajouter deux/trois annotations et le tour est quasiment joué:

package org.chamerling.blog.cxf.serviceprovider;

import javax.xml.soap.SOAPMessage;
import javax.xml.ws.Provider;
import javax.xml.ws.ServiceMode;
import javax.xml.ws.WebServiceProvider;

/**
* @author chamerling
*
*/
@WebServiceProvider
@ServiceMode(value = javax.xml.ws.Service.Mode.MESSAGE)
public class SimpleServiceProvider implements Provider {

/*
* (non-Javadoc)
*
* @see javax.xml.ws.Provider#invoke(java.lang.Object)
*/
@Override
public SOAPMessage invoke(SOAPMessage in) {
return null;
}
}

Reste à l’exposer avec CXF en utilisant org.apache.cxf.jaxws.JaxWsServerFactoryBean. Cette approche simpliste a le mérite de marcher, il ne reste plus qu’a manipuler les SOAPMessage dans l’implémentation de la méthode invoke.

Besoin de l’opération qui est appelée? Ajoutons javax.xml.ws.WebServiceContext en tant que javax.annotation.Resource. Ce contexte sera automatiquement rempli pour nous par CXF et accessible dans le corps de la méthode invoke. Par exemple, on peut travailler de la sorte:

/**
 *
 */
package org.chamerling.blog.cxf.serviceprovider;

import javax.annotation.Resource;
import javax.xml.namespace.QName;
import javax.xml.soap.SOAPMessage;
import javax.xml.ws.Provider;
import javax.xml.ws.ServiceMode;
import javax.xml.ws.WebServiceContext;
import javax.xml.ws.WebServiceProvider;
import javax.xml.ws.handler.MessageContext;

/**
 * @author chamerling
 *
 */
@WebServiceProvider
@ServiceMode(value = javax.xml.ws.Service.Mode.MESSAGE)
public class SimpleServiceProvider implements Provider {

	@Resource
	WebServiceContext wsContext;

	/*
	 * (non-Javadoc)
	 *
	 * @see javax.xml.ws.Provider#invoke(java.lang.Object)
	 */
	@Override
	public SOAPMessage invoke(SOAPMessage in) {
		System.out.println("Operation : " + getOperation());
		System.out.println("Message In :");
		try {
			in.writeTo(System.out);
		} catch (Exception e) {
			// bad
		}
		System.out.println();
		return null;
	}

	private QName getOperation() {
		QName result = null;
		if (wsContext != null && wsContext.getMessageContext() != null) {
			Object o = wsContext.getMessageContext().get(
					MessageContext.WSDL_OPERATION);
			if (o != null && o instanceof QName) {
				result = (QName) o;
			}
		}
		return result;
	}
}

Et invoquer le service:

package org.chamerling.blog.cxf.serviceprovider;

import org.apache.cxf.jaxws.JaxWsProxyFactoryBean;
import org.apache.cxf.jaxws.JaxWsServerFactoryBean;

import junit.framework.TestCase;

/**
 * @author chamerling
 *
 */
public class SimpleServiceTest extends TestCase {

	public void testExpose() throws Exception {
		String url = "http://localhost:9999/foo/bar/SimpleService";
		final JaxWsServerFactoryBean ssf = new JaxWsServerFactoryBean();
		ssf.setAddress(url);
		ssf.setServiceBean(new SimpleServiceProvider());
		ssf.create();

		HelloService client = getClient(url);
		client.sayHello("Rock N Roll!");
	}

	/**
	 * @param url
	 * @return
	 */
	private HelloService getClient(String url) {
		JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
		factory.setAddress(url);
		factory.setServiceClass(HelloService.class);
		Object client = factory.create();
		return HelloService.class.cast(client);
	}
}

Dans ce cas, on a une opération affichée qui est {http://serviceprovider.cxf.blog.chamerling.org/}invoke et le message SOAP

Rock N Roll!

Pas très convaincant pour l’opération avec cette approche… Le mieux est de pousser un peu plus loin et partir du contrat de service (WSDL) de notre Provider. CXF permet de le spécifier via ses factories lors de la construction du service. Cette utilisation plus avancée est détaillée en partant de org.chamerling.blog.cxf.serviceprovider.CXFExposer. L’approche utilisée dans CXFExposer permet aussi de cacher toute la tambouille JAXWS à l’utilisateur, au final il a besoin d’implementer seulement org.chamerling.blog.cxf.serviceprovider.Service

package org.chamerling.blog.cxf.serviceprovider;

import javax.xml.namespace.QName;

import org.w3c.dom.Document;

/**
 * @author chamerling
 *
 */
public interface Service {

    /**
     * Get the WSDL description
     *
     * @return
     */
    String getWSDLURL();

    /**
     * Get the service URL ie where to publish it...
     *
     * @return
     */
    String getURL();

    QName getEndpoint();

    QName getInterface();

    QName getService();

    /**
     * Invoke the service
     *
     * @param request
     * @param action
     */
    Document invoke(Document in, QName operation) throws ServiceException;

}

Maven m’a sauvé… encore!

Encore une fois Apache Maven m’a sauvé, ou presque… Je cherchais un moyen de récupérer toutes les sources des dépendances d’un module pour un faire une archive complète. Le module en question étant une distribution contenant quelques dizaines de modules gérés par Maven, la solution est de regarder ce qu’offre l’ami Maven en question. J’utilise souvent le plugin dependency pour avoir l’arbre de dépendance via le goal tree et résoudre certains conflits de versions (oui bon m2eclipse le fait surement mais je reste fan du Terminal…), et donc il me semblait bien que le plugin dependency faisait plus que ca. C’est là qu’entre en jeu le goal unpack-dependencies. Deux/trois ajustement dans le POM du projet et il est simple de récupérer les sources:

<project>
<!--...-->
  <plugins>
  		<plugin>
		        <groupId>org.apache.maven.plugins</groupId>
		        <artifactId>maven-dependency-plugin</artifactId>
		        <version>2.2</version>
		        <executions>
		          <execution>
		            <id>src-dependencies</id>
		            <phase>package</phase>
		            <goals>
		              <!-- use copy-dependencies instead if you don't want to explode the sources -->
		              <goal>unpack-dependencies</goal>
		            </goals>
		            <configuration>
		              <classifier>sources</classifier>
		              <failOnMissingClassifierArtifact>false</failOnMissingClassifierArtifact>
		              <outputDirectory>${project.build.directory}/sources</outputDirectory>
					  <includeGroupIds>org.ow2.petals</includeGroupIds>
		            </configuration>
		          </execution>
		        </executions>
		      </plugin>
   <!--...-->
  </plugins>
</project>

Mises à jour Twitter