ICE09 . playing with java, scala, groovy and spring .

Playing with Spring

Archive for November, 2009

Spring 3 on the Google App Engine: REST in Peace

Posted by ice09 on November 19, 2009

Note: The complete source and necessary jars are available in github here. For just downloading, unzipping and importing the project into Eclipse, press on the github site.
Important: this is a collection of projects, the relevant project for this post is gae-rest-sample and spring3-resttemplate.

Finally I succeeded in getting REST (with the new REST support in Spring 3) done in the Google App Engine.
It wouldn’t be an easy task, but thanks to one brilliant blog post, a forum post and a very helpful JIRA issue, it can be resolved just by searching.

  1. The brilliant blog post about Spring REST in general by Eyal Lupu.
    This post is just a rewrite of that one, just adding the GAE-related stuff.
    All details about REST content negotiation and how the Spring MVC works with REST is explained there.
  2. The forum post about problems with GAE and XStream.
    This helps to unterstand the reason for the sun.reflect.ReflectionFactory is a restricted class exception when using XStream on GAE.
  3. The JIRA issue 566, which investigates the problem in more detail and provides a solution which is valid for at least the usage in this context (note the last entry).

I finally took the snapshot jar attached to the JIRA issue, which works well for the samples in this post. Nevertheless, there might still be difficulties with XStream on the GAE, so be aware.

The client

Eyal Lupu also provides a JSON-Client in his post. The client itself is straightforward – it uses the Spring RestTemplate. The JsonHttpMessageConverter uses Spring’s FileCopyUtils for InputStream String conversion which is quite cool.

Spring RestTemplate:

    RestTemplate restTemplate = new RestTemplate();
    restTemplate.setMessageConverters(new HttpMessageConverter[]{new JsonHttpMessageConverter()});

    JSONObject result = restTemplate.getForObject("http://localhost:8080/users/.js", JSONObject.class);

    JSONArray aresult = result.getJSONArray("payload");
    for (int x=0; x<aresult.length(); ++x) {
        JSONObject currResult = (JSONObject) aresult.get(x);
        System.out.format("%-100s | %s%n", currResult.get("name"), currResult.get("lastLoggedInAt"));
    }

JSON-Unmarshaller:

	@Override
	protected JSONObject readInternal(Class<JSONObject> clazz, HttpInputMessage inputMessage) throws IOException, HttpMessageNotReadableException {
                MediaType contentType = inputMessage.getHeaders().getContentType();
		Charset charset = contentType.getCharSet() != null ? contentType.getCharSet() : Charset.forName("utf-8");
		String stringResult =  FileCopyUtils.copyToString(new InputStreamReader(inputMessage.getBody(), charset));

		JSONObject jObject;
		try {
			jObject = new JSONObject(stringResult);
		} catch (JSONException e) {
			throw new IOException (e);
		}
		return jObject;
	}

Annoyances

Here are some things I don’t understand and I had to work around.
Please comment if you know more about the problem.

  1. In the web.xml, the url-pattern has to be / instead of /* (or /rest in the original sample) to get it working in the GAE
  2. Usually, I would prefer having the jsp-views in /WEB-INF/*. However, this does not work in GAE (doesn’t find WEB-INF or anything below WEB-INF)
Advertisements

Posted in Google App Engine, REST | 4 Comments »

Scala Mashup Series: The Template Method in Java, Spring and Scala

Posted by ice09 on November 9, 2009

Continuing the Scala Mashup Series, this post is about a quite important, often used, but also controversial pattern: The Template Method.

In general, there are great explanations of the pattern in Wikipedia and of course in Head First Design Patterns.

Criticism

A very good summary of the Template Method pattern especially in comparison to the Strategy pattern is on stackoverflow.com:

The Template pattern does compile-time algorithm selection by subclassing.
The Strategy pattern does run-time algorithm selection by containment.

Example

Imagine you want to arrange a dinner. The entire algorithm for arranging a dinner is predefined:

  1. Invite your guests
  2. Go shopping
  3. Clean your fridge to provide enough space
  4. Prepare the meal
  5. Server the meal
  6. Wash the dishes

Now, some parts of this algorithm (which in its entirety is called arrange) are fixed (invariant parts). Other parts are highly dependant on the meal you want to prepare (variant parts).
In Template Method, the algorithm itself is described in a public accessible method, eg. arrange(), which combines and calls the variant and invariant methods. The invariant methods are implemented in the abstract super class. The variant parts are implemented in the concrete classes, eg. BratwurstDinner, which implements the concrete way to do shopping for the BratwurstDinner, as well as the preparation of the Bratwurst itself.
The important part is the compile-time binding – the implementing, concrete classes cannot change their implementation during runtime.
This makes the Template Method almost ideal for framework creation, when you want to provide abstract algorithms with certain concrete implementations (think of abstract database layers, with an algorithm which cares for opening & establishing connections, handles exceptions and cleans up nicely after doing … the variant part, which is implemented in the concrete classes).

Here is the dinner example using the Template Method pattern

Here is the dinner example using the Strategy pattern

There are two clever blog posts, one about disadvantages of the Template Method pattern, and one mentioning a possible alternative in languages with first class functions like Scala.

Language comparison: Java, Scala, Spring

We will happily leave out the Java example, since it is just too obvious. In the Wikipedia article a sample is given, together with the class diagram it should be pretty obvious how Template Method is implemented in Java. The only thing to keep in mind is to set the super class abstract, as well as the variant methods – which are then implemented (using @Override) in the concrete classes.

Scala

In Scala there are is the intuitive way of implemeting the Template Method, analogous to Java:

abstract class DinnerEvent {

  def arrange(): Unit = {
    makeGuestList
    doShopping
    cleanFridge
    prepareMeal
    serveMeal
    washDishes
  }

  def prepareMeal(): Unit
  def doShopping(): Unit

  def makeGuestList(): Unit = {
    println("making guest list...")
  }

  def cleanFridge(): Unit = {
    println("cleaning fridge...")
  }

  def serveMeal(): Unit = {
    println("serving meal...")
  }

  def washDishes(): Unit = {
    println("washing dishes...")
  }

}

trait TurkeyDinner {

  def doShopping(): Unit =  {
    println("...buying turkey")
  }

  def prepareMeal(): Unit =  {
    println("...preparing turkey")
  }

}

object TemplateMethod {

  def main(args:Array[String]) = {
    val dinner = new DinnerEvent with TurkeyDinner
    dinner.arrange()
  }

}

However, in languages with first class functions, it is too tempting to use the Strategy pattern:

class DinnerEventWithStrategy(shoppingStrategy: () => Unit, preparingStrategy: () => Unit) {

  def arrange(): Unit = {
    makeGuestList()
    shoppingStrategy()
    cleanFridge()
    preparingStrategy()
    serveMeal()
    washDishes()
  }

  def makeGuestList(): Unit = {
    println("making guest list...")
  }

  def cleanFridge(): Unit = {
    println("cleaning fridge...")
  }

  def serveMeal(): Unit = {
    println("serving meal...")
  }

  def washDishes(): Unit = {
    println("washing dishes...")
  }

}

object Closure {

  def turkeyShoppingStrategy(): Unit = {
    println ("...shopping turkey strategy")
  }

  def turkeyPreparationStrategy(): Unit = {
    println ("...preparing turkey strategy")
  }

  def main(args:Array[String]) = {
    val dinner = new DinnerEventWithStrategy(turkeyShoppingStrategy, turkeyPreparationStrategy)
    dinner.arrange()
  }

}

So, you should choose wisely, both patterns definitely have their scope of application – just keep in mind that Strategy also uses composition over inheritance, which is always a good advise.

…and what about Spring?

This is interesting, Spring uses the Template Method a lot – and many think it is one of their greatest strengths.
But wait … the Template Method pattern? Not really. The Spring Framework templates in fact are not implementing the Template Method pattern, even if it feels this way.
Let’s examine a small usage example of Spring’s JdbcTemplate:

package com.commons.sample.dao;

import java.util.List;

import org.springframework.jdbc.core.simple.ParameterizedBeanPropertyRowMapper;
import org.springframework.jdbc.core.simple.SimpleJdbcDaoSupport;

public class JdbcSimpleDao extends SimpleJdbcDaoSupport implements SimpleDao {

	final static String single	= "select id, name from test where id = ?"; 
	final static String list 	= "select id, name from test"; 

	public Simple getSimpleById(int id) {
		return getSimpleJdbcTemplate().queryForObject(single, 
				ParameterizedBeanPropertyRowMapper.newInstance(Simple.class), id);
	}

	public List<Simple> getSimples() {
		return getSimpleJdbcTemplate().query(list, 
				ParameterizedBeanPropertyRowMapper.newInstance(Simple.class));
	}

}

What is happening here? Even though the class extends SimpleJdbcDaoSupport, it would not have to.
The Method query(…) takes the SQL and a ParameterizedBeanPropertyRowMapper.newInstance(Simple.class), which defines a row mapper that creates an object of class Simple – this certainly describes a strategy. So, even the so-called Spring Templates actually uses the Strategy pattern.

Summary

The Template Method is a powerful pattern. Use it if you create a framework, since you can inject certain functionality in the implementing classes at compile time, which let the user of your framework extend it easily, but in a controlled and “traceable” way. For Java, it is a natural way to enable users of your classes to extends the functionality at certain points.
In languages with first-order functions (or closures), like Scala, it might feel more natural to use the Strategy pattern instead. However, keep in mind that the patterns are not completely identical.

Posted in Patterns, Scala | 2 Comments »

Simple Performancetesting with Grinder

Posted by ice09 on November 4, 2009

In the last post, I created a prototype with some technologies which were new to me, especially Gaelyk.

As it turned out, the prototype behaved quite differently in my local system and when deployed in the Google App Engine. What seems really strange to me is that even after all five number series had been cached, every 3-4th request took quite long, at least locally. In the deployed version, this seemed to be different and not too slow (even though I suppose a pure Java version would be significantly faster).

Now, how can I test without too much trouble if the deployed site is “just fast enough”?
Having looked around quite some time, Grinder looked most promising to me, with respect to easy of usage and features.

Preparation

Download Grinder and Jython 2.2.1 – it must be this version, not the current 2.5.x – this just does not work.

Install Jython.

Create grinder.properties and grinder.py – they can go in any directory, the most simple usage will be in the grinder/lib directory:

grinder.properties (change directory accordingly):

grinder.runs=0
grinder.threads=10
grinder.processes=1
grinder.jvm.arguments=-Dpython.home=/dev/jython2.2.1

grinder.py

# A simple example using the HTTP plugin that shows the retrieval of a
# single page via HTTP. The resulting page is written to a file.
#
# More complex HTTP scripts are best created with the TCPProxy.
import string
from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest

test1 = Test(1, "Request resource")
request1 = test1.wrap(HTTPRequest())
log = grinder.logger.output

class TestRunner:
    def __call__(self):
	grinder.statistics.setDelayReports(1)
        result = request1.GET("http://localhost:8080/data.groovy")
	log(result.getText())
	if string.find(result.getText(), "counter") < 1:
	    grinder.statistics.forLastTest.setSuccess(0)
        # result is a HTTPClient.HTTPResult. We get the message body
        # using the getText() method.

Usage

Now you can start the Grinder console with:
java -cp grinder.jar net.grinder.Console

Afterwards, start the Grinder agent with:
java -cp grinder.jar net.grinder.Console

Now, you must sync the data first, as shown in the screenshot. Afterwards, just start the tests.

grinder

According to the config, the agent runs indefinitely in 10 Threads. The site in the Jython script is called and in the resulting page, the expression counter is searched. If it is not present, the test fails. You can verify this by stopping the server, it will start increasing the failure counter immediately.

All in all, we have a very easy load test with very few lines of code.

As a result, the remote Google App Engine runs much more stable, so I just presume that the memcache implementation of the local Google App Engine server is not as good/stable as the remote one. But this is just a guess.

However, be careful with Gaelyk besides from using it for prototypes, because if the site is not called continuously, it kind of fells asleep really fast (some minutes) and needs its time for wakeup, which is quite annoying. I just don’t know what causes this, Gaelyk or the GAE.
You can verify this by just loading the Gaelyk home page. The first call often really feels kind of slow.

Posted in Grinder, Performance testing | 1 Comment »

Google Chart API with Gaelyk on the Google App Engine

Posted by ice09 on November 4, 2009

Note: Github available here. It works best with the current SpringSource Tool Suite.

Having had evening to spare, together with a colleague I came up with the following nano-project:

Write a prototype for creating a chart from some series of numbers, which is hosted in a cheap cloud.

So this brings me to: “Write a prototype (Groovy) for creating a chart (Google Chart API) from some series of numbers (taken from spreadsheets of Google Docs), which is hosted in a cheap cloud (Google App Engine).”
Some day, I will have to fix this Google addiction – however, Groovy is not affiliated to Google at all.

Evaluation of Frameworks

Wanting to combine Groovy and the Google App Engine, I immediately came across two different frameworks, the almighty and ubiquitous Grails and the much more pragmatic Gaelyk.
Since for this purpose Gaelyk is more than enough and because I wanted to try it, I used this for prototyping. By downloading the sources from Github, the typical Gaelyk project setup can be concluded if analyzed together with the information given in the Gaelyk tutorial – it’s quite easy.

The data (about 100-1000 random numbers between 5000 to 6000) should come from a web based, private (authorization-based) available data source – even though many possibilities exist, I chose Google Docs for two reasons:

  1. Easy creation of “some” random numbers
  2. Authorization based on Google Authorization
  3. Access class already exists (cmp. this post)

The last important feature should be the creation of a chart. Here, really a lot compelling solutions exist. Mainly, there are three different ways to realize Chart creation:

  1. Java based, eg. JFreeChart
  2. Javascript based, eg. JQuery plugin flot
  3. Pure Web based, eg. Google Chart API

Since I want to have it simple & easy, the third option is really great for this use case. A chart can very easily be created, compare this, which is rendered dynamically:

which is:

http://chart.apis.google.com/chart?chxl=0:||1:|&cht=lc&chxt=x,y&chs=500x150&chco=0077CC&chm=B,E6F2FA,0,0,0&chd=t:5563.0,5807.0,5096.0,5898.0,5306.0,5944.0,5932.0,5510.0,5088.0,5791.0,5039.0,5145.0,5459.0,5395.0,5997.0,5075.0,5587.0,5760.0,5561.0,5719.0,5108.0,5339.0,5321.0,5686.0,5213.0,5987.0,5228.0,5670.0,5594.0,5292.0,5909.0,5616.0,5978.0,5375.0,5504.0,5558.0,5840.0,5233.0,5947.0,5028.0,5592.0,5992.0,5020.0,5991.0,5953.0,5471.0,5243.0,5784.0,5472.0,5801.0,5652.0,5700.0,5183.0,5949.0,5809.0,5426.0,5130.0,5394.0,5919.0,5741.0,5134.0,5402.0,5801.0,5567.0,5478.0,5740.0,5563.0,5677.0,5346.0,5020.0,5243.0,5137.0,5787.0,5241.0,5351.0,5834.0,5630.0,5542.0,5238.0,5242.0,5312.0,5642.0,5424.0,5988.0,5796.0,5912.0,5599.0,5567.0,5832.0,5559.0,5760.0,5061.0,5629.0,5951.0,5095.0,5585.0,5602.0,5686.0,5562.0,5700.0,5267.0,5656.0,5757.0,5023.0,5498.0,5935.0,5650.0,5807.0,5345.0,5080.0,5919.0,5308.0,5316.0,5076.0,5154.0,5399.0,5537.0,5923.0,5968.0,5973.0,5528.0,5640.0,5097.0,5369.0,5374.0,5248.0,5955.0,5243.0,5949.0,5126.0,5659.0,5010.0,5473.0,5012.0,5536.0,5274.0,5115.0,5809.0,5212.0,5400.0,5550.0,5274.0,5931.0,5762.0,5343.0,5655.0,5434.0,5981.0,5844.0,5277.0,5823.0,5905.0,5741.0,5168.0,5057.0,5956.0,5841.0,5804.0,5421.0,5933.0,5667.0,5171.0,5083.0,5322.0,5410.0,5459.0,5092.0,5678.0,5401.0,5817.0,5526.0,5663.0,5753.0,5802.0,5289.0,5917.0,5020.0,5372.0,5993.0,5023.0,5773.0,5469.0,5681.0,5442.0,5736.0,5552.0,5678.0,5493.0,5758.0,5726.0,5713.0,5103.0,5201.0,5435.0,5583.0,5318.0,5501.0,5076.0,5004.0&chds=4000,7000

Now, was this easy?

Preparation/Implementation

First, the Groovy file which dispatches the request is as follows (must go to the groovy folder):

import java.net.URLConnection;

def labels = ["A", "B", "C", "D", "E"]
int ivalue = new Random().nextInt(5);

String link;
if (memcacheService.get(labels.get(ivalue)) == null) {
	def ts = new com.commons.gse.TestSheets();
	link = ts.getData(labels.get(ivalue)); 
	memcacheService.put(labels.get(ivalue), link)
} else {
	link = memcacheService.get(labels.get(ivalue))
}
request.setAttribute ('counter', com.commons.gse.TestSheets.counter++)
request.setAttribute ('appendix', link)
forward '/data.gtpl'

The interesting part is the usage of the predefined object memcacheService which makes it possible to just use Google App Engine’s memcache without having to deal with creation, exception handling, etc.
One of five (“A-E”) cached number series is used. If no cache for the character is found, the number is extracted from a speadsheet called data in Google Docs (see screenshots below).
After having calculated or retrieved the numerb series from the cache, the values (which is the complete link with the numbers included) are put into the ServletRequest (in this case the predefined object request). Finally, the request is dispatched to the view, a Groovy template (data.tgpl).

<html>
<head/>
<body>
  <span>counter:<%= request.getAttribute('counter') %></span>
  <form>
    <input type="button" value="Reload" onClick="window.location='/data.groovy'">
  </form>
  <img src='<%= request.getAttribute('appendix') %>'/>
</body>
</html>

A final note: if you want to use this from Github, you will have to change the Google authorization data in TestSheets.java (which is completely left out here, but is described here).

Here is the setup of the Google Docs spreadsheets:

namespreadsheet
randomnumbers

Running

If you have installed the SpringSource Tool Suite, you can just import the Eclipse project and choose “Run as…/Web Application”. The local server should be startet. Afterwards, you can call http://localhost:8080/data.groovy.

Note: Be sure to change the username/password combination in TestSheets.java

Posted in Gaelyk, Google App Engine, Google Spreadsheet API | 2 Comments »