Analysing Memory Leaks with BTrace

After having experimented with Eclipse Galileo and the now included Memory Analyzer in the last post, I came across BTrace, which very nicely fits to the tool chain for analyzing Memory Leaks and/or performance problems in applications.

Continuing the problem analysis from the last post, after having found out the resulting object for the memory leak (leakingSet), it would be interesting to find out where the objects are inserted.

Preparation

1. Download visualvm
2. Install the BTrace plugin as described here
3. Extend the object insertion with a Thead.sleep (I used 100ms)
4. Start the sample application

Usage

1. Start visualvm and search for the started application PID
2. Start tracing…
3. Install the following script

import com.sun.btrace.annotations.*;
import com.sun.btrace.AnyType;
import static com.sun.btrace.BTraceUtils.*;

@BTrace public class TraceLeakingSet {
    @OnMethod(
        clazz="+java.util.Set",
        method="add"
    )
    public static void method(AnyType[] arg) {
        print(strcat("entered ", name(probeClass())));
        print(strcat(".", probeMethod()));
        println(strcat(" with argument ", str(arg[1])));
        jstack();
        println();
    }
} 

The script will log every call to the method “add” on all subclasses of “java.util.Set” (that’s the “+”). The submitted argument is included in the first index of the AnyType array.
probeClass and probeMethod display the called class and method, jstack() displays the real stack trace. This is most useful, together with the display of class, method and argument.

Sample output is as follows:

BTrace code successfuly deployed
===========================================================
entered java.util.HashSet.add with argument scopedTarget.scopetest
java.util.Collections$SynchronizedCollection.add(Collections.java:1577)
org.springframework.beans.factory.support.AbstractBeanFactory.markBeanAsCreated(AbstractBeanFactory.java:1257)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:256)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:175)
org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:33)
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.getTarget(Cglib2AopProxy.java:657)
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:608)
com.commons.Scopetest$$EnhancerByCGLIB$$3824588a.uniqueCode()
com.commons.Tester.start(Tester.java:25)
com.commons.Tester.main(Tester.java:14)

entered java.util.HashSet.add with argument -841086979
com.commons.Tester.start(Tester.java:25)
com.commons.Tester.main(Tester.java:14)


So there are two alternating locations where add is called on Set and it’s subclasses, one location is in Spring itself (which is interesting but of no concern for the problem) and – of course – the line where the object is added in the start method. This is the location we are looking for – the question would be why so many objects are added with different arguments (instead of always the same argument with singleton instead of prototype usage).

So, BTrace is very easy to use and very useful for tracking problems in running systems with very low overhead and low intrusion (there is actually no change at the deployed code necessary).

Eclipse Galileo: Memory Analyzer (MAT) …and a little Spring…

The source can be downloaded here. The Spring libraries have to be added manually to the local maven repo (compare this.) The pom is inspired by this post.

I wanted to play with Eclipse Galileo and the Memory Analyzer (MAT), which I used successfully some time ago to identify possible memory leaks.

Using an old example, I construed a case where instead of using a singleton, accidentially a prototype is used (once again a nice sample for Spring’s capabilities – changing one line changes the behaviour completely).

<?xml version="1.0" encoding="UTF-8"?>
<beans [...]>

<!-- using is the "correct" singleton pattern
     <bean id="scopetest" class="com.commons.Scopetest">
     </bean>
-->

<!-- using is the "leaking" prototype pattern
     <bean id="scopetest" class="com.commons.Scopetest" scope="prototype">
         <aop:scoped-proxy/>
     </bean>
-->

     <bean id="testclass" class="com.commons.Testclass">
     	<property name="scopetest" ref="scopetest"/>
     </bean>

</beans>

What’s the difference? Let’s have a look at the main class:

Testclass testclass = (Testclass) app.getBean("testclass");
for (; i < 1000000; i++) {
     Scopetest proxy = testclass.getScopetest();
     leakingSet.add(proxy.uniqueCode());
}
System.out.println("size: " + leakingSet.size());

Using a singleton, the uniqueCode is always the same (I did not use hashCode since it collides with the Proxy hashCode), which leads to a Set with just one object (the uniqueCode of the singleton – which is always the same).
Using the prototype, the class is proxied by cglib and each call to uniqueCode() creates a new Scopetest object, each has a timestamp-based uniqueCode – therefore each object (the uniqueCode-Integer) is added to the Set.

As being said, this is highly construed, however, if you forget to change the object type from prototype to singleton somewhere in your application, you might experience a non-expected behaviour (eg. 1000000 objects instead of one).

How can this be found with Eclipse MAT?

Preparation

First, Eclipse MAT has to be installed (anybody knows why?):

Afterwards, the sample application is started and a heap dump is taken. There are several options for doing this, I used jvisualvm.

Usage

Back in Galileo, the heap dump can now be opened with Eclipse MAT (after opening the MAT Perspective):

Using the generated report, the source of the memory leak is quite easy to find – it is immediately identified as a “suspect” automatically by the “Leak Suspect Report”. Even the correct name of the private member (the Set with the 1000000 objects) is shown (leakingSet), which makes pinpointing the causing object an ease.

Google App Engine, GWT, Spring 3, JPA and Google Spreadsheet

The previous post describes the main steps, this post just extends two classes and adds a “sync function”, which allows for syncronizing with data im a Google Spreadsheet.

The github project is here.

Caveats

• There is a known problem with cookie handling, therefore the static block is used.
• The spreadsheet document must be named tabledata.

• The sample is quite stupid, to test the sync function, the usename sync must be used.

GreetingsServiceImpl.java

package com.commons.ssheet.server;

import java.io.IOException;
import java.net.URL;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;

import javax.persistence.EntityManager;
import javax.persistence.EntityTransaction;
import javax.persistence.PersistenceException;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.orm.jpa.JpaCallback;
import org.springframework.orm.jpa.JpaTemplate;
import org.springframework.stereotype.Component;

import com.commons.ssheet.client.GreetingService;
import com.google.gdata.client.http.GoogleGDataRequest;
import com.google.gdata.client.spreadsheet.SpreadsheetService;
import com.google.gdata.data.spreadsheet.CellEntry;
import com.google.gdata.data.spreadsheet.CellFeed;
import com.google.gdata.data.spreadsheet.SpreadsheetEntry;
import com.google.gdata.data.spreadsheet.SpreadsheetFeed;
import com.google.gdata.data.spreadsheet.WorksheetEntry;
import com.google.gdata.util.ServiceException;
import com.google.gwt.user.server.rpc.RemoteServiceServlet;

@SuppressWarnings("serial")
@Component
public class GreetingServiceImpl extends RemoteServiceServlet implements GreetingService {

 	static {
		System.setProperty(GoogleGDataRequest.DISABLE_COOKIE_HANDLER_PROPERTY, "true");
 	}

	private boolean first;

	@Autowired
	private EntityManager entityManager;
	@Autowired
	private SpreadsheetService spreadsheetService;
	@Autowired
	private URL metafeedUrl;

	private String username = "YOURUSERNAME";
	private String password = "YOURPASSWORD";

	public JpaTemplate getTemplate() {
		return new JpaTemplate(entityManager);
	}

	public String greetServer(String input) {
		try {
			return getData(input);
		} catch (IOException e) {
			e.printStackTrace();
		} catch (ServiceException e) {
			e.printStackTrace();
		}
		return null;
	}

	private String getData(String input) throws IOException, ServiceException {
		String all = "";
		if (input.equals("sync")) {
			first = false;
		}
		for (Customer customer : getCustomers()) {
			if (!first) {
				removeCustomer(customer);
			}
		}
		first = true;
		persistCustomer(createCustomer(input));
		if (input.equals("sync")) {
			for (Customer customer : syncWithSpredsheet()) {
				persistCustomer(customer);
			}
		}
		for (Customer customer : getCustomers()) {
			all += "id: " + customer.getId() + " - firstname: " + customer.getFirstName() + " - name:" + customer.getLastName() + "<br>";
		}
		return all;
	}

	private void removeCustomer(Customer customer) {
		EntityTransaction trx = entityManager.getTransaction();
		trx.begin();
		entityManager.remove(customer);
		trx.commit();
	}

	private Customer createCustomer(String input) {
		Customer newCustomer = new Customer();
		newCustomer.setFirstName(input + System.currentTimeMillis());
		newCustomer.setLastName(input + System.currentTimeMillis());
		return newCustomer;
	}

	private void persistCustomer(Customer customer) {
		EntityTransaction trx = entityManager.getTransaction();
		trx.begin();
		entityManager.persist(customer);
		trx.commit();
	}

	private Collection<Customer> getCustomers() {
		return getTemplate().execute(new JpaCallback<Collection<Customer>>() {
			@Override
			public Collection<Customer> doInJpa(EntityManager arg0) throws PersistenceException {
				return (Collection<Customer>) entityManager.createQuery("SELECT cust FROM com.commons.ssheet.server.Customer cust").getResultList();
			}
		});
	}

	private List<Customer> syncWithSpredsheet() throws IOException, ServiceException {
		spreadsheetService.setUserCredentials(username, password);
		List<Customer> result = new ArrayList<Customer>();
		SpreadsheetFeed feed = (SpreadsheetFeed) spreadsheetService.getFeed(metafeedUrl, SpreadsheetFeed.class);
		List<SpreadsheetEntry> spreadsheets = feed.getEntries();
		for (int i = 0; i < spreadsheets.size(); i++) {
			SpreadsheetEntry entry = spreadsheets.get(i);
			if (entry.getTitle().getPlainText().trim().equals("tabledata")) {
				List<WorksheetEntry> worksheets = entry.getWorksheets();
				for (int j = 0; j < worksheets.size(); j++) {
					WorksheetEntry worksheet = worksheets.get(j);
					URL cellFeedUrl = worksheet.getCellFeedUrl();
					CellFeed cfeed = spreadsheetService.getFeed(cellFeedUrl, CellFeed.class);
					Customer customer = new Customer();
					boolean wasAdded = true;
					for (CellEntry cell : cfeed.getEntries()) {
						String value = cell.getCell().getInputValue();
						if (cell.getTitle().getPlainText().startsWith("A")) {
							if (!wasAdded) {
								result.add(customer);
								customer = new Customer();
							}
							customer.setFirstName(value);
						} else if (cell.getTitle().getPlainText().startsWith("B")) {
							customer.setLastName(value);
							wasAdded = false;
						}
					}
					result.add(customer);
				}
			}
		}
		return result;
	}

}

dispatcher-servlet.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
	xmlns:context="http://www.springframework.org/schema/context"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

	<context:component-scan base-package="com.commons" />

	<bean id="spreadsheetService" class="com.google.gdata.client.spreadsheet.SpreadsheetService">
		<constructor-arg><value>appname</value></constructor-arg>
	</bean>

	<bean id="metafeedUrl" class="java.net.URL">
		<constructor-arg><value>http://spreadsheets.google.com/feeds/spreadsheets/private/full</value></constructor-arg>
	</bean>

	<bean id="entityManager" factory-bean="EMF" factory-method="entityManager" />

	<bean class="org.springframework.web.servlet.handler.SimpleUrlHandlerMapping">
		<property name="mappings">
			<value>
				/googlespreadsheet/greet=gwtController
			</value>
		</property>
	</bean>

  	<bean name="gwtController" class="com.commons.ssheet.server.GWTController">
    	<property name="remoteService" ref="greetingServiceImpl"/>
  	</bean>	

</beans>

Google App Engine, GWT, Spring 3 and JPA

UPDATE: The github repository for this post is here. An prepared Eclipse project can be downloaded and imported into Eclipse with the installed Eclipse Google plugin (instructions below).

Being really enthusiastic about Java on the Google App Engine, I tried out some simple examples, which I will document here.
Most interesting for me was the combination of the Google App Engine, GWT and – of course – the Spring Framework.
Since the current version of Spring is version 3.0, I tried this version, together with the new Google Eclipse plugin, which currently brings the App Engine SDK 1.2.1 and GWT 1.6.4.
The Eclipse version is Ganymede 3.4.2.

References

Most of this stuff is already described, I will list all useful sites here with a short summary – this post is just a mash-up of the mini-tutorials on these sites.

The first entry point for developing for the Google App Engine is obviously the documentation of Google itself. It is well written, easy comprehensible and there is a lot to find for this early stage.

• Always a useful helper is http://appengine-cookbook.appspot.com/, e.g. for this tipp, which was quite useful for me.
• The best introduction for the Google App Engine with Spring 3.0 topic is here.
• The best post about Spring with Google GWT is this (the gwtController below is taken from this post).

There are different other solutions described here, here and here. However, these solutions did not fit the demands of low impact, current version compatibility or just simple usage.

Preparation

• First, a “Google” Web Application is created by File > New > Web Application Project
  

Project setup

First, the added libraries.

• Libraries added to WEB-INF/lib and the to the project’s classpath



• Servlet jar is included in lib directory but not added to classpath

Besides the jar libraries, several configuration files and Java classes had to be included

• Java classes to be included or changed (detailed description below)



• Configuration files to be included or changed (detailed description below)

Function

The most innovative part is taken from here and described in detail.

The following parts have been added:

• The EMF class was made a spring component, it is created by Spring’s factory-method.
• The context:component-scan is used for autowiring.
• The JpaTemplate is used for simplified resource management.

Below are the missing parts.

EMF.java

package com.commons.server;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;

import org.springframework.stereotype.Component;

@Component
public class EMF {

	private static final EntityManagerFactory emfInstance =
        Persistence.createEntityManagerFactory("transactions-optional");

    public EMF() {}

    public EntityManager entityManager() {
        return emfInstance.createEntityManager();
    }
}

persistence.xml

<?xml version="1.0" encoding="UTF-8" ?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
        http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="1.0">
<persistence-unit name="transactions-optional">
<provider>org.datanucleus.store.appengine.jpa.DatastorePersistenceProvider</provider>
        <class>com.commons.server.Customer</class>
<properties>
<property name="datanucleus.NontransactionalRead" value="true"/>
<property name="datanucleus.NontransactionalWrite" value="true"/>
<property name="datanucleus.ConnectionURL" value="appengine"/>
        </properties>
    </persistence-unit>

</persistence>

Customer.java

package com.commons.server;

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;

@Entity
public class Customer {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String firstName;

    private String lastName;

    public Long getId() {
        return id;
    }
    public void setId(Long id) {
        this.id = id;
    }

    public String getFirstName() {
        return firstName;
    }
    public void setFirstName(String firstName) {
        this.firstName = firstName;
    } 

    public String getLastName() {
        return lastName;
    }
    public void setLastName(String lastName) {
        this.lastName = lastName;
    } 

}

GreetingsServiceImpl.java

package com.commons.sigae.server;

import java.util.Collection;

import javax.persistence.EntityManager;
import javax.persistence.EntityTransaction;
import javax.persistence.PersistenceException;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.orm.jpa.JpaCallback;
import org.springframework.orm.jpa.JpaTemplate;
import org.springframework.stereotype.Component;

import com.commons.sigae.client.GreetingService;
import com.google.gwt.user.server.rpc.RemoteServiceServlet;

@SuppressWarnings("serial")
@Component
public class GreetingServiceImpl extends RemoteServiceServlet implements GreetingService {

 	private boolean first;
	protected EntityManager entityManager;

	@Autowired
	public void setEntityManager(EntityManager entityManager) {
		this.entityManager = entityManager;
	}

	public JpaTemplate getTemplate() {
		return new JpaTemplate(entityManager);
	}

	public String greetServer(String input) {
		return getData(input);
	}

	private String getData(String input) {
		String all = "";
		for (Customer customer : getCustomers()) {
			if (!first) {
				removeCustomer(customer);
			}
		}
		first = true;
		for (Customer customer : getCustomers()) {
			all += "id: " + customer.getId() + " - firstname: " + customer.getFirstName() + " - name:" + customer.getLastName();
		}
		createCustomer(input);
		return all;
	}

	private void removeCustomer(Customer customer) {
		EntityTransaction trx = entityManager.getTransaction();
		trx.begin();
		entityManager.remove(customer);
		trx.commit();
	}

	private void createCustomer(String input) {
		EntityTransaction trx = entityManager.getTransaction();
		Customer newCustomer = new Customer();
		newCustomer.setFirstName(input + System.currentTimeMillis());
		newCustomer.setLastName(input + System.currentTimeMillis());
		trx.begin();
		entityManager.persist(newCustomer);
		trx.commit();
	}

	private Collection<Customer> getCustomers() {
		return getTemplate().execute(new JpaCallback<Collection<Customer>>() {
			@Override
			public Collection<Customer> doInJpa(EntityManager arg0) throws PersistenceException {
				return (Collection<Customer>) entityManager.createQuery("SELECT cust FROM com.commons.sigae.server.Customer cust").getResultList();
			}
		});
	}

}

dispatcher-servlet.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
	xmlns:context="http://www.springframework.org/schema/context"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

	<context:component-scan base-package="com.commons" />

	<bean id="entityManager" factory-bean="EMF" factory-method="entityManager" />

	<bean class="org.springframework.web.servlet.handler.SimpleUrlHandlerMapping">
		<property name="mappings">
			<value>
				/springingae/greet=gwtController
			</value>
		</property>
	</bean>

  	<bean name="gwtController" class="com.commons.sigae.server.GWTController">
    	        <property name="remoteService" ref="greetingServiceImpl"/>
  	</bean>	

</beans>

web.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE web-app
    PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
    "http://java.sun.com/dtd/web-app_2_3.dtd">

<web-app>

	<!-- Default page to serve -->
	<welcome-file-list>
		<welcome-file>SpringInGAE.html</welcome-file>
	</welcome-file-list>

	<servlet>
		<servlet-name>dispatcher</servlet-name>
		<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
		<load-on-startup>1</load-on-startup>
	</servlet>

	<servlet-mapping>
		<servlet-name>dispatcher</servlet-name>
		<url-pattern>/springingae/greet</url-pattern>
	</servlet-mapping>

</web-app>

Tipps & Tricks:

• About JPA 2.
• About Using JPA

The mini project series: Web Testing with WebDriver and Spring

All files and the zipped Eclipse project can be downloaded at the end of this post.

It’s more fun to learn Spring by example, that’s why I start a mini project series, where I use Spring as glue code for completely different purposes.
First, I introduce a Web Testing “Framework” which builds upon WebDriver and obviously uses Spring for registering and managing the several entities.

Idea

The idea is to enable the user to write simple test modules which can easily be combined to use cases which themsevles can be combined to scenarios.
In terms of banking, a module could be “Login”, which performs a login on the banking site. The use case would be “Buy Fund”, which consists of several steps (modules), eg. “Login”, “Choose Fund”, etc. A complete scenario consists of several use cases, eg. “Buy Fund”, “Check Fund in Orderbook”, etc.

The structure is shown in the classdiagram.

The modules will be able to read their own test data, which itself consists of input data and test patterns, stored in either CSV or Excel files.

The testdata will be stored in directory structure.

Spring specials

At some point, we want to create Spring beans from modules and register them in some kind of map. However, we can not know if a bean is registered and we do not want to implement some “registering” logic inside the single steps.
Therefore, we create out own BeanPostProcessor and check if the bean which is currently created implements the IModule interface. If it does, we will register it.
This is a very nice way of using Springs interceptor mechanisms, which allow you to intercept each step in Springs initialization process.

The Bean Post Processor

package de.deka.webtest.springext;

import org.apache.log4j.Logger;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.beans.BeansException;

import de.webtest.entities.ModuleRegistry;
import de.webtest.entities.modules.IModule;

public class ModuleRegistryProcessor implements BeanPostProcessor, InitializingBean {

	private static Logger log = Logger.getLogger(ModuleRegistryProcessor.class);

	private ModuleRegistry registry;

	public void setRegistry(ModuleRegistry registry) {
		this.registry = registry;
	}

    public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        return bean;
    }

    public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
    	if (bean instanceof IModule) {
            log.info("Registering module '" + beanName + "'");
    		registry.addModule(beanName, (IModule)bean);
    	}
        return bean;
    }

	public void afterPropertiesSet() throws Exception {
		if (registry == null)
			throw new IllegalStateException("registry not set.");
	}

}

This bean just has to be registered in an application-context and if a bean is registered and of type IModule, it is added to the Module registry.

Instructions

• Download Mavenified webDriver. Unzip to the Maven repository.
• zipped Eclipse project
• webDriver Firefox
• webDriver Common

Installation

IMPORTANT: For the example Scenario to work you will have to switch GMail to HTML layout.

After importing the Eclipse project you will have to change the data in the provided Excel sheets (you can use CSV files as well). In the sample, Firefox will open and the workflow will try to log in to GMail with the provided account (in the Excelsheet in the “Login” subdirectory of the “Scenario…” directory). Afterwards, a mail will the composed and send and the sent mail folder will be examined for the sent mail (the subject). If it’s available, the first mail will be checked and deleted.
Finally, a logout is performed.

Besides having to change the data provided in the Excel sheet data files, the sample is far from finished. In the downloaded form, the sample will only work for the German version of Gmail (also in the current version). In the Login module, the text “Neue Nachricht” (new mail) is searched to validate a successful login. Obviously, this does not work in other languages. Furthermore, it would make much sense to refactor the sample and externalize the text search to the Excel sheet (as “test:text” column). But this is just an example and there is much to do on it.

Installation of Firefox plugin

Update 18.09.2008

After the comment by Simon I reassured that you just have to do nothing to get the Firefox extension installed – it will work by default. However, having played around with earlier versions of WebDriver which had to be installed manually, I did not understand this and was quite disturbed by not being able to find any installation instruction. So, ok, that’s why – there is no need for it.
It is important to note that you have to remove the existing “WebDriver” profile first.
If no profile exists, the driver will be installed automatically (or automagically) on the first startup of your Java application (that’s quite a lot magic for me – however, it’s really convenient).
I recommend reading his post regarding installation of firebug plugin which is really useful.

For what it’s worth, I keep the manual installation instructions in this post, for those who whyever can not delete the existing WebDriver profile.

Manual installation of WebDriver Firefox plugin

Compare the screenshots, first you have to create a Profile in Firefox named “WebDriver” (start Firefox with -p to start profile manager).
Afterwards, the zip inside the downloaded webdriver-zips must be renamed (the extension must be changed) to be able to install it as a Firefox plugin (just open the file in Firefox).
After the installation, a simple Start of the project class should be enough to start the webDriver Firefox plugin.

Consuming Services – the Spring Web Service Client

There is a downloadable zipped Eclipse project at the end of this post.

Having created a sample (mail) web service some posts ago, we would like to have a client consuming this service.

With Spring Web Services, this can easily be done by using the ubiquitous Template Pattern and, in this special case, the WebServiceTemplate . As the project structure can be downloaded, only the two important snippets are shown here:

The application-config.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

    <bean id="mailclient" class="de.mail.MailClient">
    	<property name="webServiceTemplate" ref="webServiceTemplate"/>
    </bean>

    <bean id="messageFactory" class="org.springframework.ws.soap.saaj.SaajSoapMessageFactory"/>

    <bean id="webServiceTemplate" class="org.springframework.ws.client.core.WebServiceTemplate">
        <constructor-arg ref="messageFactory"/>
        <property name="marshaller" ref="marshaller"/>
        <property name="unmarshaller" ref="marshaller"/>
        <property name="defaultUri" value="http://localhost:8080/mailservice"/>
    </bean>

    <bean id="marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
		<property name="classesToBeBound">
			<list>
				<value>org.mail.mailschema.MailRequest</value>
				<value>org.mail.mailschema.MailResponse</value>
			</list>
		</property>
	</bean>

</beans>

and the actual Web Service client:

package de.mail;

import org.mail.mailschema.MailRequest;
import org.mail.mailschema.MailResponse;
import org.springframework.ws.client.core.WebServiceTemplate;

public class MailClient {

	private WebServiceTemplate webServiceTemplate;

	public void setWebServiceTemplate(WebServiceTemplate webServiceTemplate) {
		this.webServiceTemplate = webServiceTemplate;
	}

	public boolean send(String user, String password, String from, String to, String subject, String content) {
		MailRequest request = new MailRequest();
		request.setFrom(from);
		request.setTo(to);
		request.setUser(user);
		request.setPassword(password);
		request.setContent(content);
		request.setSubject(subject);
		return ((MailResponse)webServiceTemplate.marshalSendAndReceive(request)).isResult();
	}

}

The servlet sample is not shown, since it is just for demonstration and not a nice piece of code. However, it exemplifies some Spring antipatterns, because DI is not used and the servlet is not Springified at all. But it’s only a sample showing how to use the Web Service client.
By deploying the servlet (or using mvn jetty:run again), a mail can be send by calling http://server/mailwebapp/Mailer?user=USER&password=PASSWORD&to=RECIPIENT@BLA.DE&from=me&subject=SUBJECT&content=MAILTEXT
But the actual code could be much nicer (also, it is definitely not safe – just a counter and a whitelist is added to prevent immediate abuse )

Sample Mail Web Service client project:

Instructions:

1. Download the wsclient-sample
2. Download the mailwebapp-sample
3. Download the maildatabinding

If correctly imported as Eclipse projects, the dependencies should be automatically resolved.
The mailwebapp can be deployed as a usual Web application which uses the Web Service client to call the Mail Web Service, which must be already deployed (compare the Spring Web Service post).

Patters (cont’d): The Observer Pattern

Download Eclipse project

I implemented the Observer Pattern with a central registry which is synchronized and uses WeakHashMaps, thereby trying to omit memory leaks.

The Notification Manager:

package observer;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.WeakHashMap;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantReadWriteLock;

public class NotificationManager implements Observable {

	private Map<Integer, List<Observer>> observers = new WeakHashMap<Integer, List<Observer>>();

	private final ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
	private final Lock read = readWriteLock.readLock();
	private final Lock write = readWriteLock.writeLock();

	public void notifyObservers(Object source, Object arg) {
		List<Observer> obs = null;
		read.lock();
		try {
			if (!observers.containsKey(source.hashCode())) {
				return;
			} else {
				obs = observers.get(source.hashCode());
			}
			for (Observer observer : obs) {
				observer.update(arg);
			}
		} finally {
			read.unlock();
		}
	}

	public void register(Object target, Observer observer) {
		List<Observer> obs = null;
		write.lock();
		try {
			if (!observers.containsKey(target.hashCode())) {
				obs = new ArrayList<Observer>();
			} else {
				obs = observers.get(target.hashCode());
			}
			if (!obs.contains(observer)) {
				obs.add(observer);
			}
			observers.put(target.hashCode(), obs);
		} finally {
			write.unlock();
		}
	}

	public void remove(Object target, Observer observer) {
		List<Observer> obs = null;
		write.lock();
		try {
			if (!observers.containsKey(target.hashCode())) {
				return;
			} else {
				obs = observers.get(target.hashCode());
			}
			if (!obs.contains(observer)) {
				return;
			}
			obs.remove(observer);
		} finally {
			write.unlock();
		}
	}

	public String toString() {
		return "#Observers: ".concat(String.valueOf(observers.entrySet().size()));
	}

}

The Observer interface:

package observer;

public interface Observer {

	public void update(Object arg);

}

The Observable interface implemented by the NotificationManager:

package observer;

public interface Observable {

	public void register(Object target, Observer observer);
	public void remove(Object target, Observer observer);

	public void notifyObservers(Object source, Object arg);

}

The unit test class extending AbstractDependencyInjectionSpringContextTests:

package observer;

import org.springframework.test.AbstractDependencyInjectionSpringContextTests;

public class ObserverTest extends AbstractDependencyInjectionSpringContextTests implements Observer {

	protected NotificationManager notificationmanager;

	public ObserverTest() {
		setPopulateProtectedVariables(true);
	}

	public ObserverTest(String s) {
		super(s);
		setPopulateProtectedVariables(true);
	}

	public void testNM() {
		ObservableObject oo = new ObservableObject();
		notificationmanager.register(oo, this);
		oo.notifyObs();
		notificationmanager.remove(oo, this);
		oo.notifyObs();
	}

	@Override
	protected String[] getConfigLocations() {
		return new String[]{"applicationContext-test.xml"};
	}

	public void update(Object arg) {
		System.out.println("called with " + arg);
	}

	private class ObservableObject {

		public void notifyObs() {
			notificationmanager.notifyObservers(this, "bla");
		}

	}

}

The config, the notificationmanager is injected into the protected variable in the junit test:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN"
"http://www.springframework.org/dtd/spring-beans.dtd">
<beans>

	<bean id="notificationmanager" class="observer.NotificationManager"/>

</beans>

Patterns with Spring

Download Eclipse project

There are several articles about Patterns with Spring in “Enterprise” context, ie. on the server side and simulating more or less typical JEE Patterns. These are important, however, Spring has its use in small, batch and client side apps as well.
Therefore, it makes sense to recall old school Design Patterns, which can be used independently of the programming context, just to reuse good solutions for old problems.
However, working with Spring, it’s easy to recognize that Patterns must be adapted to Spring, since some of them do not make sense when working with Spring.
I will elaborate the most important and famous ones here in the context of Spring usage.

Factory Method Pattern

For two reasons, the Factory Method Pattern is not really useful in the Spring context:

1. Spring’s BeanFactory implements a kind of Factory Pattern itself. It is easy to use different BeanFactories and therefore create different beans according to context. For example, using several XML-configurations and instantiating them in different ApplicationContexts gives you several bean factories configured with XML.
2. Since with Spring injecting different beans is done by XML-configurations and pseudo-Singletons are used, the advantage of having a single point of changing bean instantiation is inherent to Spring.

Nevertheless, I implemented the almost original Factory Method Pattern below.

Usually, the beans-factory-lookup-method-injection is referred as the factory method implementation in spring. Thought this is certainly true with respect to the actual pattern, it is difficult to see the advantage of this approach in contrast to just injecting a bean. This is even more transparent and there is no magic in it.
To actually make use of creating different objects, I made up the following scenario, which allows for creating different objects during runtime. There is nothing special in it, just plain usage of Spring, magic excluded.

package sample.factory;

public enum Type {

	FIRST, SECOND;

}

An interface to identifying the objects:

package sample.factory;

public interface ClassType {

	Type getId();

}

First type of object:

package sample.factory;

public class FirstType implements ClassType {

	@Override
	public Type getId() {
		return Type.FIRST;
	}

	@Override
	public String toString() {
		return getId().toString();
	}

}

Second type of object:

package sample.factory;

public class SecondType implements ClassType {

	@Override
	public Type getId() {
		return Type.SECOND;
	}

	@Override
	public String toString() {
		return getId().toString();
	}

}

The factory itself, objects will be injected:

package sample.factory;

import java.util.Map;

public class FactorySample {

	private Map<Type, ClassType> identifiers;

	public void setConcreteClasses(Map<Type, ClassType> i) {
		this.identifiers = i;
	}

	public ClassType createClassOfType(Type type) {
		switch (type) {
			case FIRST : return identifiers.get(type);
			case SECOND : return identifiers.get(type);
			default : return null;
		}
	}

}

Main class which retrieves factory bean and creates a object of second type:

package sample.factory;

import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class Main {

	public static void main(String[] args) {
		ApplicationContext app = new ClassPathXmlApplicationContext("beans.xml");
		FactorySample factory = (FactorySample) app.getBean("factory");
		ClassType i = factory.createClassOfType(Type.SECOND);
		System.out.println(i);
	}

}

Finally, the spring config (entry key Strings are converted automagically by Spring):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN"
"http://www.springframework.org/dtd/spring-beans.dtd">
<beans>

	<bean id="factory" class="sample.factory.FactorySample">
		<property name="concreteClasses">
			<map>
				<entry key="FIRST">
					<ref local="FirstType" />
				</entry>
				<entry key="SECOND">
					<ref local="SecondType" />
				</entry>
			</map>
		</property>
	</bean>

	<bean id="FirstType" class="sample.factory.FirstType"/>

	<bean id="SecondType" class="sample.factory.SecondType"/>

</beans>

Mailing with Velocity

Download Eclipse project

Java Mailing is also leveraged by Spring, in the usual templating way some optimisations have been added. Therefore, we will write a small mailing application here.
We will add Velocity for mail templating.

• As always, start with a simple Maven project.
• The following part is almost copied from the Spring Email manual

MailSample:

package sample.mail;

import java.util.HashMap;
import java.util.Map;

import javax.mail.internet.MimeMessage;

import org.apache.velocity.app.VelocityEngine;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.springframework.mail.javamail.JavaMailSender;
import org.springframework.mail.javamail.MimeMessageHelper;
import org.springframework.mail.javamail.MimeMessagePreparator;
import org.springframework.ui.velocity.VelocityEngineUtils;

public class MailSample {

	private JavaMailSender mailSender;
	private VelocityEngine velocityEngine;

	public void setMailSender(JavaMailSender mailSender) {
		this.mailSender = mailSender;
	}

	public void setVelocityEngine(VelocityEngine velocityEngine) {
		this.velocityEngine = velocityEngine;
	}

	public static void main(String[] args) throws Exception {
		ApplicationContext app = new ClassPathXmlApplicationContext(
				"applicationContext.xml");
		MailSample mailer = (MailSample) app.getBean("mailer");
		mailer.send();
	}

	private void send() {
	      MimeMessagePreparator preparator = new MimeMessagePreparator() {
	          public void prepare(MimeMessage mimeMessage) throws Exception {
	             MimeMessageHelper message = new MimeMessageHelper(mimeMessage);
	             message.setTo("mail.receiver@web.de");
	             message.setFrom("mail.sender@web.de"); // could be parameterized...
	             Map model = new HashMap();
	             model.put("key", "testtext");
	             String text = VelocityEngineUtils.mergeTemplateIntoString(
	                velocityEngine, "test.vm", model);
	             message.setText(text, true);
	          }
	       };
	       this.mailSender.send(preparator);
	}
}

• the following part is also copied from the Email tutorial.

applicationContext.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

	<bean id="mailSender" class="org.springframework.mail.javamail.JavaMailSenderImpl">
<property name="host">
			<value>server.com</value>
		</property>
<property name="javaMailProperties">
<props>
<prop key="mail.smtp.auth">true</prop>
			</props>
		</property>
<property name="username"><value>username</value></property>
<property name="password"><value>password</value></property>
	</bean>

	<bean id="mailer" class="sample.mail.MailSample">
<property name="mailSender" ref="mailSender" />
<property name="velocityEngine" ref="velocityEngine" />
	</bean>

   <bean id="velocityEngine" class="org.springframework.ui.velocity.VelocityEngineFactoryBean">
<property name="velocityProperties">
         <value>
            resource.loader=class
            class.resource.loader.class=org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader
         </value>
      </property>
   </bean>

</beans>

• we just need two more files, of course the POM.

POM:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>velocitymail</groupId>
  <artifactId>velocitymail</artifactId>
  <name>velocitymail</name>
  <version>0.0.1-SNAPSHOT</version>
  <description/>
  <dependencies>
  	<dependency>
  		<groupId>org.springframework</groupId>
  		<artifactId>spring</artifactId>
  		<version>2.5.5</version>
  	</dependency>
  	<dependency>
  		<groupId>javax.mail</groupId>
  		<artifactId>mail</artifactId>
  		<version>1.4</version>
  	</dependency>
  	<dependency>
  		<groupId>velocity</groupId>
  		<artifactId>velocity</artifactId>
  		<version>1.4</version>
  	</dependency>
  </dependencies>
</project>

• and the velocity template, which is very simple in this case.

test.vm:

hello $key

and finally the project structure:

Quick Tip: Testing Spring

Download Eclipse project

You can read detailed and good written documentation about unit testing with Spring here.
But as always, when combining Spring with Maven using Eclipse, some things have to be considered.

• Create a “no archetype” Maven project in Eclipse (compare the other posts).
  
• Create the test case first (following TDD) with Eclipse. This can be a JUnit 3 or 4 test case.
• Afterwards, within Eclipse the class generation can easily be done using the Quick Fix funtionality (Strg + 1).

The test case must extends Spring’s AbstractDependencyInjectionSpringContextTests, making it much easier to use Spring in unit tests. It is best combined with the call to setPopulateProtectedVariables(true) in the constructor. Using this feature, Spring can inject Beans into protected fields, avoiding the need for setters in test cases.
By defining getConfigLocations(), special Spring configfiles for unit tests can be chosen. However, using Maven, we are always seperating test and main resources (compare project structure).

Test case:

package test.sample;

import org.springframework.test.AbstractDependencyInjectionSpringContextTests;

public class SpringTest extends AbstractDependencyInjectionSpringContextTests {

	protected BeanUnderTest testbean;

	public SpringTest() {
		setPopulateProtectedVariables(true);
	}

	public void testBeanUnderTest() {
		assertEquals(1, testbean.return1());
	}

	@Override
	protected String[] getConfigLocations() {
		return new String[]{"applicationContext-test.xml"};
	}
}

Class under test (BeanUnderTest):

package test.sample;

public class BeanUnderTest {

	public int return1() {
		return 1;
	}

}

applicationContext-test.xml (currently this is the only config):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN"
"http://www.springframework.org/dtd/spring-beans.dtd">
<beans>

	<bean id="testbean" class="test.sample.BeanUnderTest"/>

</beans>

And the Maven POM. It is important to include the Compiler 5 switches if JUnit 4 is used:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>testingsample</groupId>
	<artifactId>testingsample</artifactId>
	<name>testingsample</name>
	<version>0.0.1-SNAPSHOT</version>
	<description />
	<build>
		<finalName>testingsample</finalName>
                <plugins>
                        <plugin>
				<artifactId>maven-compiler-plugin</artifactId>
				<configuration>
					<source>1.5</source>
					<target>1.5</target>
				</configuration>
			</plugin>
		</plugins>
	</build>
	<dependencies>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring</artifactId>
			<version>2.5.5</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-test</artifactId>
			<version>2.5.5</version>
		</dependency>
	</dependencies>
</project>

That’s it, Maven, JUnit and Spring can be combined very smoothly and Spring again shows with the AbstractDependencyInjectionSpringContextTests how it provides the useful little glue between these components.