HOWTO Use Datanucleus with OSGi and Spring DM

This guide is based on my personal experience and is not the authoritative guide to using DataNucleus with OSGi and Spring DM.

This guide is work in progress

This guide assumes the reader is familiar with concepts like OSGi, Spring, JDO, DataNucleus etc.. This guide only explains how to wire these technologies together and not how they work.

Technologies used in this guide are:

• IDE (Eclipse 3.5.1)
  • Spring IDE Plugin
  • Datanucleus IDE Plugin
• OSGi (Equinox 3.5)
• JDO (DataNucleus 2.0.0)
• Dependency injection (Spring / Spring DM 2.0.0M1)
• Datastore (PostgreSQL 8.3, altough any datastore could be used)

Create a target platform

We are going to start by creating a clean OSGi target platform. Start by creating an empty directory which is going to house all the bundles for our target platform.

Step 1 - OSGi

It is important to realize that the Datanucleus plugin system uses the Eclipse extensions system and NOT the plain OSGi facilities. For DataNucleus' plugin system to work correctly we need to have the Eclipse extension system bundles in our target platform. So we need Eclipse Equinox to create the standardized OSGi platform and we need some 'non-standard' Eclipse bundles for the registry platformDataNucleus uses for its plugin system. This means we need the following bundles:

• org.eclipse.core.contenttype_3.4.1.R35x_v20090826-0451.jar (Grab from your Eclipse install or from the Eclipse Core project)
• org.eclipse.core.jobs_3.4.100.v20090429-1800.jar (Grab from your Eclipse install or from the Eclipse Core project)
• org.eclipse.core.runtime_3.5.0.v20090525.jar (Grab from your Eclipse install or from the Eclipse Core project)
• org.eclipse.equinox.app_1.2.0.v20090520-1800.jar (Download from the Eclipse Equinox project)
• org.eclipse.equinox.common_3.5.1.R35x_v20090807-1100.jar (Download from the Eclipse Equinox project)
• org.eclipse.equinox.preferences_3.2.300.v20090520-1800.jar (Download from the Eclipse Equinox project)
• org.eclipse.equinox.registry_3.4.100.v20090520-1800.jar (Download from the Eclipse Equinox project)
• org.eclipse.osgi.services_3.2.0.v20090520-1800.jar (Download from the Eclipse Equinox project)
• org.eclipse.osgi-3.5.1.R35x_v20090827.jar (Part of the Spring DM with Deps distribution or download from the Eclipse Equinox project)

Step 2 - Adding DI

We are now going to add the Spring, Spring ORM, Spring JDBC, Spring Transaction and Spring DM bundles to our target platform.

The following bundles come from the Spring distribution (The 'real' Spring Framework that is, not the Spring Dynamic Modules sub project). The Spring DM 2.0.0M1 distribution comes with some of the bundles described below, however these are release candidates and not final. Its advisable to use the non-RC versions from the 'real' Spring distribution:

• org.springframework.aop-3.0.0.RELEASE.jar
• org.springframework.asm-3.0.0.RELEASE.jar
• org.springframework.beans-3.0.0.RELEASE.jar
• org.springframework.context-3.0.0.RELEASE.jar
• org.springframework.context.support-3.0.0.RELEASE.jar
• org.springframework.core-3.0.0.RELEASE.jar
• org.springframework.expression-3.0.0.RELEASE.jar
• org.springframework.orm-3.0.0.RELEASE.jar
• org.springframework.jdbc-3.0.0.RELEASE.jar
• org.springframework.transaction-3.0.0.RELEASE.jar
• org.springframework.expression-3.0.0.RELEASE.jar

The following bundles come from the Spring DM distribution:

• spring-osgi-annotation-2.0.0.M1.jar
• spring-osgi-core-2.0.0.M1.jar
• spring-osgi-extender-2.0.0.M1.jar
• spring-osgi-io-2.0.0.M1.jar

Step 3 - Adding ORM

We are now going to add JDO and DataNucleus to our target platform.

• datanucleus-core-2.0.0-release.jar
• datanucleus-rdbms-2.0.0-release.jar
• jdo2-api-2.3-ec.jar

Step 4 - Adding miscellaneous bundles

The following bundles are dependencies of our core bundles and can be downloaded from the Spring Enterprise Bundle Repository ( http://www.springsource.com/repository/app/ ):

• com.springsource.org.aopalliance-1.0.0.jar (Dependency of Spring AOP, the core AOP bundle. )
• com.springsource.org.apache.commons.logging-1.1.1.jar (Dependency of various Spring bundles, logging abstraction library.)

Overview

This is how the directory housing the target platform looks on my PC:

$ ls -las
total 9228
   4 drwxrwxr-x  2 siepkes siepkes    4096 2010-01-13 22:17 .
   4 drwxrwxr-x 12 siepkes siepkes    4096 2010-01-13 21:42 ..
   8 -rw-rw-r--  1 siepkes siepkes    4615 2009-09-27 23:56 com.springsource.org.aopalliance-1.0.0.jar
  68 -rw-rw-r--  1 siepkes siepkes   61464 2009-12-31 10:50 com.springsource.org.apache.commons.logging-1.1.1.jar
1928 -rw-r--r--  1 siepkes siepkes 1968523 2010-01-07 10:46 datanucleus-core-2.0.0-release.jar
1188 -rw-r--r--  1 siepkes siepkes 1210341 2010-01-07 10:46 datanucleus-rdbms-2.0.0-release.jar
 200 -rw-r--r--  1 siepkes siepkes  198552 2010-01-07 10:46 jdo2-api-2.3-ec.jar
  92 -rw-rw-r--  1 siepkes siepkes   87505 2009-10-05 14:44 org.eclipse.core.contenttype_3.4.1.R35x_v20090826-0451.jar
  84 -rw-rw-r--  1 siepkes siepkes   81848 2009-06-12 00:38 org.eclipse.core.jobs_3.4.100.v20090429-1800.jar
  76 -rw-rw-r--  1 siepkes siepkes   69761 2009-06-12 00:38 org.eclipse.core.runtime_3.5.0.v20090525.jar
  84 -rw-rw-r--  1 siepkes siepkes   78162 2009-09-17 20:21 org.eclipse.equinox.app_1.2.0.v20090520-1800.jar
 104 -rw-rw-r--  1 siepkes siepkes   99448 2009-12-31 13:45 org.eclipse.equinox.common_3.5.1.R35x_v20090807-1100.jar
 108 -rw-rw-r--  1 siepkes siepkes  105368 2009-09-17 20:21 org.eclipse.equinox.preferences_3.2.300.v20090520-1800.jar
 176 -rw-rw-r--  1 siepkes siepkes  173316 2009-12-31 13:42 org.eclipse.equinox.registry_3.4.100.v20090520-1800.jar
1104 -rw-rw-r--  1 siepkes siepkes 1125860 2009-09-27 23:56 org.eclipse.osgi-3.5.1.R35x_v20090827.jar
  72 -rw-rw-r--  1 siepkes siepkes   66065 2009-12-31 14:58 org.eclipse.osgi.services_3.2.0.v20090520-1800.jar
 324 -rw-r--r--  1 siepkes siepkes  324242 2009-12-16 16:00 org.springframework.aop-3.0.0.RELEASE.jar
  56 -rw-rw-r--  1 siepkes siepkes   53081 2009-12-31 12:49 org.springframework.asm-3.0.0.RELEASE.jar
 544 -rw-r--r--  1 siepkes siepkes  551967 2009-12-16 16:00 org.springframework.beans-3.0.0.RELEASE.jar
 648 -rw-r--r--  1 siepkes siepkes  657492 2009-12-16 16:00 org.springframework.context-3.0.0.RELEASE.jar
 104 -rw-r--r--  1 siepkes siepkes  101176 2009-12-16 16:00 org.springframework.context.support-3.0.0.RELEASE.jar
 356 -rw-r--r--  1 siepkes siepkes  359338 2009-12-16 16:00 org.springframework.core-3.0.0.RELEASE.jar
 156 -rw-r--r--  1 siepkes siepkes  154200 2009-12-16 16:00 org.springframework.expression-3.0.0.RELEASE.jar
 380 -rw-r--r--  1 siepkes siepkes  381893 2009-12-16 16:00 org.springframework.jdbc-3.0.0.RELEASE.jar
 328 -rw-r--r--  1 siepkes siepkes  330198 2009-12-16 16:00 org.springframework.orm-3.0.0.RELEASE.jar
 232 -rw-r--r--  1 siepkes siepkes  231660 2009-12-16 16:00 org.springframework.transaction-3.0.0.RELEASE.jar
  24 -rw-r--r--  1 siepkes siepkes   24204 2009-09-28 02:29 spring-osgi-annotation-2.0.0.M1.jar
 560 -rw-r--r--  1 siepkes siepkes  565742 2009-09-28 02:25 spring-osgi-core-2.0.0.M1.jar
 184 -rw-r--r--  1 siepkes siepkes  180718 2009-09-28 02:26 spring-osgi-extender-2.0.0.M1.jar
  36 -rw-r--r--  1 siepkes siepkes   35871 2009-09-28 02:24 spring-osgi-io-2.0.0.M1.jar

Creating our application base

Here I will show how one can create a base for an application with our newly created target platform.

Setup Eclipse

Create a Target Platform in Eclipse by going to 'Window' -> 'Preferences' -> 'Plugin Development' -> 'Target Platform' and press the 'Add' button. Select 'Nothing: Start with an empty target platform', give the platform a name and point it to the directory we put all the jars/bundles in. When you are done press the 'Finish' button. Indicate to Eclipse we want to use this new platform by ticking the checkbox in front of our newly created platform in the 'Target Platform' window of the 'Preferences' screen.

Create a new project in Eclipse by going to 'File' -> 'New...' -> 'Project' and Select 'Plug-in Project' under the 'Plugin development' leaf. Give the project a name (I'm going to call it 'nl.siepkes.test.project.a' in this example). In the radiobox options 'This plugin is targetted to run with:' select 'An OSGi framework' -> 'standard'. Click 'Next'. Untick the 'Generate an activator, a Java class that....' and press 'Finish'.

Obviously Eclipse is not the mandatory IDE for the steps described above. Other technologies can be used instead. For this guide I used Eclipse because it is easy to explain, but for most of my projects I use Maven.

If you have the Spring IDE plugin installed (which is advisable if you use Spring) you can add a Spring Nature to your project by right clicking your project and then clicking 'Spring Tools' -> 'Add Spring Nature'. This will enable error detection in your Spring bean configuration file.

Create a directory called 'spring' in your 'META-INF' directory. In this directory create a Spring bean configuration file by right clicking the directory and click 'New...' -> 'Other...'. A menu called 'New' will popup, select 'Spring Bean Configuration File'. Call the file beans.xml.

Declare a Persistence Manager Factory Bean inside the beans.xml:

<bean id="pmf" class="nl.siepkes.util.DatanucleusOSGiLocalPersistenceManagerFactoryBean">
	<property name="jdoProperties">
		<props>
			<prop key="javax.jdo.PersistenceManagerFactoryClass">org.datanucleus.jdo.JDOPersistenceManagerFactory</prop>
			<!-- PostgreSQL DB connection settings -->
			<!-- Add '?loglevel=2' to Connection URL for JDBC Connection debugging. -->
			<prop key="javax.jdo.option.ConnectionURL">jdbc:postgresql://localhost/testdb</prop>
			<prop key="javax.jdo.option.ConnectionDriverName">org.postgresql.Driver</prop>
			<prop key="datanucleus.validateColumns">true</prop>
			<prop key="javax.jdo.option.ConnectionUserName">foo</prop>
			<prop key="javax.jdo.option.ConnectionPassword">bar</prop>
			<prop key="datanucleus.storeManagerType">rdbms</prop>
			<prop key="datanucleus.transactionIsolation">repeatable-read</prop>

			<prop key="datanucleus.autoCreateSchema">true</prop>
			<prop key="datanucleus.validateTables">true</prop>
			<prop key="datanucleus.validateConstraints">true</prop>
			<prop key="datanucleus.rdbms.CheckExistTablesOrViews">true</prop>
		</props>
	</property>
</bean>

<osgi:service ref="pmf" interface="javax.jdo.PersistenceManagerFactory" />

You can specify all the JDO/DataNucleus options you need with '<prop key="foo_key">bar_value</prop>'.

Notice the '<osgi:service ref="pmf" interface="javax.jdo.PersistenceManagerFactory" />' line. This exports our persistence manager as an OSGi sevice and makes it possible for other bundles to access it.

Also notice that the Persistence Manager Factory is not the normal LocalPersistenceManagerFactoryBean class, but instead the OSGiLocalPersistenceManagerFactoryBean class. The OSGiLocalPersistenceManagerFactoryBean is NOT part of the default DataNucleus distribution. So why do we need to use the OSGiLocalPersistenceManagerFactoryBean instead of the default LocalPersistenceManagerFactoryBean ? The default LocalPersistenceManagerFactoryBean is not aware of the OSGi environment and expects all classes to be loaded by one single classloader (this is the case in a normal Java environment without OSGi). This makes the LocalPersistenceManagerFactoryBean unable to locate its plugins.

The OSGiLocalPersistenceManagerFactoryBean is a subclass of the LocalPersistenceManagerFactoryBean and is aware of the OSGi environment:

public class OSGiLocalPersistenceManagerFactoryBean extends LocalPersistenceManagerFactoryBean implements BundleContextAware {

	private BundleContext bundleContext;
	private DataSource dataSource;

	public DatanucleusOSGiLocalPersistenceManagerFactoryBean() {

	}

	@Override
	protected PersistenceManagerFactory newPersistenceManagerFactory(String name) {
		return JDOHelper.getPersistenceManagerFactory(name, getClassLoader());
	}

	@Override
	protected PersistenceManagerFactory newPersistenceManagerFactory(Map props) {
		ClassLoader classLoader = getClassLoader();

		props.put("datanucleus.primaryClassLoader", classLoader);

		PersistenceManagerFactory pmf = JDOHelper.getPersistenceManagerFactory(props, classLoader);

		return pmf;
	}

	private ClassLoader getClassLoader() {
		ClassLoader classloader = null;
		Bundle[] bundles = bundleContext.getBundles();

		for (int x = 0; x < bundles.length; x++) {

			if ("org.datanucleus.store.rdbms".equals(bundles[x].getSymbolicName())) {
				try {
					classloader = bundles[x].loadClass("org.datanucleus.JDOClassLoaderResolver").getClassLoader();
				} catch (ClassNotFoundException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}
				break;
			}
		}

		return classloader;
	}

	@Override
	public void setBundleContext(BundleContext bundleContext) {
		this.bundleContext = bundleContext;

	}
}

If we create an new, similear (Plug-in) project, for example 'nl.siepkes.test.project.b' we can import/use our Persistance Manager Factory service by specifying the following in its beans.xml:

<osgi:reference id="pmf" interface="javax.jdo.PersistenceManagerFactory" />

The Persistance Manager Factory (pmf) bean can then be injected into other beans as you normally would do when using Spring and JDO/DataNucleus together.

Accessing your services from another bundle

The reason why you are probably using OSGi is because you want to separate/modularize all kinds of code. A common use case is that you have your service layer in bundle A and another bundle, bundle B, who invokes methods in your service layer. Bundle B knows absolutely nothing about DataNucleus (ie. no imports and dependencies on DataNucleus or Datastore JDBC drivers) and will just call methods with signatures like 'public FooRecord getFooRecord(long fooId)'.

When you create such a setup and access a method in bundle A from bundle B you might be surprised to find out a ClassNotFound Exception is being thrown. The ClassNotFound exception will probably be about some DataNucleus or Datastore JDBC driver class not being found. How can bundle B complain about not finding implementation classes which only belong in bundle A (which has the correct imports) ? The reason for this is that when you invoke the method in bundle A from bundle B the classloader from bundle B is used to execute the method in bundle A. And since the classloader of bundle B does not have DataNucleus imports things go awry.

To solve this we need to change the ClassLoader in the ThreadContext which invokes the method in Bundle A. We could of course do this manually in every method in Bundle A but since we are already using Spring and AOP its much easier to do it that way.

Create the following class (which is our aspect that is going to do the heavy lifting) in bundle A:

package nl.siepkes.util;

/**
 * <p>
 * Aspect for setting the correct class loader when invoking a method in the
 * service layer.
 * </p>
 * <p>
 * When invoking a method from a bundle in the service layer of another bundle
 * the classloader of the invoking bundle is used. This poses the problem that
 * the invoking class loader needs to know about classes in the service layer of
 * the other bundle. This aspect sets the <tt>ContextClassLoader</tt> of the
 * invoking thread to that of the other bundle, the bundle that owns the method
 * in the service layer which is being invoked. After the invoke is completed
 * the aspect sets the <tt>ContextClassLoader</tt> back to the original
 * classloader of the invoker.
 * </p>
 *
 * @author Jasper Siepkes <jasper@siepkes.nl>
 * 
 */
public class BundleClassLoaderAspect implements Ordered {

	private static final int ASPECT_PRECEDENCE = 0;

	public Object setClassLoader(ProceedingJoinPoint pjp) throws Throwable {
		// Save a reference to the classloader of the caller
		ClassLoader oldLoader = Thread.currentThread().getContextClassLoader();
		// Get a reference to the classloader of the owning bundle
		ClassLoader serviceLoader = pjp.getTarget().getClass().getClassLoader();
		// Set the class loader of the current thread to the class loader of the
		// owner of the bundle
		Thread.currentThread().setContextClassLoader(serviceLoader);

		Object returnValue = null;

		try {
			// Make the actual call to the method.
			returnValue = pjp.proceed();
		} finally {
			// Reset the classloader of this Thread to the original
			// classloader of the method invoker.
			Thread.currentThread().setContextClassLoader(oldLoader);
		}

		return returnValue;
	}

	@Override
	public int getOrder() {
		return ASPECT_PRECEDENCE;
	}
}

Add the following to you Spring configuration in bundle A:

<tx:advice id="txAdvice" transaction-manager="txManager">
	<tx:attributes>	
		<tx:method name="get*" read-only="true" />		
		<tx:method name="*" />
	</tx:attributes>
</tx:advice>

<aop:pointcut id="fooServices" expression="execution(* nl.siepkes.service.*.*(..))" />
	<aop:advisor advice-ref="txAdvice" pointcut-ref="fooServices" />
		
	<!-- Ensures the class loader of this bundle is used to invoke public methods in the service layer of this bundle. -->
	<aop:aspect id="bundleLoaderAspect" ref="bundleLoaderAspectBean">
		<aop:around pointcut-ref="fooServices" method="setClassLoader"/>		
	</aop:aspect>
</aop:config>

Now all methods in classes in the package 'nl.siepkes.service' will always use the class loader of bundle A.