Java allocate objects in the runtime memory data area called heap. The heap is created on virtual machine start-up. The memory allocated to objects are reclaimed by Garbage Collectors when the object is no longer referenced (See Object References). The heap may be of a fixed size, but can also be expanded when more memory is needed or contracted when no longer needed. If a larger heap is needed and it cannot be allocated an OutOfMemory is thrown. See JVM Specification.

Native memory is used by the JVM to perform its operations like creation of threads, sockets, jdbc drivers using native code, libraries using native code, etc.

The maximum size of heap memory is determined by the -Xmx on the java command line. If Xmx is not set, then the JVM decides for the maximum heap. The heap and native memory are limited to the maximum memory allocated by the JVM. For example, if the JVM Xmx is set to 1GB and currently use of native memory is 256MB then the heap can only use 768MB.

Common causes of out of memory:

• Not enough heap - The JVM needs more memory to deal with the application requirements. Queries returning more objects than usual can be the cause.
• Memory Leaks - The application does not close the resources, like the PersistenceManager, EntityManager or Queries, and the JVM cannot reclaim the memory.
• Caching - Caching in the application or inside JPOX holding strong references to objects.
• Garbage Collection - If no full garbage collection is performed before the OutOfMemory it can indicate a bug in the JVM Garbage Collector.
• Memory Fragmentation - A large object needs to be placed in the memory, but the JVM cannot allocate a continous space to it because the memory is fragmented.
• JDBC driver - a bug in the JDBC driver not flushing resources or keeping large result sets in memory.

JVM

Collect garbage collection information by adding -verbosegc to the java command line. The verbosegc flag will print garbage collections to System output.

The Sun JVM 1.4 or upper also accepts the flag -XX:+PrintGCDetails, which prints detailed information on Garbage Collections.

JPOX

JPOX keeps in cache persistent objects using weak references by default. Enable debug mode JPOX.Cache category to investigate the size of the cache in JPOX.

JPOX can be configured to reduce the number of objects in cache. JPOX has cache for persistent objects, metadata, datastore metadata, fields of type Collection or Map, or query results.

Query Results Cache

The query results hold strong references to the retrieved objects. If a query returns too many objects it can lead to OutOfMemory error. To be able to query over large result sets, change the result set type to scroll-insensitive in the pmf setting org.jpox.query.resultSetType.

Query leak

The query results are kept in memory until the PersistenceManager or Query are closed. To avoid memory leaks caused by queries in memory, it's capital to explicitly close the query as soon as possible. The following snipet shows how to do it.

Query query = pm.newQuery("SELECT FROM org.jpox.samples.store.Product WHERE price < :limit");
List results = (List)query.execute(new Double(200.0));
//...
//...
//closes the query
query.closeAll(); 

PersistenceManager leak

It's also a best practice to ensure the PersistenceManager is closed in a try finally block. The PersistenceManager has level 1 cache of persistence objects. See the following example:

PersistenceManager pm = pmf.getPersistenceManager();
Transaction tx = pm.currentTransaction();
try
{
    tx.begin();
    //...
    tx.commit();
}
finally
{
    if (tx.isActive())
    {
        tx.rollback();
    }
    pm.close();
}

Cache for fields of Collection or Map

If collection or map fields have large number of elements, the caching of elements can be disabled with the pmf org.jpox.cache.collections setting it to false.

Persistent Objects cache

The cache control of persistent objects is described in the cache documentation.

Metadata and Datastore Metadata cache

The metadata and datastore metadata caching cannot be controled by the application, because the memory required for it is insignificant.

OutOfMemory when persisting new objects

When persistent many objects, the flush operation should be periodically invoked. This will give a hint to JPOX to flush the changes to the database and release the memory. In the below sample the pm.flush() operation is invoked on every 10.000 objects persisted.

PersistenceManager pm = pmf.getPersistenceManager();
Transaction tx = pm.currentTransaction();
try
{
    tx.begin();
    for( int i=0; i<100000; i++)
    {
        Wardrobe wardrobe = new Wardrobe();
        wardrobe.setModel("3 doors");
        pm.makePersistent(wardrobe);
        if( i % 10000 == 0)
        {
            pm.flush();
        }
    }
    tx.commit();
}
finally
{
    if (tx.isActive())
    {
        tx.rollback();
    }
    pm.close();
}

The application pauses for short or long periods or hangs during very long time.

Common causes:

• Database Locking - Database waiting other transactions to release locks due to deadlock or locking contentions.
• Garbage Collection Pauses - The garbage collection pauses the application to free memory resources.
• Application Locking - Thread 2 waiting for resources locked by Thread 1.

Database locking

Use a database specific tool or database scripts to find the current database locks.

In Microsoft SQL, the stored procedured sp_lock can be used to examinate the database locks.

Query Timeout

To avoid database locking to hang the application when a query is performed, set the query timeout. See Query Timeout.

Garbage Collection pauses

Check if the application freezes when the garbage collection starts. Add -verbosegc to the java command line and restart the application.

Application Locking

Thread dumps are snapshots of the threads and monitors in the JVM. Thread dumps help to diagnose applications by showing what the application is doing at a certain moment of time.

To generate Thread Dumps in MS Windows, press <ctrl><break> in the window running the java application.

To generate Thread Dumps in Linux/Unix, execute kill -3 process_id

To effectivelly diagnose the problem, take 5 Thread Dumps with 3 to 5 seconds internal between each one.

See An Introduction to Java Stack Traces.