|
|
|
|
| Index | Recent Threads | Unanswered Threads | Who's Online | Help |
DataNucleus Forum
DataNucleus Support
Documentation
Thread: Effect of datanucleus.cache.collections=false
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 15
|
|
| Author |
|
chrisco
Expert Australia Joined: Nov 24, 2005 Post Count: 706 Status: Offline |
The datanucleus.cache.collections option appears to be a DataNucleus specific feature (i.e. not part of the JDO spec). Are the collections used to implement -to-many relationships in a class model at all usable when datanucleus.cache.collections=false? It appears not. It appears as though the intention of datanucleus.cache.collections=false is that a developer will write the app in such a way as to retrieve all objects in a -to-many via queries and not use the collection class (ArrayList, HashSet etc.,) that implements a -to-many relationship at all. I've tried using such collections and they always appear empty. Is this the correct understanding of the effect of datanucleus.cache.collections=false or am I doing something wrong? ---------------------------------------- Chris Colman http://visualclassworks.com Javelin: Visual modeling & coding with automatic JDO 2/DataNucleus meta data generation. http://expojo.com expojo: Simple dependency injection and exposed POJO domain model pattern |
||
|
|
andy
Expert UK Joined: Mar 13, 2004 Post Count: 5686 Status: Offline |
Are the collections used to implement -to-many relationships in a class model at all usable when datanucleus.cache.collections=false? It appears not. I've tried using such collections and they always appear empty. "It appears not" based on no evidence? I see. Simple access to a collection field will obviously send SQL, so see the log. That SQL statement returns a result set and so retrieves objects, also in the log. Obviously it works for me. ---------------------------------------- -Andy   |
||
|
|
chrisco
Expert Australia Joined: Nov 24, 2005 Post Count: 706 Status: Offline |
Are the collections used to implement -to-many relationships in a class model at all usable when datanucleus.cache.collections=false? It appears not. I've tried using such collections and they always appear empty. "It appears not" based on no evidence? I see. Simple access to a collection field will obviously send SQL, so see the log. That SQL statement returns a result set and so retrieves objects, also in the log. Obviously it works for me. "It appears not" refers to inuendo, rumours and heresay - the most frequently used and widely accepted forms of 'evidence' in the world today :) No seriously, it was based on the fact that a snapshot I had on 2009/2/13 was causing index out of bounds exceptions whenever I 'iterator.remove() ed' the current element pointed to by the iterator. It didn't matter what position the deleted element was in - which made me think that, at the time the element was being removed the collection was empty. No other conclusion could explain why deleting element at index 0 could generate an out of bounds exception. When I reverted to cache collections = true everything worked perfectly without generating the out of bound exceptions. I just downloaded the latest snapshot 2009/2/26 and the behaviour definitely seems different/better than before: now when I turn cache collections to false I don't see any out of bounds exceptions when I delete elements from the collection. The new behaviour isn't identical to cache collections = true for some reason - UI list box elements that update with the items in the collection are no longer updating after new items are added or deleted - they just continue to show the previous set of elements until I close the form and reload the form again. ---------------------------------------- Chris Colman http://visualclassworks.com Javelin: Visual modeling & coding with automatic JDO 2/DataNucleus meta data generation. http://expojo.com expojo: Simple dependency injection and exposed POJO domain model pattern |
||
|
|
chrisco
Expert Australia Joined: Nov 24, 2005 Post Count: 706 Status: Offline |
Wow! This newer build really rocks when it comes to setting cachecollections=false - no more out of bounds exceptions and it allows each session to always deal with the latest possible versions of objects - which makes is work heaps better in a multi user environment. I just have one point of confusion: JDO spec says: 12.6.1. Cache Management When the transaction in which persistent instances are created, deleted, or modified completes, eviction is automatically done by the transaction completion mechanisms. In my scenario I use "Open PersistentManager In View" via the expojo framework where the expojo servlet filter does this for each HTTP request: 1. Causes a new transaction to begin() 2. Services the HTTP request 3. Causes the transaction to be committed() The spec says that eviction is automatic after the transaction completes so theoretically none of my objects remain in the L1 cache at that point. This leads me to ask the question - why do I need to use explicit refresh calls to ensure I get fresh versions of objects from the database? i.e. if the objects have been evicted shouldn't reinstantiating them result in a fresh pull from the DB or a fetch from the L2 cache - which should always have the latest version of any object? Perhaps I'm confusing the situation by holding references to some of the objects in UI objects of the framework I'm using whose lifecycle can span many HTTP requests (welcome to AJAX!). If I hold a reference to an object and I'm not using detach on commit and not doing any explicit commit (not serializing either) then I hold references to objects that are persistent non transactional (I guess). What happens when the next HTTP request arrives and the transaction is opened? Are the instances that the UI elements hold references to added to the cache during the process of handling that HTTP request or are new instances created and added to the cache? ---------------------------------------- Chris Colman http://visualclassworks.com Javelin: Visual modeling & coding with automatic JDO 2/DataNucleus meta data generation. http://expojo.com expojo: Simple dependency injection and exposed POJO domain model pattern |
||
|
|
chrisco
Expert Australia Joined: Nov 24, 2005 Post Count: 706 Status: Offline |
I've worked out how to force update - refreshAll seems to not quite cut it as is works on transactional instances only - would be nice if we could do a refresh on the entire L1 cache. The only way I've found that works is by explicitly calling refresh(Object) on individual objects - which becomes a long and boring coding process when complex object graphs are involved. Is there an easier way that I haven't stumbled upon yet? ---------------------------------------- Chris Colman http://visualclassworks.com Javelin: Visual modeling & coding with automatic JDO 2/DataNucleus meta data generation. http://expojo.com expojo: Simple dependency injection and exposed POJO domain model pattern |
||
|
|
ganzuoni
Expert Joined: Nov 16, 2005 Post Count: 1009 Status: Online |
Wow! This newer build really rocks when it comes to setting cachecollections=false - no more out of bounds exceptions and it allows each session to always deal with the latest possible versions of objects - which makes is work heaps better in a multi user environment. I just have one point of confusion: JDO spec says: 12.6.1. Cache Management When the transaction in which persistent instances are created, deleted, or modified completes, eviction is automatically done by the transaction completion mechanisms. In my scenario I use "Open PersistentManager In View" via the expojo framework where the expojo servlet filter does this for each HTTP request: 1. Causes a new transaction to begin() 2. Services the HTTP request 3. Causes the transaction to be committed() The spec says that eviction is automatic after the transaction completes so theoretically none of my objects remain in the L1 cache at that point. This leads me to ask the question - why do I need to use explicit refresh calls to ensure I get fresh versions of objects from the database? i.e. if the objects have been evicted shouldn't reinstantiating them result in a fresh pull from the DB or a fetch from the L2 cache - which should always have the latest version of any object? Perhaps I'm confusing the situation by holding references to some of the objects in UI objects of the framework I'm using whose lifecycle can span many HTTP requests (welcome to AJAX!). If I hold a reference to an object and I'm not using detach on commit and not doing any explicit commit (not serializing either) then I hold references to objects that are persistent non transactional (I guess). What happens when the next HTTP request arrives and the transaction is opened? Are the instances that the UI elements hold references to added to the cache during the process of handling that HTTP request or are new instances created and added to the cache? The scenario is a little bit more complex than this. You should consider chapter 5 for a full understanding. Just to simplify, upon evict() call, the instance becomes hollow (if not dirty - if you want to discard change you have to call refresh()) Upon transaction commit it becomes hollow if retainValues = false. Guido ---------------------------------------- Guido Anzuoni http://www.objectmagic.org |
||
|
|
ganzuoni
Expert Joined: Nov 16, 2005 Post Count: 1009 Status: Online |
If I hold a reference to an object and I'm not using detach on commit and not doing any explicit commit (not serializing either) then I hold references to objects that are persistent non transactional (I guess). What happens when the next HTTP request arrives and the transaction is opened? Are the instances that the UI elements hold references to added to the cache during the process of handling that HTTP request or are new instances created and added to the cache? I am not sure about your PM lifecycle across different HTTP requests. As from my previous post, retainValues flag and transaction type (datastore/optimistic) affect the state transition of instances. Since you say that some persistent objects are referenced by UI, I guess that you are using PM-per-session and txn-per-request. If you have retainValues=false, when you access field members of persistent objects, data is fetched from the DB since objects are hollow. If you have retainValues=true, since objects are hard-referenced, they are not GC (I guess that even on retainValues=true DN weakly references clean objects so that if memory gets low VM can GC these objects) and so no automatic refresh is performed. Guido ---------------------------------------- Guido Anzuoni http://www.objectmagic.org |
||
|
|
chrisco
Expert Australia Joined: Nov 24, 2005 Post Count: 706 Status: Offline |
I am not sure about your PM lifecycle across different HTTP requests. As from my previous post, retainValues flag and transaction type (datastore/optimistic) affect the state transition of instances. Since you say that some persistent objects are referenced by UI, I guess that you are using PM-per-session and txn-per-request. If you have retainValues=false, when you access field members of persistent objects, data is fetched from the DB since objects are hollow. If you have retainValues=true, since objects are hard-referenced, they are not GC (I guess that even on retainValues=true DN weakly references clean objects so that if memory gets low VM can GC these objects) and so no automatic refresh is performed. Hi Guido, thanks for the valuable info. I think my problem was the my setting of retainValues=true. My scenario is: Transaction type: optimistic PM Lifecyle: PM-per-session (I have always been concerned about the performance of PM per http request in a high activity AJAX application with so many http requests occurring) Transation lifecycle: each HTTP request auto wrapped inside a transaction by the dependency injection framework's (expojo) servlet filter. retainValues - I have always had this set to true for performance but now realizing that this is probably what's causing my need for explicit refreshes on individual objects - which takes lots of coding effort for large, complex, deep object graphs as it appears as though the refresh(o) works but the refreshAll, affecting only transactional objects, doesn't work as a lot of the objects that require refreshing aren't actually transactional. I've just set retainValues=false with no noticeable drop in performance and it does seem slightly better at maintaining data consistency when testing with mutliple users editing the same objects however it's still fairly easy reproduce a situation where I can make a change in one session and the other session maintains it's old value - even across multiple http requests (ie., multiple transactions - which is weird if non dirty objects are meant to be going hollow at the end of each transaction) ---------------------------------------- Chris Colman http://visualclassworks.com Javelin: Visual modeling & coding with automatic JDO 2/DataNucleus meta data generation. http://expojo.com expojo: Simple dependency injection and exposed POJO domain model pattern ---------------------------------------- [Edit 1 times, last edit by chrisco at Feb 26, 2009 10:55:38 PM] |
||
|
|
chrisco
Expert Australia Joined: Nov 24, 2005 Post Count: 706 Status: Offline |
Arh! Ok, from what you say about evict above and later about retainValues it appears that neither of these on their own are sufficient on their own to maintain data consistency in a concurrent scenario as per this part of the JDO spec: If evictAll with no parameters is called, then all persistent-clean instances are evicted (they transition to hollow). If users wish to automatically evict transactional instances at transaction commit time, then they should set RetainValues to false. Similarly, to automatically evict transactional instances at transaction rollback time, then they should set RestoreValues to false. I think the issue I am seeing is that the persistent-clean instances aren't being updated with changes made in a different PM. evictAll prior to starting to work with those objects will make sure I've got the latest version of them all - hopefully! Yes evictAll is certainly a lot more effective than refreshAll - which makes sense now that I understand how they work on completely different, but most likely complimentary, sets of objects. I think my new strategy should be: If I'm about to display a read only page of persistent objects or about to open a form to edit a persistent object (note that in both cases as no objects have yet been changed none will be transactional - all persistent clean!) then I should issue an evictAll to ensure that accessing these will bring in fresh data and therefore pick up any updates made by users in another session. I think my previous erroneous ways were caused by not realizing that refreshAll has absolutely no effect on persistent-clean instances i.e. the majority of instances in my cache! And also that retainValues=false also has no effect on persistent-clean instances at commit time. Arh, it's all so obvious now! =) ---------------------------------------- Chris Colman http://visualclassworks.com Javelin: Visual modeling & coding with automatic JDO 2/DataNucleus meta data generation. http://expojo.com expojo: Simple dependency injection and exposed POJO domain model pattern |
||
|
|
ganzuoni
Expert Joined: Nov 16, 2005 Post Count: 1009 Status: Online |
Hi Chris, let us know the results of your experiments. It would be nice if you could write some sort of best practice, lesson learned, or whatever in wiki if Andy agrees. BTW, can you share what AJAX toolkit are you using ? I have used zk in a project, managing lifecycle in ExecutionInit and ExecutionCleanup (PM-per-request) and it has been a nice experience. Guido ---------------------------------------- Guido Anzuoni http://www.objectmagic.org |
||
|
|
|
|
Current timezone is GMT May 17, 2012 12:18:08 PM |
