Issue Details (XML | Word | Printable)

Key: NUCLDAP-12
Type: New Feature New Feature
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Stefan Seelmann
Reporter: Stefan Seelmann
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
DataNucleus Store LDAP

Implement native queries using LDAP filters

Created: 29/Oct/08 06:41 PM   Updated: 28/Dec/09 10:58 AM   Resolved: 07/Dec/08 01:31 PM
Component/s: Queries
Affects Version/s: 1.0.2
Fix Version/s: 1.0.3, 1.1.0.m1

File Attachments: 1. Text File NUCLDAP-12-3.patch (7 kB)
2. Text File NUCLDAP-12-4.patch (11 kB)
3. Text File NUCLDAP-12-local-repository.patch (6 kB)
4. Text File NUCLDAP-12-store-ldap.patch (41 kB)
5. Text File NUCLDAP-12-test-jdo-ldap.patch (20 kB)


Datastore: LDAP


 Description  « Hide
Currenty when quering objects by JDOQL Datanucleus only uses a basic filter based on objectClass. For large directories this means that all entries are fetched. Later the In-Memory evaluator filters all not matching objects. This leads to very poor performance

I'd like to implement a native query using LDAP filters, it is already on the planned feature list here: http://www.jpox.org/servlet/wiki/display/ENG/LDAP

I started to implement a class by extending AbstractExpressionEvaluator to see if it works and it seems to work for equal and notequal filters. Before I continue I just wanted to know if this is the right way to implemement this?

Another question: I'd like to reuse some classes from Apache Directory Shared project. It includes some nice classes to construct LDAP filters. It is necessary to add the following dependencies to the store.ldap project:
- shared-asn1-0.9.13.jar
- shared-ldap-0.9.13.jar
- shared-ldap-constants-0.9.13.jar
- slf4j-api-1.5.2.jar
- slf4j-log4j12-1.5.2.jar
I just want to ask if it is OK to add those dependencies? I guess it sholdn't be a problem because on the wiki page you already mention to use the LdapDN class :-)


Sort Order: Ascending order - Click to sort in descending order
Andy Jefferson added a comment - 29/Oct/08 07:05 PM
Hi Stefan,
having an option to have queries evaluated in LDAP obviously makes sense. Comments :-
1. Look at "store.db4o" src/java/org/datanucleus/store/db4o/query/JDOQLQuery.java for "datanucleus.query.evaluateInMemory". This is used to allow users to evaluate a query in memory (like now), or natively (what you're adding). Add something like that as the way of deciding if they use the current process or yours.

2. slf4j logging : why ? No logging from any of DN should use that, instead NucleusLogger. Obviously Apache Directory code can call slf4j if it wishes, but are these deps you're talking about compile time or runtime ?

3. shared-XXX deps : no problem with those but they should be "optional" (MANIFEST.MF, and pom.xml) so people who just use the current code don't need to include them. They'd also need to go in local.repository.

4. Using AbstractExpressionEvaluator for the process - yes, same basic idea as we had with db4o/NeoDatis.


If you are going to be working an amount of time on store.ldap then I can certainly give you (back) commit rights, just that I've spent 5+ yrs giving ppl rights and then after a couple of commits they aren't seen again. The project needs your time.

Stefan Seelmann added a comment - 29/Oct/08 08:17 PM
Hi Andy,

1. I'll take a look to the db4o implementation, thanks for the hint.

2. Yes. Apache Directory Shared depends on it, and it is a runtime dependency. We use slf4j as wrapper around log4j.

Hey, I fully agree with your policy. I was busy the last months, now I will take some time. I'm interested on Object-LDAP-Mapping, and I think it is a good approach to use an existing standard (JDO) and implementation. But maybe I'll disappear again ;-)

Stefan Seelmann added a comment - 09/Nov/08 09:12 PM
Patch for store.ldap, relative to platform/store.ldap/trunk

Stefan Seelmann added a comment - 09/Nov/08 09:13 PM
Patch for test.jdo.ldap, relative to test/accessplatform/trunk/test.jdo.ldap

Stefan Seelmann added a comment - 09/Nov/08 09:14 PM
Patch for local.repository, relative to platform/local.repository/trunk


Erik Bengtson added a comment - 09/Nov/08 10:23 PM
patches committed with minor changes due to remove binary linkage to the native query

Erik Bengtson added a comment - 09/Nov/08 10:39 PM
to run the tests with native ldap queries

mvn clean install -P ldap-native

otherwise use

mvn clean install -P ldap

Stefan Seelmann added a comment - 10/Nov/08 12:17 PM
Patch with minor changes:
* Native query wasn't working, due to missing slf4j dependencies
* Added log message when native query fails
* Removed some System.out.prinln()

Andy Jefferson added a comment - 10/Nov/08 04:16 PM
I can no longer build using Eclipse.
It tries to bring in
/usr/local/eclipse-3.4/plugins/org.apache.directory.studio.jars_1.1.0.v20080331/lib/shared-ldap-0.9.8.jar
(for some Eclipse reason).

This is brought in via Directory Studio. How do I build in Eclipse now ?

Stefan Seelmann added a comment - 10/Nov/08 04:30 PM
You could remove the Directroy Studio plugins in Preferences->Plug-in Development->Target Platform.

I'll investigate if we could reduce the visiblity of the exported packaged in the Directory Studio jars.

Andy Jefferson added a comment - 10/Nov/08 06:41 PM
Thx Stefan, Eclipse builds fine now.
Only comment on the QueryToLDAPFilterMapper : there's a check for method "contains" and a comment that refers to "String" type. The "contains" method is actually for Collection fields as per
http://www.datanucleus.org/products/accessplatform_1_1/jdo/jdoql_methods.html
and as per the JDK.

Stefan Seelmann added a comment - 10/Nov/08 07:14 PM
Oh, ok. I thought that this method means the String.contains() method, then I implemented a test and noticed that it doesn't work, however I forgot to remove the "contains" method.

Stefan Seelmann added a comment - 10/Nov/08 11:33 PM
Patch NUCLDAP-12-3.patch includes:
* Native query wasn't working, due to missing slf4j dependencies
* Added log message when native query fails
* Removed some System.out.prinln()
* Removed "contains" method in QueryToLDAPFilterMapper.processInvokeExpression()

Andy Jefferson added a comment - 11/Nov/08 09:02 AM
Patch 12-3 applied with the exception of adding slf4j to pom.xml of "store.ldap" - SLF4J is not a DataNucleus code requirement so we don't list it; if Apache Directory needs it to run then it goes in its dependencies.

Stefan Seelmann added a comment - 12/Nov/08 02:14 AM
Patch NUCLDAP-12-4.patch includes:
* Added support for ParameterExpressions

Note: the changes in this patch depends on new methods in org.datanucleus.query.QueryUtils, these new methods are included in NUCCORE-149.

Andy Jefferson added a comment - 04/Dec/08 05:16 PM
Stefan, is this issue complete ? I'm going to do 1.1 M3/1.0.3 releases next week (week starting 4th Dec)

Stefan Seelmann added a comment - 05/Dec/08 09:18 AM
I'll review the code and tests this weekend and update documentation.

Stefan Seelmann added a comment - 06/Dec/08 09:21 PM
Is it possible to add the dependencies to use native LDAP queries to the datanucleus-accessplatform-ldap and datanucleus-accessplatform-full-withdeps distributions? These dependencies are
- shared-asn1-0.9.13.jar
- shared-ldap-0.9.13.jar
- shared-ldap-constants-0.9.13.jar
- slf4j-api-1.3.1.jar
- slf4j-log4j12-1.3.1.jar

I added these dependencies to the documentation: http://datanucleus.svn.sourceforge.net/viewvc/datanucleus?view=rev&revision=4213

Andy Jefferson added a comment - 07/Dec/08 09:34 AM
accessplatform zips include LDAP deps now