ApacheDS LDAP authentication, stay away

While developing a relatively large application, we required a single sign-on authentication. After a brief investigation about what open source solutions there are we decided that we go for the ApacheDS LDAP authentication. I don’t usually do this, I’m not hater, but take this warning from me: stay away from ApacheDS!

ApacheDS LDAP

What is ApacheDS? It is an open source directory server developed under the Apache Foundation. To be honest, being mainly a Java programmer I have to say that I’m very happy with all most of the solutions found under Apache.

I started using ApacheDS around a year ago, but little did I suspect what was going to happen.

The setup

We deployed version M11 of ApacheDS for one big client we had (around 15,000 users). This was part of a bigger project in which the LDAP integration was about 10%.

We had a portal and we used ApacheDS LDAP authentication in front of that. Since the installation out of the box was pretty simple for ApacheDS, we just deployed it on the server and did not give it a second thought.

First failures

Initial tests for importing user information went smooth, however we only tested batches of 10-20 users at a time. We then moved to importing larger batches, and we did notice that after about 500 users we started getting errors upon trying to add more. Well, restarting ApacheDS seemed to be the solution, so we did that a couple of times. However, after a couple restarts we got to about 3-4000 users imported it stopped working even after restart.

Even more, some of the data already in the system seemed to be corrupted. The corruption of data made it such that you could not find a user by searching for it, you could not delete it, and even more frustrating, while trying to add the same user you got a duplicate entry error.

We scratched our heads, and started again from 0. In the mean time version M12 came out, so we decided to go for that. We also found documentation indicating that there are some cache settings that need to be incremented, so we did that. Things seemed to be OK.

More failures

Since we were pushing towards the deadline for our project and things seemed to have been resolved, we went ahead and imported all 15000 users, with “only” one restart required.

A month or two into using the application, in which aprox. 50-60 insert/update operations have been done in the ApacheDS LDAP directory, and we had data corruption again, with all sorts of misleading error messages. We had no choice but to restore from our backups. We also updated to the latest version which at the time was M17, and hoped for the best.

Final blow

Just a few more months later into the use, with no significant addition of user entries, we did not encounter any more problems. However, when our client acquired a new company and it was necessary to input new employees as users in the system all sort of bad things started to happen. Once every 50-60 user entries the system stopped accepting any more entries, and the last entry put in became corrupted. The error that we got looked something like:

javax.naming.NamingException: [LDAP: error code 80 - OTHER: failed for MessageType : ADD_REQUEST
Message ID : 3
    Add Request :
Entry
    dn[n]: [email protected],ou=users,ou=portal,o=mycompany
    objectClass: organizationalPerson
    objectClass: person
    objectClass: inetOrgPerson
    objectClass: portalUser
    objectClass: top
    userPassword: 0x7B 0x23 0x48 0x41 0x7D 0x19 0x76 0x76 0x53 0x51 0x35 0x4E 0x54 0x57 0x4B 0x75 ...
    sn: [email protected]
    cn: [email protected]
    email: [email protected]
    lastName: Johnson
    portalInstance: apps.domain.com=63414
    defaultPortalInstance: subdomain.domain.com=63408
    firstName: John
    ManageDsaITImpl Control
        Type OID    : '2.16.840.1.113730.3.4.2'
        Criticality : 'false'
'
: java.lang.ArrayIndexOutOfBoundsException: -19960:
org.apache.directory.api.ldap.model.exception.LdapException: java.lang.ArrayIndexOutOfBoundsException: -19960
        at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.add(AbstractBTreePartition.java:862)
        at org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.add(DefaultPartitionNexus.java:352)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor$1.add(BaseInterceptor.java:165)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:422)

Well, that was that, just a few kicks in the back later from the boss and the client we knew we have to make a decision, whether to keep using ApacheDS with all this pain in the neck or to choose a different LDAP solution.

The decision

First we did a quick upgrade to the latest version (again) which in only one year it came from version M11 to M20. We still got the same problem, so we decided to contact the ApacheDS support about this. Their answer was SHOCKING… read below my message and their answer:

> Hi,

Hi,
>
> we have been using ApacheDS 2.0.0-M17 for about a year and from time to
> time the data gets corrupted leaving us no other choice but to create a
> fresh installation, export then import what data we can. This is very
> frustrating and also dangerous for us to deploy in a production environment.
>
> What happens is that after a while of usage adding an entry (a user) gives
> an error, and any additions after that fail. All entries for which the
> error occurs cannot be found anymore (using ApacheDS Studio or via Java),
> when trying to re-add them we get an error saying that an entry with the
> same cn already exists, and we cannot delete the entry since we can't find
> it.
>
> Please help, what are we doing wrong? Do we have to set some settings?
> We've had similar problems with version since M11, and upgraded hoping this
> will not occur anymore.

You are not doing anything wrong... Sadly :/

There is a known problem in the JDBM backend we are using, and that
leads to data corruption from time to time. We are working on replacing
this backend by Mavibot, but this is far from being simple - beside teh
fact that we are lacking of time to work on it -.

As of today, we have something working, but we are still lacking the
cross-btree transaction taht is critical from a performance POV. This is
what we are working on.

I can't give you any expectation for a release that use Mavibot, that
could be a few weeks, or a few monts (depends on day job/familly life
and the number of hours we can squeeze out of insomnia...)

WOW! What?!!? So, this means that this solution actually doesn’t work because the back-end  is funky and changing it is really hard. Also, no people on it…

I know it’s an open source solution, and cannot ask the moon from it, but the obvious fact is that ApacheDS DOES NOT WORK!

Just if you are wondering, we did switch to OpenLDAP, which was very straight forward to install on Ubuntu, but on Centos it was NOT (took us about a day and a half). So far, no problems.