Building an LDAP address book for Thunderbird and Evolution

Overview

Households—and small companies—where there is more than one email user can derive considerable benefit from a shared address book.  Aside from proprietary systems such as Microsoft Exchange, implementation of an LDAP server is a well-recommended way to do it.

However, LDAP is “as user-friendly as a cornered rat”, and Thunderbird’s support of LDAP is more or less broken (and has been so for decades, without any obvious plan to fix it); together these make the whole process far harder than one would like.

It can be done, and almost all the benefits gained, but documentation on how to do it is sketchy and hard to find, and some of it is out of date with respect to current levels of software.  This document sets down my own route to success, in the hopes that others may find it helpful.

The detailed instructions assume a linux server (I’m using Debian 10); Windows users will, I hope, find that the outline of what to do is the same, just with different command syntax.

Henry Law, Manchester UK, November 2021

A note on LDAP itself

It’s important to note that LDAP is not a directory system.  It isn’t even a piece of software: it’s a description of the protocol that can be used to interact with a directory. So the word “LDAP” does the same job as “TCP/IP”; it describes something that has to be implemented and configured by software developers and users.

And like all protocols it has a very complex definition covering every eventuality and use case; it’s got lots of specific jargon which you have to learn, and configuring it isn’t immediately straightforward.

Fortunately you don’t need to know more than the bare bones of it to implement and configure an LDAP server for a well-understood requirement like a directory (or “address book” if you prefer). You’ll find what you need in the sections below.

If you do want to know all about LDAP, you can find it in a host of technical documents elsewhere on the web (and good luck).

A note on OpenLDAP

The commonest open-source LDAP server is OpenLDAP.  It is available on all the popular Linux distributions and also on Windows.  That’s what I used; if you want to use something else then a lot of the specifics of what follows may not be helpful.

Thunderbird and LDAP

If you read the program description you will find that Mozilla Thunderbird “supports” LDAP, and indeed it does.  You can use an LDAP directory as one of your address books, and get it to auto-complete when sending an email; it can also use LDIF (a text format used when loading information into an LDAP directory) for export and import.  But despite this apparent goodness the LDAP support in Thunderbird is very poor, and in some places broken.  More particularly:

• LDAP directories in Thunderbird are read-only; the user cannot directly create or change an entry in an LDAP directory.
• The LDIF that Thunderbird emits (via the “export” function) is not valid for input to an LDAP directory.  It needs to be reformatted before it can be used.
• Thunderbird does not support multiple email addresses in a single address book entry.  It does permit a second email address, using a Mozilla-specific field to do it, but this is special to Thunderbird; some other mail client using the same directory is unlikely to see that field.
• Because mailing lists (lists of people to whom a single email may be sent) cannot span different address books, a Thunderbird user cannot create a list which contains entries from a connected LDAP directory.  In fairness this is a standard limitation of Thunderbird, not a particular facet of its LDAP support

Placeholders in this document

The detailed instructions that follow do, of course, have to give values for various parameters which will not match your system.  Here are the values which you will need to change everywhere:

Your domain:            example.org
Your organisation name    myplace
Your master user        ldapuser
Master user’s password    ldaptest
Your address book name    abook

Install the code

sudo apt install slapd ldap-utils

For some reason, this doesn’t perform a configuration of LDAP; it just asks for an admin password and that’s it.  Re-configuring it asks you all the questions:

sudo dpkg-reconfigure slapd

Omit OpenLDAP config?: no (of course not!)
Domain name: example.org
Organisation name: myplace
Administrator password: (as before)
Backend: MDB
Remove when slapd is purged? No
Move old database? Yes

You can test your installation with this command:

ldapsearch -x -b dc=example,dc=org

The last lines of the output should be “search: 2 … result: 0 Success”.

Create ou for address book

Create defineAddressBook.ldif with the following contents

dn: ou=abook,dc=example,dc=org
ou: abook
objectClass: organizationalUnit

Add it to the directory with

ldapadd -vx -D "cn=admin,dc=example,dc=org" -H ldapi:/// -w ldaptest -f defineAddressBook.ldif

The configuration may be queried thus

sudo slapcat -b cn=config

Add Thunderbird mail schema

Download the Mozilla schema from the Mozilla web site.  and copy it to /etc/ldap/schema.

In a work directory, create slapd.conf containing the following lines:

include /etc/ldap/schema/core.schema
include /etc/ldap/schema/cosine.schema
include /etc/ldap/schema/nis.schema
include /etc/ldap/schema/inetorgperson.schema
include /etc/ldap/schema/mozillaAbPersonAlpha.schema

Change to the work directory and generate the LDIF files thus:

sudo slaptest -f slapd.conf -F $PWD

But the ldif file that is generated is not suitable for loading as a schema, for some unfathomable reason.  It needs to be modified; the following sed script mozillaAbPersonAlpha.sed should do it:

/dn:/ s|\{[0-9]\}||
/dn:/ s|$|,cn=schema,cn=config|
/cn:/ s|\{[0-9]\}mozillaabpersonalpha|mozillaabpersonalpha|
/structuralObjectClass/d
/entryUUID/d
/creatorsName/d
/createTimestamp/d
/entryCSN/d
/modifiersName/d
/modifyTimestamp/d

Modify the generated LDIF file by running the sed script (in the work directory).

sudo sed -i -rf mozillaAbPersonAlpha.sed './cn=config/cn=schema/cn={4}mozillaabpersonalpha.ldif'

Now you can add the schema

sudo ldapadd -Y EXTERNAL -H ldapi:/// -f './cn=config/cn=schema/cn={4}mozillaabpersonalpha.ldif' -w ldaptest

No, I don’t understand why we need to specify an SASL method (with -Y) here, but it didn’t work for me without.

Verify the addition with

sudo ldapsearch -Y EXTERNAL -H ldapi:/// -w ldaptest -b 'cn={4}mozillaabpersonalpha,cn=schema,cn=config'

Check that the mozillaabpersonalpha schema has been loaded.

Add Thunderbird address book

The Thunderbird address book must be exported from Thunderbird itself, and the resulting ldif file heavily reformatted.  See the appendix for this.  Then it can be loaded into the directory:

sudo ldapadd -vx -D "cn=admin,dc=example,dc=org" -H ldapi:/// -w ldaptest -c -f thunderbird.ldif

Use the -c option; otherwise the load will stop on any error, including finding a duplicate entry.

Configuring remote access for update

In order to use Evolution to create entries, and in order to use a remote LDAP management applications (I use JXplorer) you need a user which has the ability to create and delete address book entries; it’s bad practice to use the root user.  So create AddMasterUser.ldif with the following contents:

dn: cn=ldapuser,ou=abook,dc=example,dc=org
cn: ldapuser
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
objectclass: mozillaAbPersonAlpha
sn: ldapuser

Likewise, create AddWriteAccessForMasterUser.ldif thus:

dn: olcDatabase={1}mdb,cn=config
changetype: modify
add: olcAccess
olcAccess: {0}to *
  by dn="cn=ldapuser,ou=abook,dc=example,dc=org" write
  by self write
  by * read

Use these two ldif files to add the master user and configure read-write access:

sudo ldapadd -vx -D "cn=admin,dc=example,dc=org" -H ldapi:/// -w ldaptest -c -f AddMasterUser.ldif
sudo ldapmodify -Y EXTERNAL -D "cn=admin,dc=example,dc=org" -H ldapi:/// -f AddWriteAccessForMasterUser.ldif

Finally, set the master user’s password like this:

sudo ldappasswd -D "cn=admin,dc=example,dc=org" -H ldapi:/// -w ldaptest -S "cn=ldapuser,ou=abook,dc=example,dc=org"

Testing the LDAP server

You should now be able to see the LDAP address book from Thunderbird by configuring it appropriately.  There are lots of web pages which will tell you how to do this.  

Once configured, Thunderbird should use the LDAP directory for auto-completion, and you should be able to search it.

Configuring Evolution is similar, but if you use the “ldapuser” account and supply its password you will be able to create entries also.

Appendix: Editing the Thunderbird address book file

Thunderbird provides a means of exporting an address book to an LDIF file, but the data in the LDIF is not suitable for loading into an LDAP database with the mozilla schema. Deficiencies include, for example:

• It doesn’t have a version line.
• The attribute mozillaUseHtmlMail has numeric values (corresponding, presumably, to the three possible values in the Tbird address book, that is “No”, “Yes” and “Don’t care”), whereas Mozilla’s own address book schema requires TRUE/FALSE. You couldn't make this stuff up, really.
  

In order to reformat the exported ldif file I hacked up a utility program abookrft, which will do a lot of the heavy lifting for you.  It's not marvellous Perl, in case you worry about that.

Installing abookrft

Download the code from here.  By way of pre-requisite modules, you will need to install Email::Address; everything else is in the standard list.

You can run abookrft from any directory.  For help, enter ./abookrft --help

Features of abookrft

Using a Thunderbird LDIF export as its source, abookrft will:

• Extract a value for cn from the dn line if it’s not supplied elsewhere.  If nothing else, the cn will be set to an email address.
• Detect a cn in the form “Blatz, Joe” and reformat it to “Joe Blatz”.
• Strip out unwanted stuff from most of the fields, especially quote characters, backslash escapes and leading/trailing spaces.
• Fix the value of mozillaUseHTMLMail if it’s present.
• Removes the modifytimestamp attribute which, being read-only, cannot be loaded.
  
• Reformat member attributes in mailing lists to use the appropriate dn’s; it also checks whether the members in a list exist elsewhere in the address book.
• Handles multiple entries for a single individual containing different email addresses, in the following slightly convoluted fashion in order to satisfy Thunderbird’s lame email handling as much as possible.
• Every unique entry results in an LDAP entry.  Numbers in square brackets are appended to the cn to distinguish multiple entries.
• The first entry found—the one with no number appended—is treated as the “master” entry.
• The last primary email found in multiple entries for the same person is established as the primary email for the master entry, and all the others are also added as separate mail: lines.
  
• The “secondary email” attribute for the last entry found is also placed in the mozillaSecondEmail attribute of the master entry.  Thunderbird will not see the multiple mail: lines, but it will at least see the "second email" field.
  
• Thus the first (“master”) entry contains all the mail addresses for that person which were found; the other—numbered—entries each contain just the mail address which appeared in their particular LDIF stanza.
• Rejects duplicate entries within a single LDIF file—those that are the same in every respect.  It does not query the LDAP directory to detect duplicates.
• Establish surname (sn) and given name (givenName) from the cn wherever possible.

Appendix: converting a .mab file to ldif for loading

Later note: I don't use Thunderbird any more (migrated to Evolution) so this change passed me by.  But I now know that recent versions of Thunderbird don't store the address book information in abook.mab but in an SQLite database, the structure of which is opaque to me. So what follows pertains only to Thunderbird releases before 78.

If you can’t obtain an exported LDIF version of the Thunderbird address book, you can use the original Thunderbird address book, usually called abook.mab.  As a companion to the ldif-reformatting program referred to above, you may download my even less elegant Perl program which will read the mab file and produce LDIF.

To run it, enter ./thunderbird-ldif abook.mab

Appendix: Configuring Evolution mail for LDAP access

Follow this tutorial, with the following additions and changes.

Encryption: none

Authentication method: Using distinguished name (DN)

Username: cn=ldapuser,ou=abook,dc=example,dc=org

Search base: ou=book,dc=example,dc=org

Evolution is able to add entries to the LDAP address book, which Thunderbird is not.  If you are a Thunderbird user you will have to use something else to add your entries: JXplorer works well for me.