Server-side address autocompletion

[thfi]

In this projects README, you write as planned feature:

Server-side address autocompletion (will need to sort out some KDE PIM limitations)

What are those limitations and is someone working on those? Can you please provide some details? What help to do you require?

[KrissN]

Unless something has changed recently, KMail find its address auto-completion candidates from three sources:

1. List of seen addresses
2. Akonadi search for contact folders (offline)
3. LDAP

The only path to Akonadi is 2, however as stated above the search is offline-only (most likely due to performance concerns), which means that only contacts downloaded from the server will be taken into account.

On the Exchange server there are two sources of contact information:

1. Contacts folders (usually the Personal Address Book folder, but you're free to create more). These only contain contacts which were explicitly added.
2. The GAL (Global Address List) - this usually contains all contacts in the corporate database taken from Active Directory.

In order to use Exchange contacts in KMail in its current form, the EWS resource would need to download them. This is quite simple for the contact folders (and is already being done), but as mentioned before these folders will usually only contain some auxiliary contacts that the user has explicitly added (for ex. contacts from outside of the company).

What is of most day-to-day use is the GAL. Downloading it comes with two issues:

1. Exchange versions up to 2010 SP1 IIRC didn't have EWS API to fully download the GAL. You could only perform search bases on substrings (the API was designed for autocompletion in mind).
2. Even if 1. is solved (either some later Exchange or use of OAB (Offline Address Book)), the full contact list for large corporations can have even half a million entries or more (all employees, virtual users, mailable groups). Imagine pulling all that into Akonadi database - it would take hours to download it (provided the server doesn't give up at some stage), gigabytes to store it, not to mention making sure it's in sync.

In order to support auto-completion in KMail there needs to be some API added to support online search for resources that are able to do it in a performance-acceptable manner. My current idea is to build a new abstract interface for online auto-completion, port the existing LDAP engine to it and use it in Akonadi EWS.

There is in theory a dirty workaround possible - to build a fake LDAP server that only supports enough of LDAP to work with the existing LDAP auto-complete engine, but I'd like to avoid that if possible.

[thfi]

Thanks for the detailed feedback!

There is in theory a dirty workaround possible - to build a fake LDAP server that only supports enough of LDAP to work with the existing LDAP auto-complete engine, but I'd like to avoid that if possible.

That is essentially DavMail's solution, which did not work reliably for me, maybe both due to KMail's (aging/incomplete) LDAP support and DavMail limitations.

Akonadi search for contact folders (offline)

That is quite a surprising limitation, given that Akonadi was sold as the grand unifying answer to all our PIM needs (warning: irony or sarcasm). I mean, any cloud-based or CardDav-based system faces the same problem.

In order to support auto-completion in KMail there needs to be some API added to support online search for resources that are able to do it in a performance-acceptable manner.

I recall reading something on Planet KDE about a refactoring/replacement of KRunner to allow for more efficient/non-blocking online searches. Unfortunately, I cannot find the posting right now, but I guess if it the new code/refactoring was successful, it found its way into the KRunner repository. Maybe you'll find some inspiration for your code there.

My current idea is to build a new abstract interface for online auto-completion, port the existing LDAP engine to it and use it in Akonadi EWS.

As an intermediate step, may I suggest that you provide a simple stand-alone application (maybe even without going through Akonadi initially) that allows online queries on the GAL. This way, interested users like myself can test your Exchange-facing code in their Exchange setup. Later the matured code can be ported into Akonadi directly or Akonadi-EWS.

[newsages]

you can see how evolution-ews make,
https://github.com/GNOME/evolution-ews/blob/master/src/addressbook/oab-decode-test.c
ews_oab_decoder