Skip to content

ews: fix FindPeople recipient autocomplete (Outlook) - #270

Open
nourdineb-ops wants to merge 6 commits into
grommunio:masterfrom
nourdineb-ops:findpeople-fix-pr
Open

ews: fix FindPeople recipient autocomplete (Outlook)#270
nourdineb-ops wants to merge 6 commits into
grommunio:masterfrom
nourdineb-ops:findpeople-fix-pr

Conversation

@nourdineb-ops

Copy link
Copy Markdown
Contributor

Summary

FindPeople is what Outlook (Mac, and other EWS clients) call live on every keystroke to autocomplete a recipient in the To/Cc/Bcc fields. On this deployment it never worked - every query returned no usable results, so the client always showed "person not found" even for addresses that definitely exist.

Two commits, found and fixed in that order (the first alone was not enough).

ews: fix FindPeople returning personas with no usable email/type

  • FindPeople/GetPersona read a persona'''s email via fetch_prop(PR_SMTP_ADDRESS), which is never populated in ab_tree'''s generic propvals map - the match check silently never succeeded for anyone. Switched to the dedicated user_info(ab_tree::userinfo::mail_address) accessor, which ResolveNames already uses correctly for the same lookup.
  • Added the missing PersonaType field - Outlook Mac silently discards a persona even with a valid email if this is absent.

ews: fix FindPeople response structure and add recipient-history search

Even with the above fixed, the client still reported no results for every query. Root cause: FindPeopleResponse was wrapped in the ResponseMessages/FindPeopleResponseMessage batch envelope used by most other EWS operations, but FindPeople is not a batch operation - a response captured from a genuine Exchange server shows ResponseClass sits directly on the FindPeopleResponse root element, with People/TotalNumberOfPeopleInView/FirstMatchingRowIndex/FirstLoadedRowIndex as direct children, no wrapper. The client was silently discarding every response regardless of content correctness because of this extra nesting level.

Confirmed by MITM-injecting a real Exchange response body in place of gromox'''s own for one query - the client accepted it immediately, isolating the bug to response structure rather than content. Fixed by making mFindPeopleResponse inherit mResponseMessageType directly (dropping the separate mFindPeopleResponseMessage type), matching the flat pattern already used correctly by mGetPersonaResponseMessage/mGetAppManifestsResponse elsewhere in this codebase.

Also bundled into this commit:

  • PersonaId (new tPersonaId type), GivenName/Surname, RelevanceScore, and a proper nested EmailAddress (Name/EmailAddress/RoutingType/MailboxType) on tPersona - the client silently drops personas missing these.
  • A "Mailbox" query-source search over grommunio-web'''s PR_EC_RECIPIENT_HISTORY_JSON, so autocomplete also surfaces people actually emailed before, not just directory/GAL entries - reusing the same mechanism the webapp itself already relies on for its own (working) autocomplete.
  • A real bug fix: msg.People.emplace() was called on every loop iteration in both the ab_tree match loop and the history-merge step; optional<vector>::emplace() destroys/reconstructs the vector each call, so only the last ab_tree match ever survived a response with multiple matches.

Testing

Verified live against a real Outlook for Mac client (16.110.26070318): typing a partial name now correctly surfaces both directory and previously-emailed contacts in the autocomplete dropdown. Running in production since 2026-07-12.

Two bugs found while investigating Outlook Mac's recipient-autocomplete
"person not found" ("désolé, nous n'avons pas trouvé la personne"):

- FindPeople used node.fetch_prop(PR_SMTP_ADDRESS, val) to get a
  persona's email, but PR_SMTP_ADDRESS is never present in the ab_tree
  node's generic propvals map. The sibling ResolveNames handler already
  gets this right via the dedicated node.user_info(mail_address)
  accessor - use the same one here. Confirmed live: responses went from
  DisplayName-only to DisplayName+EmailAddress.

- tPersona never modeled PersonaType at all, so every FindPeople
  response omitted <t:PersonaType>, which Outlook likely needs to
  classify the suggestion. Added the field (structures.hpp), its
  serialization (XMLDUMPT in serialization.cpp), and set it to "Person"
  for every ab_tree match in the FindPeople handler.

Both verified live via mitm capture. Outlook Mac still doesn't surface
the suggestion after these two fixes - next suspect is PersonaId, which
doesn't exist anywhere in exch/ews/ (unlike tItemId/tFolderId) and may
be required for Outlook to actually act on a persona result; not yet
implemented.

@jengelh jengelh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, on the way to finding that:

Quoting from an older version of the QEMU wiki:

"Also, ..." is often a good candidate for splitting into multiple patches.

If possible, please try this.

optional<vector>::emplace() destroys and reconstructs the vector on
every call, so calling msg.People.emplace().emplace_back(persona) inside
the ab_tree match loop wiped out any previously collected persona each
time - only the last match in a multi-result response ever survived to
be returned to the client.

Fixed by emplacing the vector once (if not already present) and pushing
into it thereafter.
Same root cause as the FindPeople fix two commits back:
PR_SMTP_ADDRESS is never populated in ab_tree's generic propvals map,
so GetPersona's email match check (fetch_prop(PR_SMTP_ADDRESS, val))
never succeeded for anyone and the request always reported "not
found", even for a valid address.

Switched to the dedicated user_info(ab_tree::userinfo::mail_address)
accessor, same as ResolveNames and the already-fixed FindPeople path
use.
…ailAddress to tPersona

A real Exchange FindPeople response capture showed Outlook Mac silently
drops a persona missing any of these:

- PersonaId - not a real MAPI EntryID, just a stable opaque handle
  (base64 of the SMTP address) that lets Outlook accept the entry as
  resolvable/insertable.
- GivenName/Surname - Outlook Mac's FindPeople request explicitly asks
  for both via AdditionalProperties.
- A nested EmailAddress (Name/EmailAddress/RoutingType/MailboxType)
  instead of a flat string - captured Exchange responses always use
  this Mailbox-shaped type.
- RelevanceScore.

Wired the new fields into both existing ab_tree persona builders
(FindPeople and GetPersona).
FindPeople was wrapping its response in the ResponseMessages/
FindPeopleResponseMessage batch envelope used by most other EWS
operations, but a real Exchange capture showed FindPeopleResponse is
not a batch operation: ResponseClass sits directly on the
FindPeopleResponse root element, with People/TotalNumberOfPeopleInView/
FirstMatchingRowIndex/FirstLoadedRowIndex as direct children. Outlook
Mac silently discarded every response regardless of content
correctness because of this extra nesting level - confirmed by
injecting the real Exchange response body via mitmproxy (worked
immediately) versus our own (never worked, including after fixing
every other field-level gap).

Also added FirstMatchingRowIndex/FirstLoadedRowIndex: a real Exchange
capture always includes both alongside TotalNumberOfPeopleInView, even
for a single-page result. Real paging isn't implemented, so both are
always the start of (and only) page.
…ople matches

FindPeople's QuerySources typically ask for both "Directory" (the
ab_tree GAL) and "Mailbox" - people the user has actually
corresponded with, regardless of whether they're in any directory.
gromox itself doesn't track this for EWS, but grommunio-web already
does: it maintains PR_EC_RECIPIENT_HISTORY_JSON (named property
"websettings_recipienthistory" under PSETID_Gromox on the store) as
a JSON blob of {display_name, smtp_address, count, last_used} updated
whenever the user sends mail via the webapp.

Added findpeople_search_recipient_history(), reusing that same store
for Outlook's autocomplete - merged into FindPeople's ab_tree results
and deduped by SMTP address.
@nourdineb-ops

nourdineb-ops commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Good call, done - split the second commit into five:

  • ews: fix FindPeople only returning the last ab_tree match (the pre-existing optional<vector>::emplace() bug)
  • ews: fix GetPersona returning personas with no usable email (same PR_SMTP_ADDRESS issue as the first commit, but for GetPersona)
  • ews: add PersonaId/GivenName/Surname/RelevanceScore and structured EmailAddress to tPersona (the new fields Outlook needs to accept a persona)
  • ews: un-batch-wrap FindPeopleResponse to match real Exchange shape (the actual headline fix)
  • ews: search recipient history (grommunio-web mailbox JSON) for FindPeople matches (the new "Mailbox" query source)

Each builds and links (libgromox_ews.la) on its own, and the resulting tree is identical to what was there before - just re-cut into smaller pieces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants