From 9ddb9b08e149c7b0accef6e1227590e4cdf04657 Mon Sep 17 00:00:00 2001 From: phg Date: Thu, 23 Jul 2026 00:19:57 +0200 Subject: [PATCH] docs: define RFC recipient parsing boundaries --- docs/research/rfc-recipient-parsing.md | 105 +++++++++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 docs/research/rfc-recipient-parsing.md diff --git a/docs/research/rfc-recipient-parsing.md b/docs/research/rfc-recipient-parsing.md new file mode 100644 index 0000000..1fb71a3 --- /dev/null +++ b/docs/research/rfc-recipient-parsing.md @@ -0,0 +1,105 @@ +# RFC recipient parsing boundaries + +Research for [Establish RFC recipient parsing boundaries](https://git.s1q.dev/phg/fuse-mail-dir-by-tag/issues/4). + +## Decision + +Parse only the RFC message header section, preserve every occurrence of the four configured fields, and derive plus tags only from syntactically valid mailbox local-parts. Do not scan header text with regular expressions and do not salvage address-looking substrings from a field that failed complete syntactic validation. RFC 5322 makes the header section a sequence of fields terminated by the empty line, defines unfolding before semantic analysis, and distinguishes structured address fields from unstructured text ([RFC 5322 §§2.1, 2.2.2–2.2.3](https://datatracker.ietf.org/doc/html/rfc5322#section-2.1)). + +Apply the already-agreed precedence as a two-tier algorithm: + +1. Parse every `Delivered-To` and `X-Original-To` occurrence and collect the plus tags from all valid occurrences. +2. If that tag set is non-empty, return it and do not inspect `To` or `Cc` for tags. +3. Otherwise, parse every `To` and `Cc` occurrence, flatten all mailbox members of all groups, and collect their plus tags. +4. Deduplicate the selected tier's tags by exact equality of the semantic local-part suffix, preserving first-seen order internally for deterministic diagnostics and tests. + +The fallback condition is therefore **no valid plus tag from the delivery tier**, not “delivery headers are absent” and not “delivery headers contain no valid recipient.” This makes an untagged or malformed delivery field fall back to visible recipients, while any valid delivery plus tag suppresses all visible-recipient tags. + +## Parsing contract + +### Header framing and occurrences + +- Field names are matched case-insensitively, and a folded field is unfolded by removing each CRLF immediately followed by whitespace before its structured value is parsed ([RFC 5322 §§2.2–2.2.3](https://datatracker.ietf.org/doc/html/rfc5322#section-2.2.3)). Folding is syntax, not a new field, so continuation lines must remain attached to the preceding occurrence. +- Keep occurrences separate and retain document order. RFC 9228 intentionally permits a sequence of `Delivered-To` fields, one per delivery or address transformation, and says their relative order must not be changed ([RFC 9228 §4](https://www.rfc-editor.org/rfc/rfc9228.html#section-4)). RFC 5322 normally limits `To` and `Cc` to one occurrence each, but its mandatory obsolete-input grammar permits repeated destination fields and says to interpret them as concatenated address lists ([RFC 5322 §§3.6, 4.5.3](https://datatracker.ietf.org/doc/html/rfc5322#section-4.5.3)). +- Stop at the first empty line; never interpret address-like content in the message body. A header-only parser is sufficient. For example, `mail-parser` exposes `parse_headers`, which returns a message containing only headers, and `header_values`, which iterates every matching occurrence ([mail-parser 0.11.5 `MessageParser`](https://docs.rs/mail-parser/0.11.5/mail_parser/struct.MessageParser.html#method.parse_headers), [`Message::header_values`](https://docs.rs/mail-parser/0.11.5/mail_parser/struct.Message.html#method.header_values)). + +### Delivery tier + +- A `Delivered-To` occurrence is valid only when its complete unfolded field body, apart from the grammar's permitted FWS, is exactly one RFC 5322 `addr-spec`. The standardized field records one individual address and its grammar is `FWS addr-spec FWS`; a display name, angle-address, comma-list, or group is not valid in this field ([RFC 9228 §4](https://www.rfc-editor.org/rfc/rfc9228.html#section-4), [RFC 5322 §3.4.1](https://datatracker.ietf.org/doc/html/rfc5322#section-3.4.1)). +- Treat each `X-Original-To` occurrence with the same one-complete-`addr-spec` rule. This is a project interoperability rule grounded in Postfix's contract: when original-recipient support is enabled, Postfix stores the original recipient address in this field, and its local delivery agent prepends one `X-Original-To` containing the recipient address given to Postfix ([Postfix `enable_original_recipient`](https://www.postfix.org/postconf.5.html#enable_original_recipient), [Postfix `local(8)`](https://www.postfix.org/local.8.html)). +- Parse all occurrences from both names; neither name outranks the other. Multiple occurrences may produce multiple plus tags. A malformed occurrence contributes nothing but does not invalidate valid sibling occurrences. + +### Visible-recipient tier + +- Parse each complete `To` and `Cc` occurrence as RFC 5322 `address-list`, including the obsolete forms receivers are required to understand. An address is either a mailbox or a group; a group contains zero or more mailboxes. Extract only actual mailbox `addr-spec` nodes and ignore display names, group names, comments, and empty groups ([RFC 5322 §§3.4, 3.6.3, 4](https://datatracker.ietf.org/doc/html/rfc5322#section-3.4)). +- Comments may nest and may contain escaped parentheses; folding may occur inside them. They are semantically separate from address data, while quoted pairs represent the escaped character without the backslash. These rules require a stateful grammar parser, not delimiter splitting ([RFC 5322 §§3.2.1–3.2.2](https://datatracker.ietf.org/doc/html/rfc5322#section-3.2.2)). +- If one `To` or `Cc` occurrence does not parse completely as an address list, reject that whole occurrence and continue with other occurrences. This is the project's deterministic safety boundary: RFC 5322 requires parsers to understand its obsolete grammar and to handle still-malformed data without crashing, but does not require trusting partially recovered addresses ([RFC 5322 §4](https://datatracker.ietf.org/doc/html/rfc5322#section-4)). + +### Semantic local-part and plus tag + +For each accepted `addr-spec`, obtain the parser's **semantic local-part**, not a substring found by searching the serialized address. RFC 5322 defines `addr-spec = local-part "@" domain` and permits local-parts in dot-atom, quoted-string, and obsolete forms; an `@` can therefore occur as data inside a quoted local-part ([RFC 5322 §3.4.1](https://datatracker.ietf.org/doc/html/rfc5322#section-3.4.1)). + +Canonicalize syntax only: + +- remove delimiting quotes; +- resolve quoted pairs by removing the quoting backslash while retaining the quoted character; +- remove comments and folding whitespace only where the grammar marks them as semantically invisible; +- do **not** case-fold, Unicode-normalize, decode RFC 2047 encoded-words inside an `addr-spec`, or reinterpret the domain. + +SMTP requires quoted forms of a local-part to compare equivalently, while preserving local-part case because it can be significant ([RFC 5321 §§2.3.11, 4.1.2](https://datatracker.ietf.org/doc/html/rfc5321#section-4.1.2)). RFC 2047 encoded-words are allowed in phrases and comments but explicitly forbidden in every part of an `addr-spec`, so an encoded-word-looking local-part is malformed rather than an alternate encoding of a plus tag ([RFC 2047 §5](https://datatracker.ietf.org/doc/html/rfc2047#section-5)). + +Then split the semantic local-part at its first literal `+`: + +- no `+` means no tag; +- an empty suffix means no tag; +- otherwise the entire remaining suffix is one flat plus tag, including any later `+` characters; +- an empty prefix is allowed, because `+` is an RFC 5322 `atext` character and this project has no base-recipient constraint ([RFC 5322 §3.2.3](https://datatracker.ietf.org/doc/html/rfc5322#section-3.2.3)). + +### SMTPUTF8 and identity + +Accept a non-ASCII local-part only when the header bytes are valid UTF-8 and the address is valid under the RFC 6532 extensions. RFC 6532 extends atoms, quoted strings, and domains with UTF-8, while RFC 6531 leaves local-part delimiters and parsing otherwise unchanged ([RFC 6532 §§3.1–3.2](https://datatracker.ietf.org/doc/html/rfc6532#section-3.2), [RFC 6531 §§3.2–3.3](https://datatracker.ietf.org/doc/html/rfc6531#section-3.3)). Invalid UTF-8 must never be lossily replaced because replacement could invent a tag or collapse two byte sequences. + +Preserve the semantic suffix's exact Unicode scalar sequence and case. Do not normalize it during extraction or deduplication: internationalized-email guidance says unnormalized local-parts remain valid and warns that normalization behavior belongs to the destination system; local-parts also remain case-sensitive ([RFC 6530 §10.1](https://datatracker.ietf.org/doc/html/rfc6530#section-10.1), [RFC 5321 §2.4](https://datatracker.ietf.org/doc/html/rfc5321#section-2.4)). Consequently `Tag` and `tag`, and canonically equivalent but differently encoded Unicode strings, are distinct plus tags. Quoted and unquoted syntactic forms that decode to the same semantic suffix are duplicates. + +### Malformed data and diagnostics + +A field is malformed for this feature when header framing fails, UTF-8 is invalid where RFC 6532 syntax is needed, parsing does not consume the complete unfolded value, the value has the wrong top-level shape for its field, the `addr-spec` or its domain is incomplete, or an encoded-word occurs inside the `addr-spec`. Such a field contributes no recipient and no plus tag. The message remains available in the Source Maildir and other valid occurrences continue to be processed; emit a rate-limited diagnostic containing the source filename, field name/occurrence, and reason, but avoid logging the full address because delivery fields can disclose recipient and routing information ([RFC 9228 §6](https://www.rfc-editor.org/rfc/rfc9228.html#section-6)). + +## Parser-library implication + +`mail-parser` is a plausible header tokenizer for a Rust implementation: its documented API can parse headers only, configure arbitrary names such as `Delivered-To` and `X-Original-To` with `header_address`, retain repeated fields, and flatten groups through `Address::iter` ([mail-parser 0.11.5 `MessageParser`](https://docs.rs/mail-parser/0.11.5/mail_parser/struct.MessageParser.html), [`Address`](https://docs.rs/mail-parser/0.11.5/mail_parser/enum.Address.html#method.iter)). + +It is **not, by itself, the validity oracle required above**. Its public contract explicitly promises best-effort parsing of non-conformant messages and returns `Option` rather than per-field syntax errors; its address parser returns recovered `Address`/`Empty` values rather than a full-consumption `Result` ([mail-parser crate contract](https://docs.rs/mail-parser/0.11.5/mail_parser/), [`MessageParser::parse`](https://docs.rs/mail-parser/0.11.5/mail_parser/struct.MessageParser.html#method.parse), [address-parser source](https://github.com/stalwartlabs/mail-parser/blob/339a4306d3bd60bbfeae8f5de0e8f29c17d665a1/src/parsers/fields/address.rs#L203-L374)). This is an inference from its documented/source contract: an implementation may use it for framing and candidate AST extraction only if a strict wrapper validates complete field consumption, UTF-8, top-level shape, and semantic local-part decoding. Alternatively, choose a parser that exposes those checks directly. Whichever library is selected must pass the vectors below; no library-specific recovery behavior becomes part of the product contract. + +## Required acceptance vectors + +Assume no other relevant fields unless shown. `=>` lists the resulting plus tags. + +| Input condition | Result | +|---|---| +| `Delivered-To: mail+one@example` and repeated `X-Original-To: other+two@example` | `one`, `two` | +| `Delivered-To:` followed by CRLF + WSP + `mail+folded@example` | `folded` | +| `Delivered-To: plain@example`; `To: mail+visible@example` | `visible` (delivery tier produced no tag) | +| malformed `Delivered-To: mail+bad@`; valid `To: mail+visible@example` | `visible` plus a delivery-field diagnostic | +| `Delivered-To: mail+delivery@example`; `To: mail+visible@example` | `delivery` only | +| delivery fields absent; repeated `To: mail+one@example` and `To: mail+two@example` | `one`, `two` (obsolete repeated-field input) | +| `To: Friends: mail+one@example, mail+two@example;` | `one`, `two` | +| `To: "sales+display" (comment+ignored)` | `real` | +| `To: mail+real(comment+ignored)@example` | `real` after CFWS/comment semantics | +| `To: "mail+quoted"@example` | `quoted` | +| `To: "mail\+quoted"@example` | `quoted` (quoted-pair syntax is semantically invisible) | +| `To: "a@b+tag"@example` | `tag` (the local-part's data `@` is not the addr-spec separator) | +| `To: mail+foo+bar@example` | the one flat tag `foo+bar` | +| `To: +tag@example` | `tag` | +| `To: mail+@example` | no tag | +| `To: mail+same@example, other+same@elsewhere` | one `same` tag | +| `To: "mail+same"@example, mail+same@example` | one `same` tag after semantic dequoting | +| `To: mail+Tag@example, mail+tag@example` | distinct `Tag` and `tag` tags | +| `To: δοκιμή+税@example` as valid UTF-8/RFC 6532 | `税` | +| same logical Unicode spelling in two different normalization forms | two distinct tags | +| invalid UTF-8 in an otherwise address-looking local-part | no tag and a diagnostic | +| RFC 2047 encoded-word used as a local-part | no tag and a diagnostic | +| a syntactically bad `To` list containing one address-looking substring | that occurrence contributes no tags; other valid occurrences still count | +| `Delivered-To: Name ` or a comma-separated delivery value | no tag and a wrong-shape diagnostic | + +The repository's supplied sample exercises the ordinary Postfix/Dovecot path: both `Delivered-To` and `X-Original-To` contain the bare `mail+analouge@philip-henning.com`, so the selected set is exactly `analouge`; the duplicate visible `To` value is never consulted because the delivery tier is already non-empty.