docs/grammar.md diverges from the parser in 6 places (string/char escapes, ranges, exponent)

[enomado]

Summary

While building a differential between a PEG reference transcribed verbatim from
docs/grammar.md
and the actual parser, I found 6 places where the documented grammar
disagrees with what ron parses. In every case the parser behaves sensibly —
it's the grammar doc that is wrong. All are present on master.

Method: a reference acceptor (pest) checked against
Deserializer::from_str -> IgnoredAny::deserialize -> end() (pure syntactic
validation of one value; Value can't be used as the oracle since it lacks
struct/enum/range/byte variants). The reference was then hardened with mutation
fuzzing of valid seeds, which is what surfaced the range / char-apostrophe /
exponent findings below — cases too easy to miss by hand.

Repro convention below: "ron: accept/reject" is the result of the oracle above.
Line references are from v0.12.2 but the productions are unchanged on master.

---

1. string_std / byte_string_std — a lone \ is allowed as a literal

string_std = "\"", { no_double_quotation_marks | string_escape }, "\"";

no_double_quotation_marks excludes only ", so the grammar accepts a backslash
that does not begin a valid escape as a literal character — e.g. "a\c". The
parser rejects this: inside a std string a \ must introduce a valid escape
(parse_escape → InvalidEscape("Unknown escape character")). Same for b"…".

• "a\c" → grammar: accept, ron: reject
• "a\"b\c" → grammar: accept, ron: reject

Fix: the non-escape character class must exclude \ as well as ", i.e. a
\ always introduces a string_escape:

string_std = "\"", { no_double_quote_or_backslash | string_escape }, "\"";

2. char — escape set is too narrow

char = "'", (no_apostrophe | "\\\\" | "\\'"), "'";

Only \\ and \' are allowed, but char() calls the same
parse_escape(is_char = true) as strings, so the full escape set works:

• '\n', '\t', '\0' → grammar: reject, ron: accept
• '\x41' → grammar: reject, ron: accept (single byte, must be <= 0x7F)
• '\u{41}' → grammar: reject, ron: accept

Fix: allow the string escape set in char (with the note that \xHH in a
char must be a single valid scalar, no multi-byte chaining):

char = "'", (no_apostrophe | char_escape), "'";

3. escape_unicode — uses no braces; should require \u{…}

escape_unicode = "u", digit_hexadecimal, [digit_hexadecimal, [ … up to 6 ]];

The parser (parse_escape, the 'u' arm) requires \u{ 1..=6 hex } and a
valid Unicode scalar: expect_char('{'), 1–6 hex digits, expect_char('}'),
char_from_u32.

• "�" → grammar: accept, ron: reject (Missing { in Unicode escape)
• "\u{FFFF}" → grammar: reject, ron: accept
• "\u{}" (0 digits), "\u{110000}" (out of range), "\u{D800}" (surrogate)
  → ron: reject

Fix:

escape_unicode = "u", "{", digit_hexadecimal, { digit_hexadecimal }, "}";

(1–6 hex digits; must be a valid Unicode scalar — a semantic note.)

4. Range operators — the grammar allows whitespace around .. / ..=, the parser does not

range_from_to_exclusive = number, ws, "..", ws, number;
range_from_to_inclusive = number, ws, "..=", ws, number;
range_from              = number, ws, "..";
range_to_inclusive      = "..=", ws, number;
range_to_exclusive      = "..", ws, number;

handle_float_range_or_value consumes ..= / .. with consume_str and no
skip_ws — neither before the operator nor after it. So any whitespace around
the range operator is rejected.

• 1..2, ..2, .. → ron: accept
• 1 ..2, 1.. 2, 1 .. 2, ..= 2, .. 5, 1 .. → grammar: accept, ron: reject

Fix: drop the ws from the six range productions (the operator is adjacent
to its operands). As a bonus this removes the ..= before .. ambiguity note and
makes 1. vs 1..2 unambiguous.

5. char — an unescaped ' is accepted as the content

char = "'", (no_apostrophe | …), "'";

no_apostrophe says an apostrophe inside a char must be escaped (\'). But
char() reads next_char() and, if it isn't \, takes the character as-is —
including a bare '. So ''' parses as the char '.

• ''' → grammar: reject, ron: accept (the char ')

(This is the same char production as #2, wrong on two counts.)

Fix: document that char takes any single character or an escape (the parser
does not restrict the unescaped case), or, if this is considered a parser bug,
treat it as such.

6. float_exp — the parser allows _ before the exponent sign

float_exp = ("e" | "E"), ["+" | "-"], { digit | "_" }, digit, { digit | "_" };

The grammar places the sign strictly before any underscores. The float scanner
allows _ after any digit / e / E, so _ may also precede the sign:

• 1e_+0, 1E_-3, 1e__+9, 1_e_+0 → grammar: reject, ron: accept

Fix: describe the actual underscore rule (allowed after a digit/e/E,
forbidden at the start of the mantissa/exponent), which is broader than the
positional EBNF.

7 (minor). Leading whitespace before extensions

RON = [extensions], ws, value, ws;

Nothing precedes [extensions], but the parser accepts whitespace/comments
before #![enable(…)].

• #![enable(unwrap_newtypes)]\n(a: 1) → grammar: reject, ron: accept

Fix: RON = ws, [extensions], ws, value, ws;

---

Not grammar bugs (listed to pre-empt noise)

These are semantic layers on top of the syntax and correctly not encoded in the
grammar; a differential surfaces them but they should not change grammar.md:

• integer not fitting its suffix (911u8), negative unsigned (-1u8);
• Some / None reserved on the self-describing (deserialize_any) path
  (Some, Some() are accepted by the grammar as unit/tuple structs);
• extension names are a closed set (implicit_some, unwrap_newtypes,
  unwrap_variant_newtypes); a typo is syntactically fine but rejected;
• a \xHH > 0x7F in a UTF-8 string must combine into valid UTF-8.

Offer

Happy to open a PR fixing grammar.md for items 1–7. Separately, ron currently
has no grammar-conformance test — I have the differential harness (pest
reference + oracle + mutation) and can contribute a tests/grammar_conformance.rs
with an isolated CI job, in the spirit of the scaling-regression test in #611.