fix: decode _xHHHH_ escapes when reading inline string cells - #991
fix: decode _xHHHH_ escapes when reading inline string cells#991nkuprins wants to merge 11 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes XLSX string decoding so OOXML _xHHHH_ escape sequences are consistently decoded for both shared-strings and inline/direct string cells (t="inlineStr" / t="str"), restoring correct read-after-write behavior (notably for values written with EscapeHexCellWriteHandler).
Changes:
- Introduces
XlsxEscapeUtils.utfDecode(...)and reuses it across both shared-string and inline/direct-string read paths. - Updates
CellTagHandlerto correctly handleDIRECT_STRINGcells by decoding escapes (instead of effectively treating them as error strings). - Adds a regression test covering inline-string decoding and the literal-escape round-trip behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| fesod-sheet/src/test/java/org/apache/fesod/sheet/analysis/v07/handlers/InlineStringUtfDecodeTest.java | Adds regression coverage for inline/direct string escape decoding and literal-escape round-trips. |
| fesod-sheet/src/main/java/org/apache/fesod/sheet/util/XlsxEscapeUtils.java | Centralizes OOXML _xHHHH_ decoding logic for reuse across read paths. |
| fesod-sheet/src/main/java/org/apache/fesod/sheet/analysis/v07/handlers/sax/SharedStringsTableHandler.java | Switches shared-strings decoding to the new shared utility. |
| fesod-sheet/src/main/java/org/apache/fesod/sheet/analysis/v07/handlers/CellTagHandler.java | Decodes escapes for DIRECT_STRING cells and normalizes the resulting type to STRING. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
alaahong
left a comment
There was a problem hiding this comment.
Nice cleanup — moving utfDecode to util/ so CellTagHandler doesn't depend on the shared-strings parser is the right call, and t="str" is covered since it maps to DIRECT_STRING. Comments inline. One more note: the escape convention is now implemented twice (XlsxEscapeUtils for reading, EscapeHexCellWriteHandler for writing); consider cross-referencing them or unifying in XlsxEscapeUtils to prevent silent drift.
| tempCellData.setStringValue(stringValue); | ||
| break; | ||
| case DIRECT_STRING: | ||
| // Undo the '_xHHHH_' escapes of characters XML forbids |
There was a problem hiding this comment.
The reader now decodes _xHHHH_, but the default writer (SXSSF inline strings) never escapes them. Verified locally: writing Product_x0002_Code with the default writer and reading it back now returns Product\u0002Code (STX control char) instead of the literal — before this change it returned the literal. For third-party files this is the correct fix for #696, but for fesod→fesod round trips it's a silent behaviour change. Suggest aligning the writer (escape by default, like EscapeHexCellWriteHandler) or documenting that literal _xHHHH_ text requires registering that handler.
There was a problem hiding this comment.
Hi @alaahong,
I see, but if I take the first option (escape by default), then do you agree that we need an opt-out - escapeHexText(false) on the write? Without it, escapes that arrived from another producer get escaped again on the way in, with no way to write them through unchanged.
Also, probably escape by default isn't a small addition to this PR, so not sure if I should do it here.
Thank you for the review!
There was a problem hiding this comment.
I chose to add javadoc note instead, as suggested in the second option. This probably belongs on the website too, but I'm not sure which section - FAQ?
Related: #696
Purpose of the pull request
Characters that XML 1.0 forbids are stored in a cell as
_xHHHH_escapes. Fesod undid them only for cells backed bysharedStrings.xml, sot="inlineStr"andt="str"reached the caller with the raw escape. A value written by Fesod did not survive being read back by Fesod. The clearest case is a value escaped withEscapeHexCellWriteHandler, which stores the literal_xB9f0_as_x005F_xB9f0_precisely so a decoding reader restores it:Product_x005F_xB9f0_CodeProduct_xB9f0_CodeProduct_x005F_xB9f0_Code/Product_xB9f0_CodeWhy this is a defect and not intended behaviour:
CompatibilityTest#readXlsxWithEscapeSequencepins it forsharedStrings.xml.XSSFCelland the streamingXSSFSheetXMLHandler, each viaXSSFRichTextString#getString().What's changed?
XlsxEscapeUtils(new) -utfDecodemoved here fromSharedStringsTableHandlerunchanged, and both read paths now call it. The smaller change would have been to make the existing methodpublicand call it fromCellTagHandler, but that leaves a cell parser depending on thesharedStrings.xmlparser for decoding that has nothing to do with shared strings, and the next caller inherits the same detour. The escape convention belongs to neither handler, so it moved toutil/. Happy to switch to the two-line version if you prefer the smaller diff.CellTagHandler- theDIRECT_STRINGbranch now decodes instead of falling through to theERRORbranch; it sets the sameSTRINGtype as before.HexEscapeRoundTripTest(new) - one round trip: a cell written asProduct_x0002_Codemust read back asProduct\u0002Code.Checklist