fix: bound untrusted GeoJSON input to prevent parser resource exhaustion - #1733
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
googlemaps#1699 and googlemaps#1710 added nesting-depth guards, but they run on the tree that Json.parseToJsonElement() has already fully materialised, so they act too late: GeoJsonParser.parse() reads and materialises the whole untrusted document before any check. A hostile layer can therefore still - overflow the parser stack with deep nesting (StackOverflowError) before the MAX_GEOMETRY_DEPTH=20 guard runs (contrast KmlParser, which bounds depth *during* streaming via DepthLimitingReader); - exhaust the heap with a wide, shallow document (OutOfMemoryError), which no depth guard addresses; - forward non-finite coordinates ("Infinity"/"NaN", accepted by String.toDouble()) straight into LatLng/LatLngBounds. The first two throw java.lang.Error subclasses, so the catch (Exception) in DataLayerLoader does not contain them and the host app crashes on load. Bound the input up front, before parseToJsonElement: - readTextBounded() caps the characters read (configurable, default 10 MiB); - checkStructuralDepth() rejects raw '{'/'[' nesting beyond a limit in a single string-aware pass (configurable, default 512 - safe on small Android stacks, far above any legitimate GeoJSON); - parseCoordinates() rejects non-finite values. Limits are constructor parameters with safe defaults, mirroring KmzParser. Existing behaviour is preserved (incl. the googlemaps#1699 depth-200 GeometryCollection test); adds tests for each case.
0e1e861 to
7a73969
Compare
|
Hello @agilira Thanks for the PR. Running So, I think you could either significantly increase the depth (or use a shape that doesn't trip the coordinate-type check first) so it actually reproduces a stack overflow on unpatched code and demonstrates the guard catching it. Or we could drop the "prevents stack overflow" framing from this test's name/comment and just fold it into the existing structural-depth coverage, since Would you be able to update the PR? Thanks! |
…ally The previous version of this test could pass on unpatched code for an unrelated reason: with the nesting inside "coordinates", a parse that survives the stack reaches parseCoordinates(), which calls jsonPrimitive on a JsonArray and throws IllegalArgumentException - the very type the test asserts. Move the nesting under "bbox", a member the parser never dereferences, so nothing can reject the document first, and run the parse on a thread with an explicit 1 MB stack so the outcome does not depend on the stack size of the thread the test framework happens to use. 50 000 levels overflow that stack whether or not the parser is JIT-compiled, while the structural-depth check rejects the document at nesting level 513 without recursing at all. A small document is parsed first so that class loading is not charged to the bounded stack. On main this test now fails with StackOverflowError instead of passing.
|
Thanks — you're right, and the reason it passes on With the nesting inside Whether the overflow happens also depends on the stack of the thread running the test, and on whether the parser is still interpreted. Measured on 5.0.0 with kotlinx-serialization 1.9.0, first nesting depth that overflows, once JIT-compiled:
Cold, the thresholds are several times lower (at 1 MB a first-run parse already overflows somewhere between 2,000 and 5,000). Also worth noting: only nested objects switch to the stackless path ( I took your first option and made it deterministic:
Result: with only this test added on top of and on this branch the whole file passes (21/21 locally; CI still needs your approval to run). Renamed to If you'd still rather not have a stack-sensitive test in the suite, I'm happy to take your second option instead and fold it into the existing structural-depth coverage — just say the word. |
dkhawk
left a comment
There was a problem hiding this comment.
Thank you @agilira for this pull request and for updating the regression test to deterministically demonstrate stack overflow prevention.
Review Summary
- Root Cause & Solution: Correctly identifies that deeply nested GeoJSON arrays or objects can cause a
StackOverflowErrorduringJson.parseToJsonElement(), and that oversized documents can cause out-of-memory errors. The streaming single-pass structural depth check (checkStructuralDepth) and bounded input reader (readTextBounded) prevent both resource exhaustion vectors cleanly. - Poisoned Coordinate Protection: Checking finite coordinates (
isFinite()) inparseCoordinates()ensures non-finite strings like"Infinity"or"NaN"cannot propagate downstream into SDK rendering models. - Verification & Testing:
- Tested
testDeeplyNestedArraysAreRejectedInsteadOfOverflowingTheParserStackon unpatchedmain: confirmed deterministic failure (java.lang.StackOverflowError). - Tested on PR #1733: confirmed 100% passing tests across the entire suite (
./gradlew test). - Verified backward compatibility for existing call sites.
- Tested
LGTM.
Summary
Harden
GeoJsonParseragainst resource exhaustion when parsing untrusted GeoJSON, by bounding the input before it is materialised. This closes the remaining gap left by the nesting-depth guards added in #1699 and #1710.Problem
#1699 (
MAX_GEOMETRY_DEPTH) and #1710 both guard against deeply nested input, but the GeoJSON guard runs during a second pass over the tree thatJson.parseToJsonElement()has already fully materialised:So for GeoJSON the guard acts too late. With an untrusted document a caller can still hit:
StackOverflowError.kotlinx-serialization's JSON reader recurses on nested arrays, so a few thousand nested arrays overflow the stack insideparseToJsonElement, beforeMAX_GEOMETRY_DEPTHis ever reached. (KmlParseravoids this by bounding depth during streaming withDepthLimitingReader; GeoJSON has no equivalent.)OutOfMemoryError. A large flat array (e.g. aMultiPointwith millions of positions) materialises to far more heap than the source text, with no nesting for a depth guard to catch."Infinity"/"NaN"are accepted byString.toDouble()and currently flow straight intoLatLng/LatLngBounds.The first two throw
java.lang.Errorsubclasses, so thecatch (e: Exception)inDataLayerLoaderdoes not contain them and the failure propagates to the host app.Fix
Bound the input up front, before
parseToJsonElement():readTextBounded()— caps the number of characters read (configurable, default 10 MiB), so an oversized document is never fully materialised.checkStructuralDepth()— rejects raw{/[nesting beyond a limit, in a single string-aware pass (configurable, default512— safe on small Android thread stacks, and far above any legitimate GeoJSON, whose structural depth stays well under 100 even with maximally nested geometries).parseCoordinates()— rejects non-finite values.Both limits are constructor parameters with safe defaults, mirroring
KmzParser's configurable zip-bomb limits (#1677). Rejected input throwsIllegalArgumentException(anException), soDataLayerLoaderdegrades to anulllayer instead of crashing.Testing
Adds unit tests for each case (deep-nested arrays, excessively nested
GeometryCollection, oversized input, non-finite coordinate). Existing behaviour is preserved, including the #1699testDeeplyNestedGeometryCollectionDoesNotThrowStackOverflowcase (depth 200 stays well within the structural limit and still parses).Compatibility
GeoJsonParser's new parameters have defaults, soGeoJsonParser()continues to work unchanged for all existing (Kotlin) call sites (DataLayerLoader,GeoJsonLayer). No@JvmOverloadsis added, matching the siblingKmzParser.