Skip to content

fix: prevent duplicate ObjectReader and ObjectWriter creation on concurrent cache misses - #7774

Open
linw-bai wants to merge 12 commits into
alibaba:mainfrom
linw-bai:fix/object-writer-concurrent-creation
Open

fix: prevent duplicate ObjectReader and ObjectWriter creation on concurrent cache misses#7774
linw-bai wants to merge 12 commits into
alibaba:mainfrom
linw-bai:fix/object-writer-concurrent-creation

Conversation

@linw-bai

@linw-bai linw-bai commented Aug 11, 2026

Copy link
Copy Markdown

Motivation

When multiple threads access the same uncached JavaBean concurrently, every thread may create its own ObjectReader or ObjectWriter before one instance is published by putIfAbsent.

putIfAbsent guarantees that only one instance is retained, but the losing instances have already created their property accessors. This can produce many unnecessary hidden Lambda classes during the first concurrent serialization or deserialization burst.

For a bean with two getters and 100 concurrent first-time serializations, fastjson2 2.0.63 loaded 200 RuleEntity$$Lambda classes instead of the expected 2. Pre-warming ObjectReader does not help serialization because toJSONString uses ObjectWriterProvider.

Changes

  • Coordinate generic reader and writer creation with provider-local locks keyed by the complete Type.
  • Use separate lock maps for normal and field-based caches in each provider.
  • Recheck the selected cache after obtaining the creation lock.
  • Use timed tryLock; if nested user extension code creates a cyclic dependency, the caller falls back to the original create-and-putIfAbsent behavior instead of waiting indefinitely.
  • Remove ClassValue to preserve Android API 26 compatibility.
  • Remove transient locks after creation, while keeping the cache-hit hot paths unchanged and lock-free.

Tests

The concurrent regression tests cover both providers and verify:

  • 32 concurrent first lookups create one shared reader/writer.
  • Normal and field-based caches remain isolated, and each field-based instance is created once.
  • Different parameterizations of the same generic class initialize concurrently.
  • Cyclic nested creation of two cold types terminates instead of deadlocking.

For the original two-getter reproduction with 100 concurrent first-time serializations:

scenario RuleEntity$$Lambda classes loaded
no pre-warming 200
pre-warm ObjectReader only 202
pre-warm ObjectWriter 2
this change, no pre-warming 2

The following commands pass after the change:

./mvnw -pl core -Dtest=ObjectReaderProviderConcurrentTest,ObjectWriterProviderConcurrentTest test
./mvnw -pl core test
./mvnw -pl core -Dfastjson2.creator=reflect test
./mvnw -pl core validate
./mvnw test

The Android Gradle test was not run locally because no Android SDK is installed. The unconditional ClassValue dependency has been removed; the replacement uses APIs available before the project's minSdk 26.

中文

动机

多个线程并发首次访问同一个尚未缓存的 JavaBean 时,每个线程都可能在某个实例通过 putIfAbsent 发布到缓存之前,分别创建自己的 ObjectReaderObjectWriter

putIfAbsent 只能保证最终缓存中保留一个实例,其他竞争失败的实例已经创建了属性访问器,因此首次并发序列化或反序列化时可能产生大量无用的 Lambda 隐藏类。

对于包含两个 getter 的 Bean,使用 100 个线程并发首次序列化时,fastjson2 2.0.63 会加载 200 个 RuleEntity$$Lambda 类,而正常只需要 2 个。预热 ObjectReader 无法规避序列化问题,因为 toJSONString 使用的是 ObjectWriterProvider

改动

  • 使用 Provider 内部、按完整 Type 划分的锁协调通用 reader 和 writer 创建。
  • 每个 Provider 的普通缓存和 FieldBased 缓存分别使用独立锁表。
  • 获得创建锁后再次检查选中的缓存。
  • 使用有超时的 tryLock;如果用户扩展代码的嵌套创建形成循环依赖,则回退到原来的创建并 putIfAbsent 行为,而不是无限等待。
  • 移除 ClassValue,保持 Android API 26 兼容性。
  • 创建结束后移除临时锁,缓存命中的热路径保持不变且不加锁。

测试

Reader 和 Writer 的并发回归测试均覆盖:

  • 32 个线程并发首次获取时只创建一个共享 reader/writer。
  • 普通缓存与 FieldBased 缓存保持隔离,FieldBased 实例也只创建一次。
  • 同一泛型类的不同参数化 Type 可以并行初始化。
  • 两个冷类型的循环嵌套创建能够结束,不会永久死锁。

对于原始的双 getter Bean,使用 100 个线程并发首次序列化:

场景 加载的 RuleEntity$$Lambda 类数
不做预热 200
只预热 ObjectReader 202
预热 ObjectWriter 2
应用本修复,不做预热 2

应用修复后,以下命令均执行通过:

./mvnw -pl core -Dtest=ObjectReaderProviderConcurrentTest,ObjectWriterProviderConcurrentTest test
./mvnw -pl core test
./mvnw -pl core -Dfastjson2.creator=reflect test
./mvnw -pl core validate
./mvnw test

本机没有安装 Android SDK,因此未运行 Android Gradle 测试。无条件使用的 ClassValue 已移除,替代实现使用的 API 均早于项目的 minSdk 26。

@CLAassistant

CLAassistant commented Aug 11, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@linw-bai
linw-bai force-pushed the fix/object-writer-concurrent-creation branch from 91293c9 to fac67db Compare August 11, 2026 02:38
@linw-bai
linw-bai force-pushed the fix/object-writer-concurrent-creation branch from fac67db to 7ab89f3 Compare August 11, 2026 02:41
@linw-bai linw-bai changed the title fix: prevent duplicate ObjectWriter creation on concurrent cache misses fix: prevent duplicate ObjectReader and ObjectWriter creation on concurrent cache misses Aug 11, 2026

@wenshao wenshao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not explored to full depth (tool budget reached): This PR adds per-class ClassValue-based locks in ObjectRe...: none — no check was left unfinished at the ceiling.; This PR adds per-class ClassValue-based locks in ObjectRe...: none — all planned checks completed. (One unverifiable item is embedded in finding 1: whether java.lang.ClassValue exists in the Android SDK at minSdk 26, whi…; This PR adds per-class ClassValue-based locks in ObjectRe...: none — all checks I started were completed within budget..

中文说明

未探索到全部深度(达到工具调用预算):This PR adds per-class ClassValue-based locks in ObjectRe...:none — no check was left unfinished at the ceiling.;This PR adds per-class ClassValue-based locks in ObjectRe...:none — all planned checks completed. (One unverifiable item is embedded in finding 1: whether java.lang.ClassValue exists in the Android SDK at minSdk 26, whi…;This PR adds per-class ClassValue-based locks in ObjectRe...:none — all checks I started were completed within budget.

— qwen3.8-max via Qwen Code /review (v0.21.10)

Comment thread core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
@linw-bai
linw-bai requested a review from wenshao August 12, 2026 09:03
Comment thread core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
@linw-bai
linw-bai requested a review from wenshao August 14, 2026 02:14

@wenshao wenshao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not explored to full depth (tool budget reached): "agent 1a": ; "agent 6c": none — no check was cut short..

中文说明

未探索到全部深度(达到工具调用预算):"agent 1a""agent 6c"none — no check was cut short.

— qwen3.8-max via Qwen Code /review (v0.21.11)

Comment thread core/src/main/java/com/alibaba/fastjson2/internal/CodecCreationCoordinator.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/internal/CodecCreationCoordinator.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
@linw-bai
linw-bai requested a review from wenshao August 20, 2026 02:18

@wenshao wenshao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferred under the convergence posture (round 4, not a blocker) — recorded, not requested in this round:

  • core/src/test/java/com/alibaba/fastjson2/writer/ObjectWriterProviderConcurrentTest.java:341 (+1 locations) — [review] unused threadCount locals + unread createCount result fields in both provider concurrent tests (reader twin included)
中文说明

收敛姿态下延后(第 4 轮,非阻断)——已记录,本轮不要求修改:共 1 条(原文未翻译,列表见上方英文部分)。

— qwen3.8-max via Qwen Code /review (v0.21.11)

Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
@linw-bai
linw-bai requested a review from wenshao August 23, 2026 04:53

@wenshao wenshao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferred under the convergence posture (round 5, not a blocker) — recorded, not requested in this round:

  • core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java:1162 (+2 locations) — [probe] Error-type creator failure path has no test — narrowing the catch to RuntimeException survives the whole suite (writer twin identical)
  • core/src/test/java/com/alibaba/fastjson2/writer/ObjectWriterProviderConcurrentTest.java:419 (+2 locations) — [probe] testParameterizedTypesCreateConcurrently discards returned codecs — a raw-Class-keyed publish/lookup mutant survives (reade…

Convergence: round 5 posted 5 inline comment(s), 5 of them reported for the first time; the previous round posted 12. Findings keep coming back to the same files: core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java (findings in rounds 3, 4; 1 more now); core/src/main/java/com/alibaba/fastjson2/internal/CodecCreationCoordinator.java (findings in round 4; 1 more now); core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java (findings in round 4; 1 more now), and 2 more file(s). A cluster that keeps producing siblings usually means the fixes are treating instances of a shared root cause — triaging that cause before the next round, or splitting an independent cluster into its own pull request, tends to end the loop faster than fixing them one at a time. (Observation only — nothing was withheld from this review because of this observation.)

中文说明

收敛姿态下延后(第 5 轮,非阻断)——已记录,本轮不要求修改:共 2 条(原文未翻译,列表见上方英文部分)。

收敛情况:第 5 轮发布了 5 条行内评论,其中 5 条是首次提出;上一轮发布了 12 条。发现反复回到同一批文件:core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java(第 3、4 轮已出过发现,本轮又有 1 条);core/src/main/java/com/alibaba/fastjson2/internal/CodecCreationCoordinator.java(第 4 轮已出过发现,本轮又有 1 条);core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java(第 4 轮已出过发现,本轮又有 1 条),另有 2 个文件。一个不断再生兄弟发现的簇,通常意味着逐条修复只在处理同一根因的实例——先定位并处理该根因,或把独立的簇拆成单独的 PR,通常比逐条修复更快结束循环。(仅为观察——本轮评审未因此扣留任何内容。)

— qwen3.8-max via Qwen Code /review (v0.22.0)

Comment thread core/src/main/java/com/alibaba/fastjson2/reader/ObjectReaderProvider.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/writer/ObjectWriterProvider.java Outdated
Rate-limit timed fallback admissions atomically without blocking behind older fallbacks. Track every active fallback for cycle detection and let queued readers and writers reuse published cache values.

Add external-dependency, stale-deadline, multi-owner cycle and cache-publication regressions. Validated core default/reflect suites, JDK 8/11 concurrency tests and 20 coordinator repetitions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants