Improves transfer policies

Author: IvanChernyshov

CHANGELOG.md

@@ -1,5 +1,87 @@
 # Changelog
 
+## 0.1.4 - 2026-03-15
+
+### Added
+
+- explicit nested-policy safeguards for `LinearTransfer`:
+  - `fit_sources`
+  - `fit_max_depth`
+  - `prediction_sources`
+  - `prediction_max_depth`
+- `LookupResult.transfer_depth` to record how many transfer steps produced the
+  returned value.
+
+### Changed
+
+- linear-transfer fitting now distinguishes direct predictor values from nested
+  policy-derived predictor values.
+- the default nested linear-transfer behavior is now conservative for fitting
+  and allows at most one additional completion step for the final predictor
+  value.
+- cycle detection now uses context-local resolution tokens and correctly catches
+  recursion through wrapper policies such as `RadiiPolicy` and `XHPolicy`.
+- docs were expanded to explain nested-policy predictors, transfer depth, and
+  cycle detection.
+
+## 0.1.4 - 2026-03-15
+
+### Added
+
+- `LookupResult.transfer_depth`, which records how many transfer steps were
+  involved in the returned numeric value.
+- Source/depth controls for nested linear-transfer workflows via
+  `LinearTransfer.fit_sources`, `LinearTransfer.fit_max_depth`,
+  `LinearTransfer.prediction_sources`, and `LinearTransfer.prediction_max_depth`.
+- Regression tests covering generic-policy cycles, wrapper-policy cycles,
+  conservative nested-fit defaults, and explicit opt-in for deeper nested
+  linear workflows.
+
+### Changed
+
+- Nested policy-backed linear transfers are now guarded in two phases:
+  conservative defaults are used for fit training, while one additional nested
+  completion step remains allowed at prediction time.
+- Cycle detection now tracks both generic policies and wrapper policies using a
+  context-local activation stack, so recursion through freshly materialized
+  wrapper policies is detected reliably and safely.
+- Radii and X–H convenience helpers now resolve through wrapper-aware cycle
+  tracking rather than materializing a fresh generic policy for each public
+  lookup call.
+
+### Documentation
+
+- Expanded the transfer and policy docs to explain nested-policy safeguards,
+  `transfer_depth`, and cycle detection.
+- Added guidance on when chained correlations are scientifically reasonable and
+  how to opt in deliberately when broader fit training is desired.
+
+## 0.1.4 - 2026-03-15
+
+### Added
+
+- `LookupResult.transfer_depth` is now used consistently across nested
+  substitution and linear-transfer workflows so callers can tell how many
+  transfer steps contributed to a returned value.
+- New tests covering nested-policy fit controls, prediction-depth limits, and
+  cycle detection for both generic and wrapper policies.
+
+### Changed
+
+- `LinearTransfer` now distinguishes between values that may participate in
+  fitting (`fit_sources`, `fit_max_depth`) and values that may be used for the
+  final element-specific predictor lookup (`prediction_sources`,
+  `prediction_max_depth`).
+- The default linear-transfer behavior is now conservative for fitting
+  (direct predictor values only) while still allowing one nested completion
+  step during final prediction.
+- Policy-resolution cycle detection now tracks wrapper-policy identities as
+  well as generic `ValuePolicy` objects and is stored in a context-local stack
+  instead of a process-global mutable list.
+- Quantity wrappers continue to use the generic policy core, but now route
+  through wrapper-aware lookup helpers so cycle checks remain effective for
+  `RadiiPolicy` and `XHPolicy`.
+
 ## 0.1.3 - 2026-03-15
 
 ### Added

README.md

@@ -22,6 +22,8 @@ What you get in the current `0.1.x` line:
 - dataset provenance and coverage metadata,
 - deterministic lookup policies,
 - substitution and linear transfer from support datasets or policies into target datasets,
+- guarded nested policy-backed transfers with explicit transfer depth,
+  conservative fit/prediction controls, and cycle detection,
 - user-defined custom element-indexed scalar sets.
 
 ## Core terms
@@ -65,6 +67,13 @@ The default `0.1.x` behavior is intentionally simple and practical:
   elements inferred from **Cordero covalent radii** through a fitted linear
   policy.
 
+Nested policy predictors are supported too. In `0.1.4`, `LinearTransfer`
+separates **fit-time** use of nested predictor values from
+**prediction-time** use. By default, the fit may use only direct nested
+values, while the final requested element may still use one additional
+nested completion step. That is a useful compromise for workflows such as
+provisional X–H inference from a chosen covalent-radii policy.
+
 ## Quick example
 
 ```pycon
@@ -80,13 +89,15 @@ The default `0.1.x` behavior is intentionally simple and practical:
 2.8972265395148358
 >>> lookup.source
 'transfer_linear'
+>>> lookup.transfer_depth
+1
 >>> lookup.resolved_from
 (DatasetRef(quantity='atomic_radius', set_id='rahm2016'),)
 ```
 
 `get_*` returns only the number. `lookup_*` returns a `LookupResult` that also
-records where the value came from and whether a transfer model or policy source
-was involved.
+records where the value came from, whether a transfer model or policy source was
+involved, and how many transfer steps were needed (`transfer_depth`).
 
 You can inspect the packaged quantity and dataset catalog directly:
 

docs/api/policy.md

@@ -3,7 +3,7 @@
 This module contains the generic resolver that sits below the radii-specific and
 X–H-specific convenience APIs.
 
-Use it when you want to work directly with the common value-selection engine:
+Use it when you want to work directly with the shared value-selection engine:
 
 - `ValuePolicy` — generic element-domain policy configuration,
 - `lookup_value(...)` — resolve one value together with provenance,
@@ -18,5 +18,9 @@ A few practical notes:
   wrapper policies that expose `as_value_policy()`.
 - `LookupResult.is_placeholder` refers to the returned numeric value itself, not
   to whether any transfer happened.
+- `LookupResult.transfer_depth` counts how many transfer steps were involved in
+  the returned numeric value.
+- Nested lookup is cycle-checked across both generic `ValuePolicy` objects and
+  wrapper policies such as `RadiiPolicy` and `XHPolicy`.
 
 ::: atomref.policy

docs/api/transfer.md

@@ -18,4 +18,22 @@ A transfer source may be:
 `LinearTransfer` currently accepts exactly one predictor source at runtime, even
 though the public API stores predictors as a tuple for forward compatibility.
 
+For policy-backed linear predictors, `LinearTransfer` separates two questions:
+
+- which nested predictor values may be used to **fit** the linear model
+  (`fit_sources`, `fit_max_depth`), and
+- which nested predictor values may be used to **predict** the final requested
+  element (`prediction_sources`, `prediction_max_depth`).
+
+The defaults are intentionally conservative:
+
+- fit only on nested predictor values that came directly from `base` or
+  `override`,
+- but allow one additional nested transfer step when evaluating the predictor
+  for the requested element.
+
+That default is meant for workflows such as a sparse X–H target set correlated
+against a partial covalent-radii policy that is itself completed from a broader
+support set.
+
 ::: atomref.transfer

docs/api/xh.md

@@ -17,6 +17,8 @@ In the default policy:
 
 - `C`, `N`, and `O` use curated ConQuest/CSD defaults,
 - other parent elements may be inferred from `cordero2008`,
+- policy-backed predictors are supported as well, with conservative nested-fit
+  defaults and one additional nested prediction step allowed by default,
 - fuller X–H literature support is planned for `0.2.x`.
 
 ::: atomref.xh

docs/dev/architecture.md

@@ -63,6 +63,31 @@ That last point is important. It means higher-level code can express
 "infer values from my chosen covalent-radii policy" instead of being forced to
 refer to one hard-coded predictor dataset.
 
+## Nested-policy safeguards and cycle detection
+
+Policy-backed transfer sources are materialized with more than just raw numeric
+values. The resolver also tracks, per element:
+
+- whether the value came from `base`, `override`, substitution, linear transfer,
+  or fallback,
+- the nested transfer depth that was required to produce it,
+- placeholder status.
+
+`LinearTransfer` uses that information twice:
+
+- once when fitting the linear relation (`fit_sources` / `fit_max_depth`),
+- again when deciding whether the predictor value for the requested element is
+  admissible (`prediction_sources` / `prediction_max_depth`).
+
+The default policy is intentionally conservative: fit only on direct nested
+predictor values, but allow one additional nested completion step when
+predicting the final requested element. This keeps the common two-stage use case
+possible without silently training on arbitrarily long inference chains.
+
+Cycle detection is handled with a context-local activation stack. Both generic
+`ValuePolicy` objects and wrapper policies are tracked, so recursion through a
+freshly materialized wrapper policy is still detected reliably and safely.
+
 ## Placeholder handling
 
 Placeholder semantics stay attached to the value that was actually returned.
@@ -73,7 +98,8 @@ This means `LookupResult.is_placeholder` can be true for:
 - a nested policy used as a transfer source.
 
 A linear transfer normally returns a computed value and therefore does not carry
-placeholder status itself.
+placeholder status itself. Instead, its provenance is carried by
+`resolved_from`, explanatory notes, and `transfer_depth`.
 
 ## Why the design stays small
 

docs/guide/policies.md

@@ -69,7 +69,7 @@ current runtime intentionally supports exactly one predictor source. That keeps
 the implementation simple now while leaving room for later multi-predictor
 linear models.
 
-Transfer sources can now be:
+Transfer sources can be:
 
 - a packaged dataset reference (`DatasetRef`),
 - a custom `ElementScalarSet`,
@@ -81,6 +81,35 @@ that policy. This lets higher-level workflows express things like “infer X–H
 lengths from my chosen covalent-radii policy” instead of hard-coding a specific
 support dataset.
 
+#### Nested policy safeguards for `LinearTransfer`
+
+When a predictor source is itself a policy, two different questions matter:
+
+1. Which nested predictor values are trustworthy enough to train the linear fit?
+2. Which nested predictor value is acceptable for the final requested element?
+
+`atomref` keeps those two decisions separate. By default:
+
+- `fit_sources=("base", "override")` and `fit_max_depth=0`,
+- `prediction_sources=("base", "override", "transfer_substitution", "transfer_linear")`
+  and `prediction_max_depth=1`.
+
+That means the fitted relationship is trained only on direct predictor values by
+default, while one additional nested completion step is still allowed at
+prediction time.
+
+This is a good default for workflows such as:
+
+- sparse target X–H data from `csd_legacy_xh_cno`,
+- a partial covalent-radii predictor policy with direct `s,p` values,
+- one inner transfer from a broader support set such as `cordero2008` to make
+  the predictor usable for `d` or `f` elements.
+
+In that setup, the outer X–H fit still uses direct predictor anchors, while the
+final requested element may use one nested predictor transfer. If you really do
+want fit training to use nested predictor values as well, you can opt in
+explicitly by widening `fit_sources` and/or increasing `fit_max_depth`.
+
 ### Fallback
 
 A fallback is a constant last-resort value. It is useful when an algorithm must
@@ -112,6 +141,24 @@ It does **not** mean “a transfer happened”. Examples:
 - a linear transfer is computed, not copied, so `is_placeholder` is normally
   `False`.
 
+## Transfer depth and cycle detection
+
+`LookupResult.transfer_depth` counts how many transfer steps were needed to
+produce the returned value:
+
+- direct base and override values have depth `0`,
+- one substitution or linear restoration has depth `1`,
+- nested transfer chains increase the depth further.
+
+This makes nested-policy behavior inspectable without trying to infer it from
+notes alone.
+
+Because policies may now depend on other policies, the resolver also performs
+cycle detection. A cyclic reference such as policy A depending on policy B while
+policy B depends back on policy A raises `PolicyError` instead of recurring
+indefinitely. The same protection applies when recursion goes through wrapper
+policies such as `RadiiPolicy` or `XHPolicy`.
+
 ## Target datasets and support datasets
 
 `atomref` separates **what a dataset is used for** from **what it scientifically
@@ -171,5 +218,5 @@ With that X–H policy:
 - missing parent elements may be inferred from the **selected covalent-radii
   policy**, not just from one hard-coded support dataset,
 - if the predictor policy itself needed a transfer to produce a covalent radius,
-  the resulting `LookupResult` still records that provenance in `resolved_from`
-  and `notes`.
+  the resulting `LookupResult` still records that provenance in `resolved_from`,
+  `notes`, and `transfer_depth`.

docs/index.md

@@ -22,6 +22,8 @@ What you get in the current `0.1.x` line:
 - dataset provenance and coverage metadata,
 - deterministic lookup policies,
 - substitution and linear transfer from support datasets or policies into target datasets,
+- guarded nested policy-backed transfers with explicit transfer depth,
+  conservative fit/prediction controls, and cycle detection,
 - user-defined custom element-indexed scalar sets.
 
 ## Core terms
@@ -65,6 +67,13 @@ The default `0.1.x` behavior is intentionally simple and practical:
   elements inferred from **Cordero covalent radii** through a fitted linear
   policy.
 
+Nested policy predictors are supported too. In `0.1.4`, `LinearTransfer`
+separates **fit-time** use of nested predictor values from
+**prediction-time** use. By default, the fit may use only direct nested
+values, while the final requested element may still use one additional
+nested completion step. That is a useful compromise for workflows such as
+provisional X–H inference from a chosen covalent-radii policy.
+
 ## Quick example
 
 ```pycon
@@ -80,13 +89,15 @@ The default `0.1.x` behavior is intentionally simple and practical:
 2.8972265395148358
 >>> lookup.source
 'transfer_linear'
+>>> lookup.transfer_depth
+1
 >>> lookup.resolved_from
 (DatasetRef(quantity='atomic_radius', set_id='rahm2016'),)
 ```
 
 `get_*` returns only the number. `lookup_*` returns a `LookupResult` that also
-records where the value came from and whether a transfer model or policy source
-was involved.
+records where the value came from, whether a transfer model or policy source was
+involved, and how many transfer steps were needed (`transfer_depth`).
 
 You can inspect the packaged quantity and dataset catalog directly:
 

docs/notebooks/03-custom-sets-and-discovery.md

@@ -33,9 +33,9 @@ for symbol in ("C", "O", "N"):
 ```
 **Output**
 ```text
-C LookupResult(value=0.77, source='base', target=DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'), resolved_from=(DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'),), is_placeholder=False, fit=None, notes=())
-O LookupResult(value=0.67, source='base', target=DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'), resolved_from=(DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'),), is_placeholder=False, fit=None, notes=())
-N LookupResult(value=0.71, source='transfer_substitution', target=DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'), resolved_from=(DatasetRef(quantity='covalent_radius', set_id='cordero2008'),), is_placeholder=False, fit=None, notes=('missing in base set; substituted from transfer source',))
+C LookupResult(value=0.77, source='base', target=DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'), resolved_from=(DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'),), is_placeholder=False, fit=None, notes=(), transfer_depth=0)
+O LookupResult(value=0.67, source='base', target=DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'), resolved_from=(DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'),), is_placeholder=False, fit=None, notes=(), transfer_depth=0)
+N LookupResult(value=0.71, source='transfer_substitution', target=DatasetRef(quantity='covalent_radius', set_id='demo_user_cov'), resolved_from=(DatasetRef(quantity='covalent_radius', set_id='cordero2008'),), is_placeholder=False, fit=None, notes=('missing in base set; substituted from transfer source',), transfer_depth=1)
 ```
 ```python
 for info in ar.list_radii_set_infos("van_der_waals", usage_role="target"):

src/atomref/__about__.py

@@ -1 +1 @@
-__version__ = "0.1.3"
+__version__ = "0.1.4"