Add public tree read/match/transform utilities

CLAUDE.md

@@ -6,6 +6,7 @@ An implicit configuration system for Python. Tracks constructor arguments, suppo
 
 - `src/confingy/` — library source (src layout)
   - `tracking.py` — core `@track`, `lazy`, `lens`, `Lazy[T]`, `disable_validation`
+  - `tree.py` — `get_init_args`, `is_fingy_of`, `walk_fingy`, `map_fingy`, `replace_args` (tree read/match/transform)
   - `fingy.py` — `serialize_fingy`, `deserialize_fingy`, `save_fingy`, `load_fingy`, `transpile_fingy`, `prettify_fingy`
   - `serde.py` — serialization/deserialization internals (handlers, registry, keys)
   - `exceptions.py` — `ValidationError`, `SerializationError`, `DeserializationError`

docs/api/tree.md

@@ -0,0 +1,3 @@
+# confingy.tree
+
+::: confingy.tree

docs/api/utils/containers.md

@@ -0,0 +1,3 @@
+# confingy.utils.containers
+
+::: confingy.utils.containers

docs/diving_deeper.md

@@ -163,10 +163,67 @@ print(modified_trainer.optimizer.lr)  # 0.01
 ```
 
 !!! warning
-    `unlens()` will **reinstantiate** any tracked objects with the updated arguments. This means the returned object is a new instance, not the original with mutated state.
+    `unlens()` will **reinstantiate** modified tracked objects, and their ancestors, with the updated arguments. The returned objects are new instances, not the originals with mutated state. Nodes you didn't modify are returned as the original objects, and an object shared by several parents stays shared.
 
 For more examples, see [Lensing and Unlensing](examples/lens.md).
 
+#### Transforming Whole Trees
+
+`lens()` works well when you know the path to the value you want to change. To change *every* object of a given class, wherever it appears in the tree, use the functions in [confingy.tree](api/tree.md):
+
+```python
+from confingy import get_init_args, is_fingy_of, map_fingy, replace_args, track, update, walk_fingy
+
+@track
+class Scaler:
+    def __init__(self, factor: float = 1.0):
+        self.factor = factor
+
+@track
+class Pipeline:
+    def __init__(self, steps: list):
+        self.steps = steps
+
+@track
+class Registry:
+    def __init__(self, entries: dict):
+        self.entries = entries
+
+registry = Registry(entries={
+    "a": Pipeline(steps=[Scaler(), "passthrough"]),
+    "b": Pipeline(steps=[Scaler(factor=2.0)]),
+})
+
+# Inspect every tracked or lazy node
+for path, node in walk_fingy(registry):
+    print(repr(path), type(node).__name__, get_init_args(node))
+# '' Registry {...}
+# "entries['a']" Pipeline {...}
+# "entries['a'].steps[0]" Scaler {'factor': 1.0}
+# "entries['b']" Pipeline {...}
+# "entries['b'].steps[0]" Scaler {'factor': 2.0}
+
+# Set arguments on every matching node
+new_registry, n_rebuilt = replace_args(registry, Scaler, factor=3.0)
+print(n_rebuilt)                                  # 2
+print(new_registry.entries["a"].steps[0].factor)  # 3.0
+print(registry.entries["a"].steps[0].factor)      # 1.0: the input is unchanged
+
+# Or apply any node-level transform
+def cap(node):
+    if is_fingy_of(node, Scaler) and get_init_args(node)["factor"] > 1.0:
+        return update(node)(factor=1.0)
+    return node
+
+capped = map_fingy(registry, cap)
+print(capped.entries["a"] is registry.entries["a"])  # True: untouched subtrees are reused
+```
+
+[map_fingy()][confingy.tree.map_fingy] rebuilds a node with [update()][confingy.tracking.update] only when one of its children changed, so untouched subtrees keep their identity. Lazy nodes stay lazy, tracked nodes stay tracked, and a node shared by several parents is transformed once and stays shared. [is_fingy_of()][confingy.tree.is_fingy_of] matches tracked and lazy objects of a class or any subclass, including the subclasses created by `track(SomeClass)`.
+
+!!! note
+    Rebuilding re-runs each rebuilt node's constructor (or, for `Lazy` nodes, its validation and `__post_config__` hook), so any side effects happen again. Children are found through recorded constructor arguments, so attributes set after construction are not carried over.
+
 ## Migrating to Confingy
 
 One of `confingy`'s central tenets is that it should be easy to migrate to without too many changes to your non-configuration code. For example, it could be painful to add [@confingy.track()][confingy.tracking.track] decorators to all of your classes, so you can instead use its functional form only when configuring your system:

mkdocs.yml

@@ -76,6 +76,7 @@ nav:
   - API Reference:
       - confingy.fingy: api/fingy.md
       - confingy.tracking: api/tracking.md
+      - confingy.tree: api/tree.md
       - confingy.serde: api/serde.md
       - confingy.exceptions: api/exceptions.md
       - confingy.mypy_plugin: api/mypy_plugin.md
@@ -85,6 +86,7 @@ nav:
           - confingy.cli.transpile: api/cli/transpile.md
           - confingy.cli.viz: api/cli/viz.md
       - Utils:
+          - confingy.utils.containers: api/utils/containers.md
           - confingy.utils.hashing: api/utils/hashing.md
           - confingy.utils.imports: api/utils/imports.md
           - confingy.utils.types: api/utils/types.md

src/confingy/__init__.py

@@ -32,6 +32,13 @@
     track,
     update,
 )
+from confingy.tree import (
+    get_init_args,
+    is_fingy_of,
+    map_fingy,
+    replace_args,
+    walk_fingy,
+)
 
 # Set up logging for better debugging
 logger = logging.getLogger(__name__)
@@ -44,6 +51,12 @@
     "track",
     "update",
     "disable_validation",
+    # Tree utilities
+    "get_init_args",
+    "is_fingy_of",
+    "map_fingy",
+    "replace_args",
+    "walk_fingy",
     # Type hints and Core classes
     "Lazy",
     "MaybeLazy",

src/confingy/tracking.py

@@ -24,6 +24,7 @@
 from confingy.exceptions import (
     ValidationError,
 )
+from confingy.utils.containers import same_structure
 from confingy.utils.hashing import hash_class
 from confingy.utils.imports import get_module_name
 from confingy.utils.types import is_lazy_instance, is_tracked_instance
@@ -158,6 +159,7 @@ def _build_validation_model(cls: type[Any]) -> type[BaseModel]:
     hints = get_type_hints(cls.__init__, include_extras=False)
 
     fields: dict[str, tuple[Any, Any]] = {}
+    accepts_var_keyword = False
     for param_name, param in init_signature.parameters.items():
         if param_name == "self":
             continue
@@ -170,14 +172,20 @@ def _build_validation_model(cls: type[Any]) -> type[BaseModel]:
             fields[param_name] = (param_type, param.default)
         elif param.kind == inspect.Parameter.VAR_KEYWORD:
             # **kwargs
+            accepts_var_keyword = True
             fields[param_name] = (param_type, {})
         else:
             fields[param_name] = (param_type, ...)  # Required field
 
     # Create the model, suppressing pydantic warnings about field names that
     # shadow BaseModel attributes (e.g. "schema", "validate", "copy").
     # These validation models are ephemeral and the shadowing is harmless.
-    model_config: ConfigDict = {"arbitrary_types_allowed": True}
+    # Unknown arguments are rejected unless __init__ accepts **kwargs, so a typo
+    # fails when a Lazy is created rather than at instantiate().
+    model_config: ConfigDict = {
+        "arbitrary_types_allowed": True,
+        "extra": "allow" if accepts_var_keyword else "forbid",
+    }
     with warnings.catch_warnings():
         warnings.filterwarnings(
             "ignore",
@@ -503,6 +511,10 @@ def unlens(self) -> Any:
         This enables a round-trip: `lens(obj) -> modify -> unlens()` preserves
         whether each node was originally Lazy or instantiated.
 
+        Only modified nodes and their ancestors are rebuilt: a node that was not
+        changed after `lens()` is returned as the original object. A node shared
+        by several parents is rebuilt once and stays shared.
+
         Returns:
             Either an instantiated object or a new Lazy, depending on how
             this Lazy was created.
@@ -522,14 +534,21 @@ def unlens(self) -> Any:
             new_lazy = l.unlens()  # Returns Lazy[Outer]
             ```
         """
+        return self._unlens({})
+
+    def _unlens(self, memo: dict[int, Any]) -> Any:
+        """Implementation of `unlens()`, memoised by id so shared nodes stay shared."""
+        if id(self) in memo:
+            return memo[id(self)]
+
         from confingy.serde import HandlerRegistry
 
         handlers = HandlerRegistry.get_default_handlers()
 
         def unlens_value(value: Any) -> Any:
             """Recursively unlens a value, using handlers for containers."""
             if is_lazy_instance(value):
-                return value.unlens()
+                return value._unlens(memo)
 
             # Use handlers for container types
             for handler in handlers:
@@ -541,17 +560,30 @@ def unlens_value(value: Any) -> Any:
         # Process all config values
         realized_config = {k: unlens_value(v) for k, v in self._confingy_config.items()}
 
-        if self._confingy_was_instantiated:
+        # Reuse the object this Lazy was lensed from if nothing changed
+        source = self.__dict__.get("_confingy_source")
+        source_config = None
+        if is_lazy_instance(source):
+            source_config = source._confingy_config
+        elif is_tracked_instance(source):
+            source_config = source._tracked_info["init_args"]
+
+        result: Any
+        if source_config is not None and _same_config(realized_config, source_config):
+            result = source
+        elif self._confingy_was_instantiated:
             # This Lazy was created from a tracked instance - instantiate
-            return self._confingy_actual_cls(**realized_config)
+            result = self._confingy_actual_cls(**realized_config)
         else:
             # This was originally a Lazy - return new Lazy
             # Always skip the post-config hook since unlens() is a structural
             # transformation, not a semantic creation. Hooks already ran on
             # setattr when values were modified.
-            return Lazy(
+            result = Lazy(
                 self._confingy_cls, realized_config, _skip_post_config_hook=True
             )
+        memo[id(self)] = result
+        return result
 
     def __call__(self, *args: Any, **kwargs: Any) -> "Lazy[T]":
         """Make Lazy callable to support the lazy(Class)(...) pattern.
@@ -720,18 +752,31 @@ def __init__(self, value: int):
         l.middle.inner.value = 100
         new_lazy = l.unlens()  # Returns Lazy since original was Lazy
         ```
+
+    An object shared by several parents is lensed once, so editing it through one
+    parent is visible through the others, and it stays shared after `unlens()`.
     """
+    if not (is_lazy_instance(obj) or is_tracked_instance(obj)):
+        raise TypeError(
+            f"lens() requires a Lazy or tracked instance, got {type(obj).__name__}"
+        )
+    return _lens(obj, {})
+
+
+def _lens(obj: Any, memo: dict[int, Any]) -> Any:
+    """Implementation of `lens()`, memoised by id so shared nodes stay shared."""
+    if id(obj) in memo:
+        return memo[id(obj)]
+
     from confingy.serde import HandlerRegistry
 
     handlers = HandlerRegistry.get_default_handlers()
 
     def lens_value(value: Any) -> Any:
         """Recursively convert tracked instances to Lazy."""
-        if is_tracked_instance(value):
-            return lens(value)
-        if is_lazy_instance(value):
-            # Recurse into Lazy config in case it contains tracked instances
-            return lens(value)
+        if is_tracked_instance(value) or is_lazy_instance(value):
+            # Recurse into Lazy config too, in case it contains tracked instances
+            return _lens(value, memo)
 
         # Use handlers for container types
         for handler in handlers:
@@ -740,35 +785,44 @@ def lens_value(value: Any) -> Any:
 
         return value
 
+    result: Any
     if is_lazy_instance(obj):
         # Recurse into config to convert any nested tracked instances
         new_config = {k: lens_value(v) for k, v in obj._confingy_config.items()}
 
         # Only create new Lazy if config actually changed
         if any(new_config[k] is not obj._confingy_config[k] for k in new_config):
-            return Lazy(
+            result = Lazy(
                 obj._confingy_cls,
                 new_config,
                 skip_validation=True,  # Config comes from valid source
                 _was_instantiated=getattr(obj, "_confingy_was_instantiated", False),
                 _skip_post_config_hook=True,  # lens() is just wrapping, don't run hooks
             )
-        return obj
-
-    if is_tracked_instance(obj):
+            result._confingy_source = obj
+        else:
+            result = obj
+    else:
         # Convert tracked instance to Lazy with _was_instantiated=True
         # Skip validation since the tracked instance was already valid
         config = {k: lens_value(v) for k, v in obj._tracked_info["init_args"].items()}
-        return Lazy(
+        result = Lazy(
             type(obj),
             config,
             skip_validation=True,
             _was_instantiated=True,
             _skip_post_config_hook=True,  # lens() is just wrapping, don't run hooks
         )
+        result._confingy_source = obj
+
+    memo[id(obj)] = result
+    return result
+
 
-    raise TypeError(
-        f"lens() requires a Lazy or tracked instance, got {type(obj).__name__}"
+def _same_config(config: dict[str, Any], source_config: dict[str, Any]) -> bool:
+    """Check whether a realized config still matches the config it was lensed from."""
+    return config.keys() == source_config.keys() and all(
+        same_structure(config[k], source_config[k]) for k in config
     )
 
 
@@ -923,10 +977,20 @@ def init_with_tracking(self: Any, *args: Any, **kwargs: Any) -> None:
             )
 
             if _validate and validation_model is not None:
+                to_validate = init_kwargs
+                if type(self).__init__ is not init_with_tracking:
+                    # An untracked subclass overrode __init__, so init_kwargs follow
+                    # its signature and may include arguments cls doesn't declare.
+                    # Only check the ones cls does.
+                    to_validate = {
+                        k: v
+                        for k, v in init_kwargs.items()
+                        if k in validation_model.model_fields
+                    }
                 try:
                     # Validate but keep original objects instead of converting to dict
                     validation_model(
-                        **init_kwargs
+                        **to_validate
                     )  # Just validate, don't use the result
                     # Store the original init_kwargs, not the dumped version
                     stored_kwargs = init_kwargs
@@ -1075,6 +1139,9 @@ def _add_tracking_to_instance(instance: Any) -> Any:
         "module": get_module_name(cls),
         "init_args": init_kwargs,
         "class_hash": hash_class(cls),
+        # init_args holds attributes, not constructor arguments, so this
+        # instance can't be rebuilt from them
+        "from_instance": True,
     }
 
     return instance