[BUG] roll back state when set_params fails validation - #589
[BUG] roll back state when set_params fails validation#589OfficialAbhinavSingh wants to merge 2 commits into
Conversation
set_params writes parameter values to self via setattr before calling reset(), which is what re-runs __init__ to validate them. If __init__ rejects a value, the raise happens after the write, so the rejected value survives on self -- a state __init__ could never have produced. get_params, clone, and later set_params calls then operate on that invalid state. Snapshot self.__dict__ before the write loop in set_params and restore it if the loop or the following reset() call raises. Adds a regression test with a fixture that actually validates in __init__, since the existing test_get_params_after_set_params never exercised the raise path (its fixture class never raises). Fixes sktime/sktime#10695
fkiraly
left a comment
There was a problem hiding this comment.
Thanks for the contribution. I think this is potentially something to add, but the current implementation has major issues.
- storing the current state can multiply memory requirements. Imagine expensive model weights, which have to exist in two copies.
- I think "silent failure" is the wrong handling of this case. I think there should be an explicit exception at the end.
Further:
- I to not think the entire code needs to be in the try/except loop - only
reset. - a clear exception message is missing, the user will not know what has happened
|
Thanks for the review, taking the points in order. Only Memory — the snapshot is "Silent failure" / missing message — the previous version did re-raise (bare chained One point for your call: Rationale for rollback vs message-only is in sktime#10695. |
Reference Issues/PRs
Fixes sktime/sktime#10695. See also sktime/sktime#10821, an earlier attempt at this issue that patched
set_paramson thesktimeside and was closed unmerged, since the bug is inBaseObject.set_paramshere inskbase, not insktime.What does this implement/fix? Explain your changes.
BaseObject.set_paramswrites new parameter values toselfviasetattrbefore callingself.reset(), which is what actually re-runs__init__to validate them. If__init__rejects a value, the raise happens after the write, so the rejected value survives onself-- a state__init__could never have produced.get_params,clone, and any laterset_paramscall then operate on that invalid state.Fix snapshots
self.__dict__before the parameter-writing loop inset_params, and restores it if anything in that loop or the followingself.reset()call raises, then re-raises the original exception. Confirmed against the estimator that surfaced this insktime(DilationMappingTransformer):set_params(dilation=0)now leavesdilationat its prior valid value instead of0, andclone()succeeds afterward.Does your contribution introduce a new dependency? If yes, which one?
No.
What should a reviewer concentrate their feedback on?
self.__dict__is the right mechanism, versus validating before writing toselfin the first place -- the issue write-up considered both directions and this seemed the smaller, more general change, since it doesn't require touching every__init__.test_get_params_after_set_paramstest intest_base.pyalready asserts rollback semantics under fuzz values, but the fixture class it uses (Parent) never raises in__init__, so that assertion was never actually exercised. Added a new test with a fixture that does validate,test_set_params_rolls_back_state_on_invalid_value.Any other comments?
An LLM was used as a search and navigation tool to help trace the bug to its root cause in
set_params/reset()and to draft this description. Every claim above -- the repro, the root cause, the before/after behavior, and the test counts -- was produced by running the code locally on both sides of the fix, including a run of the new test against the pre-fix source to confirm it fails without the change.PR checklist
For all contributions
the PR topic is related to enhancement, CI/CD, maintenance, documentation, or a bug.
For code contributions