Skip to content

[Fellowship Challenge] Expiring memory (ttl.v1) for the memory layer - #295

Open
Prathmesh010190 wants to merge 2 commits into
projnanda:mainfrom
Prathmesh010190:challenge/prathmesh010190-expiring-memory
Open

Prathmesh010190 wants to merge 2 commits into
projnanda:mainfrom
Prathmesh010190:challenge/prathmesh010190-expiring-memory

Conversation

@Prathmesh010190

Copy link
Copy Markdown

Submission for the Hexaware x Project NANDA Fellowship Technical Challenge

Summary

This PR adds expiring memory to the memory layer (layer 10): memory: ttl.v1, a plugin that stops serving a remembered entry once its time limit has passed. It comes with a negative-control plugin (ttl_off.v1), a new Lab scenario (marketplace_expiring_memory), a validator for it, and tests.

The gap

The default memory plugin, kv.v1, never forgets. In the marketplace scenario, the buyer remembers preferred_seller after round one, and in round two it quotes only that seller instead of looking up sellers again through the registry. Nothing in the layer can mark a remembered fact as stale, so an agent can keep acting on old knowledge for as long as the run lasts.

What I added (layer: memory)

  • ttl.v1 (src/nandatown/layers/memory_ttl.py): a subclass of KeyValueMemory.
    • remember stores the value as kv.v1 does. If the agent's scenario config declares a limit for that key (memory_ttl: {preferred_seller: 0.5}), it also stores expires_at = engine.now + limit and puts expires_at in the memory_written event.
    • recall returns the value as before if the key has no limit. If a limit exists and engine.now >= expires_at, it deletes the entry, emits memory_expired (key and expires_at), and returns None. Otherwise it emits memory_recalled and returns the value.
  • ttl_off.v1 (same file): the negative control. It records expires_at and emits memory_recalled with "stale": true, but still returns the expired entry.
  • marketplace_expiring_memory (src/nandatown/sim/scenarios/): a copy of the marketplace scenario with memory: ttl.v1 and memory_ttl: {preferred_seller: 0.5} on buyer-1. The duplicate-delivery fault stays in the scenario.
  • Validator for that scenario (src/nandatown/sim/validators.py), a label for the new memory_expiry stage (report.py), tests (tests/test_memory_ttl.py), and the scenario name added to ALL_SCENARIOS in tests/test_sim.py.
  • The plugin is registered by adding memory_ttl to the import list in src/nandatown/layers/__init__.py.

I did not change sim/agents.py or sim/api.py.

Design decisions

  • Absolute deadline in logical time. The plugin compares expires_at with the engine's logical clock (engine.now), not wall-clock time, so the run stays seeded and verify can reproduce it.
  • Events carry keys and times only, never values. The memory_expired event shows only key and expires_at.
  • Only declared keys expire. A key with no entry in memory_ttl behaves exactly like kv.v1.
  • The limit is read from the agent's scenario config. Lab memory plugins are constructed with only the engine and get no settings, so the plugin reads memory_ttl from the agent's entry in engine.spec.agents.
  • A new validator. The stock marketplace validator needs exactly 3 quote requests and a memory_reuse stage. With expiry, round two looks sellers up again, so there is a 4th request and no reuse. My validator accepts at least 3 quote requests for discovery, drops memory_reuse, and adds memory_expiry.

Fault and invariant

Fault: the remembered entry ages past its limit between rounds. preferred_seller is written at t=1.90 with a limit of 0.5 (so expires_at is about 2.40), and round two reads it at t=2.90. The duplicate-delivery fault still runs alongside it.

Invariant (memory_expiry stage):

  • It fails if any memory_recalled event has "stale": true (an entry past its expiry was served).
  • It reports missing evidence if no memory_expired event exists, because then the rule was never exercised.
  • It passes when an expired entry was dropped and never recalled.

With ttl.v1, the read at t=2.90 emits memory_expired (ev-43) and the buyer looks sellers up again. That run has 70 events and 4 quote requests. All eight stages pass.

Negative control

nandatown run marketplace_expiring_memory --layer memory=ttl_off.v1 fails. The first broken stage is memory_expiry ("the buyer recalled a memory entry after its expiry time"). The replay shows memory_recalled with "stale": true at t=2.90 as ev-43, the same event number that is memory_expired in the passing run. The failing run has 66 events and 3 quote requests, and every other stage still passes. So the failure comes from the memory rule alone, which is the reason I predicted.

Evidence

nandatown verify printed "bundle verified: hashes match and the evaluator reproduces the recorded result" on all four bundles below.

Run Verdict Bundle
ttl.v1 PASSED sim-1a4bfd88f4e1
ttl_off.v1 FAILED at memory_expiry sim-d3ba949c7a7e
ttl.v1 (later rerun) PASSED sim-ade7e723796f
ttl_off.v1 (later rerun) FAILED at memory_expiry sim-d883966bd119

The first pair has bundle fingerprints sha256:a0969f51...efd (passing) and sha256:a00ce6f3...fbb (failing). The rerun gave the same verdicts, but different bundle fingerprints. I did not investigate why.

Tests (tests/test_memory_ttl.py, 4 tests):

  1. An entry is served before its expiry and dropped after it.
  2. A key with no declared limit never expires.
  3. The marketplace run rediscovers after expiry (memory_expiry passes and a memory_expired event exists).
  4. The negative control serves the stale entry and the run fails.

Fail before, pass after: with src/nandatown/layers/__init__.py and src/nandatown/sim/validators.py reverted to origin/main, all 4 tests fail. With my versions restored, all 4 pass.

Full suite on this branch: 1760 passed, 1 skipped, 0 failed.

How to reproduce

nandatown run marketplace_expiring_memory
nandatown run marketplace_expiring_memory --layer memory=ttl_off.v1
python -m pytest tests/test_memory_ttl.py -q
nandatown verify runs/<folder printed by each run>
nandatown replay runs/<passing folder> --kind memory_expired
nandatown replay runs/<failing folder> --kind memory_recalled

The runs/ folder is not committed. I can attach the bundles if reviewers want them.

What this does not prove

  • No better outcome. The buyer buys from seller-b either way, so expiry does not change who gets paid. The evidence shows that an expired entry is never served and the buyer looks sellers up again.
  • Not a crash and restart. Lab memory is an in-process dict. The fault is time passing, and the 0.5 limit makes it happen on purpose.
  • One seeded run. This is one scoped observation, not a campaign.
  • Lab only. I tested this only in the Lab.
  • A looser discovery check. My validator accepts at least 3 quote requests, where the stock one requires exactly 3.

What I would do next

  • Let the engine pass settings to memory plugins, so a TTL does not have to ride on an agent's config.
  • Add a Lab restart fault, so durable memory and its expiry can be tested across a crash.
  • Try the same rule on the Track, where participants already keep a journal.
  • Let shared facts carry a validity window, so independent agents agree on when something is stale.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant