-
Notifications
You must be signed in to change notification settings - Fork 1
feat(noema-agent): add calendar conflict-check tool #1486
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
seonghobae
wants to merge
84
commits into
develop
Choose a base branch
from
claude/noema-contextualwisdomlab-commercialization-afow1j
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 81 commits
Commits
Show all changes
84 commits
Select commit
Hold shift + click to select a range
a4e97cb
feat(noema-agent): add calendar conflict-check tool
claude 206655c
Merge remote-tracking branch 'origin/develop' into claude/noema-conte…
claude 6a5365e
fix(noema-agent): fail closed on oversized existing-commitment batches
claude dff8550
feat(calendar): persist conflict judgments and human corrections (G-06)
claude 7c20155
fix(calendar): address Devin review findings on the judgment/correcti…
claude 86f4bd9
fix(calendar): scope judgments by workspace_id, fix 5 more Devin find…
claude e3e09ad
fix(calendar): no-op overrides preserve original reason; update stale…
claude a5cebe5
docs: fix self-contradiction in the just-updated doctoring doc
claude ee83eff
feat(attachments): quarantine content-type mismatches, add attachment…
claude dcc9fcd
fix(attachments,calendar): OOXML false-positive quarantine, unprocess…
claude ba9a01b
feat(attachments): add AttachmentReparseWorker to consume reparse_pen…
claude 522d422
fix(attachments,calendar): Devin review round on AttachmentReparseWorker
claude ef49fc9
fix(attachments): correct cursor starvation and advisory-lock connect…
claude da81656
fix(migrations,calendar): correct downgrade constraint drop and add o…
claude 733f22c
fix(newsdom-worker): dedicated lease connection + starvation-free cur…
claude d93cc8c
fix(workers): set AUTOCOMMIT on the dedicated advisory-lease connection
claude 603ff17
fix(security): scope Email/Attachment queries by workspace_id, not ju…
claude fae8dbc
docs(security): verify workspace-claim trust boundary raised by review
claude af9ed1e
fix(security): enforce workspace claim matches org at auth layer
claude 6cd3892
fix: add missing bootstrap_db columns and stop reparse from deleting …
claude ffed35e
fix: populate workspace_id in the ZIP-archive fixture importer's bulk…
claude b778fb6
fix: enforce workspace-safe Noema identity
seonghobae 6df8f44
fix: match email upsert/bootstrap uniqueness to the workspace-scoped …
claude 62b74a0
fix: migration 0020 also recognizes bootstrap's owner-only email index
claude d05e2a6
fix: enforce workspace identity across mail flows
seonghobae 25e8e60
Merge commit '62b74a0532cbd2f7374ccc3c6febbb77846367dc' into codex/pr…
seonghobae 9c49bd2
docs(adr-0005): correct stale auth workspace-claim rejection description
claude 2c0fe37
test: align bootstrap identity regression
seonghobae 2a6c8a5
test: accept early legacy index cleanup
seonghobae 94f02eb
fix: create workspace identity after bootstrap backfill
seonghobae b355ec3
Merge commit '9c49bd29af162b8fea88f136b89744f4154f3d85' into codex/pr…
seonghobae 611c9b5
fix: migration 0020 must check constraint before dropping backing index
claude e11cb07
Merge remote-tracking branch 'origin/claude/noema-contextualwisdomlab…
claude c3e2856
fix(pop3): resolve workspace from existing mail instead of nonexisten…
claude 399c1e5
fix(fixtures): store imported email under the configured workspace id
claude c6085ef
fix(email-import): scope owner quota count to owner, not workspace
claude 1b85703
fix(calendar): rename correction rationale column to two-word name
claude a3b5f8f
fix(db): preserve calendar rationale migration path
seonghobae a1027af
style(tests): replace unnecessary lambda with named function
claude 786d154
fix(import): scope duplicate lookup and graph projection to workspace
claude 968b21f
fix(search): exclude non-parsed attachments from hybrid search
claude 96cd0c0
fix(db): skip absent legacy email table during bootstrap
seonghobae d3422db
fix(tests): add missing skip-on-no-postgres guard to two new smoke tests
claude b9b02dd
fix(db): stop 0001's fresh-install migration from crashing, fix hidde…
claude c1f02e2
fix(newsdom): stop pending-result starvation; fix(data-api): lock att…
claude 780d910
fix(attachments): index reparsed attachment content into the content …
claude 5096d1f
fix(attachments): preload graph relationships for reparse
seonghobae beded49
fix(attachments): persist reparsed graph topology and vectors
seonghobae f63a109
fix(attachments): chunk reparsed embedding sources
seonghobae 41ae6a2
fix(db): skip absent legacy email read-state table
seonghobae a4e0119
fix(ci): treat draft governance as waiting
seonghobae bd4b5ae
Merge remote-tracking branch 'refs/remotes/origin/claude/noema-contex…
seonghobae 86074f6
test(attachments): skip the persisted-reparse postgres smoke cleanly …
claude 4b4b1cb
Merge remote-tracking branch into HEAD
claude 51245f7
test: fail reparse smoke on postgres defects
seonghobae db97962
fix(ci): wire repo-root governance contract tests into CI; fix 2 brok…
claude de11149
fix(db): make 0011_email_read_state's legacy-table guard offline-safe
claude 3cbbba8
merge: refresh attachment reparse stack base
seonghobae 3a2246d
Merge remote-tracking branch 'origin/claude/attachment-reparse-conten…
seonghobae ede7f4b
fix(ci): parse app-ci.yml as YAML in the CI-wiring regression test; s…
claude 87ef2e5
fix(db): defer 0011's legacy-table check to SQL, not Python
claude d78655a
Merge remote-tracking branch into HEAD
claude 8e47575
fix(ci): stop the root governance test step from crashing under PYTHO…
claude 09a2443
fix(db): resolve 0011's legacy-table check through search_path, not r…
claude c249096
docs(db): document 0011's op.execute as a deliberate structured-ops e…
claude 26e685e
merge: refresh reparse stack and narrow postgres skip
seonghobae d244ccc
fix(attachments): resolve reparse embedding source and migration down…
claude f316b2d
merge: bring in base-branch CI-wiring fixes merged onto this branch a…
claude 212acb9
fix(ci): add repo-root pytest.ini to stop the root-tests step crashing
claude f3af149
merge: bring in base-branch changes to resolve mergeable_state DIRTY
claude 6294b8b
fix(migrations): suppress bandit B608 false positive on 0011 DDL cons…
claude 0345eda
test(alembic): narrow the connectivity-probe exception handler in the…
claude 5cb5e49
fix(noema-agent): route LLM calls through contextual-orchestrator, no…
claude ba1b9cc
docs(noema-agent): correct 'two separate agents, share only a name' f…
claude 34c71d2
fix(newsdom-worker,attachment-reparse-worker): stop a stuck row from …
claude 66f4f0f
docs(noema-agent): stop citing an ADR file that doesn't exist on this…
claude 836c274
fix(newsdom-worker,tenant-config): wire the Noema gateway config and …
claude b8bccf7
fix(newsdom-worker,attachment-reparse-worker): rediscover rows revert…
claude cdcf2da
Merge remote-tracking branch 'origin/claude/noema-contextualwisdomlab…
claude d7e5d2d
fix(test): add missing refresh() to _LiveReparsePendingSession fake
claude ff8807a
fix(attachments): index reparsed attachment content into the content …
seonghobae b32954d
fix(migrations): suppress Semgrep false positive on 0011_email_read_s…
seonghobae 7ce6592
fix(tasks): preserve scheduler lease across transactions
seonghobae 1709ebb
test(tasks): preserve database import boundary
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,29 +1,116 @@ | ||
| """Add is_read to emails (IMAP \\Seen read state). | ||
|
|
||
| Existing rows default to read so historical/file imports do not surface as unread. | ||
|
|
||
| Deliberate exception to this repo's "Alembic migrations use structured | ||
| operations (``op.create_index``, ...), never ``sa.text(f"...")`` DDL" rule | ||
| (``AGENTS.md``/``CLAUDE.md``): ``upgrade()``/``downgrade()`` below use | ||
| ``op.execute()`` with the module-level ``_UPGRADE_SQL``/``_DOWNGRADE_SQL`` | ||
| constants instead of a structured ``op.*`` call. That rule's actual target is | ||
| DDL built from interpolated identifier strings (an injection-safety concern); | ||
| these constants interpolate only ``_IS_READ_PROVENANCE_MARKER``, a fixed | ||
| module-level literal, never an identifier or a value built from a variable, | ||
| external input, or runtime state -- the same safety property a structured | ||
| call would have. The reason a structured call isn't used is different: this | ||
| migration's behavior must be conditional on whether the legacy ``emails`` | ||
| table exists, evaluated at apply time (see the comment on ``_UPGRADE_SQL`` | ||
| below for why that check cannot live in Python), and no structured Alembic | ||
| operation expresses "run this DDL only if a runtime condition holds" -- a | ||
| ``DO $$ ... $$`` block is the correct primitive for that, not a workaround | ||
| for one. | ||
| """ | ||
|
|
||
| from alembic import op | ||
| import sqlalchemy as sa | ||
|
|
||
| # revision identifiers, used by Alembic. | ||
| revision = "0011_email_read_state" | ||
| down_revision = "0009_project_graph_projection" | ||
| branch_labels = None | ||
| depends_on = None | ||
|
|
||
| # Fresh installations materialize the current ``email_records`` model in the | ||
| # 0001 baseline, including ``is_read``. This historical side branch only | ||
| # applies to databases that still carry its legacy ``emails`` table. | ||
| # | ||
| # The condition has to be evaluated in SQL, not Python: offline SQL | ||
| # generation (``alembic upgrade --sql``, a real flag ``scripts/migrate_db.py`` | ||
| # exposes) has no live connection to introspect with and no specific target | ||
| # database to ask "does this legacy table exist" at generation time either -- | ||
| # the same static script is meant to later be applied by a DBA against | ||
| # whichever database they choose, fresh-install or legacy. A Python-side | ||
| # check (``sa.inspect(op.get_bind())``) can only ever answer that question | ||
| # for one hypothetical target chosen at generation time, so it is wrong for | ||
| # the other: skip unconditionally and the column silently never gets added | ||
| # for a legacy database that applies the generated script (while | ||
| # ``alembic_version`` still advances, permanently hiding the gap); inspect | ||
| # online and bake in one fixed answer and the same script fails outright | ||
| # against the other kind of target. A ``DO $$ ... $$`` block defers the | ||
| # check to apply time instead, so the one generated script is correct | ||
| # against either kind of target, online or offline-then-applied-later alike. | ||
| # | ||
| # ``to_regclass('emails')`` (not ``information_schema.tables`` by bare | ||
| # ``table_name``) deliberately: the unqualified ``ALTER TABLE emails`` below | ||
| # resolves through the connection's ``search_path``, and ``to_regclass`` | ||
| # resolves an unqualified name exactly the same way, returning NULL if it | ||
| # doesn't. ``information_schema.tables`` filtered only by ``table_name`` | ||
| # ignores ``search_path`` entirely and matches a same-named table in *any* | ||
| # schema the connecting role can see -- on a deployment with more than one | ||
| # accessible schema, that could find an unrelated ``emails`` table outside | ||
| # the search path while the unqualified ``ALTER TABLE emails`` targets a | ||
| # different (or no) table, passing the guard for the wrong relation or | ||
| # aborting the migration outright. Resolving both the check and the DDL | ||
| # through the same name lookup makes that mismatch structurally impossible. | ||
| # | ||
| # ``COMMENT ON COLUMN emails.is_read`` tags the column with a provenance | ||
| # marker (``_IS_READ_PROVENANCE_MARKER``) the moment upgrade() actually adds | ||
| # it. downgrade() only drops the column when that exact marker is present | ||
| # (CodeRabbit, naruon#1501): an ``emails.is_read`` column that already | ||
| # existed before this revision ran -- from some other, unrelated origin -- | ||
| # would upgrade()'s ``NOT EXISTS`` guard correctly leave alone, but an | ||
| # unconditional ``DROP COLUMN IF EXISTS`` on downgrade would still destroy it | ||
| # and its data, since a downgrade has no other way to tell "I added this" | ||
| # apart from "this happens to be present". Checking the marker via | ||
| # ``col_description`` makes downgrade drop only what this exact revision's | ||
| # upgrade created. | ||
| _IS_READ_PROVENANCE_MARKER = "0011_email_read_state:added" | ||
| _UPGRADE_SQL = f""" | ||
| DO $$ | ||
| BEGIN | ||
| IF to_regclass('emails') IS NOT NULL AND NOT EXISTS ( | ||
| SELECT 1 FROM pg_attribute | ||
| WHERE attrelid = to_regclass('emails') | ||
| AND attname = 'is_read' | ||
| AND NOT attisdropped | ||
| ) THEN | ||
| ALTER TABLE emails ADD COLUMN is_read boolean NOT NULL DEFAULT true; | ||
| COMMENT ON COLUMN emails.is_read IS '{_IS_READ_PROVENANCE_MARKER}'; | ||
| END IF; | ||
| END $$; | ||
| """ # nosec B608 | ||
|
|
||
| _DOWNGRADE_SQL = f""" | ||
| DO $$ | ||
| BEGIN | ||
| IF to_regclass('emails') IS NOT NULL AND EXISTS ( | ||
| SELECT 1 FROM pg_attribute | ||
| WHERE attrelid = to_regclass('emails') | ||
| AND attname = 'is_read' | ||
| AND NOT attisdropped | ||
| ) AND col_description(to_regclass('emails'), ( | ||
| SELECT attnum FROM pg_attribute | ||
| WHERE attrelid = to_regclass('emails') | ||
| AND attname = 'is_read' | ||
| AND NOT attisdropped | ||
| )) = '{_IS_READ_PROVENANCE_MARKER}' THEN | ||
| ALTER TABLE emails DROP COLUMN IF EXISTS is_read; | ||
| END IF; | ||
| END $$; | ||
| """ # nosec B608 | ||
|
|
||
|
seonghobae marked this conversation as resolved.
|
||
|
|
||
| def upgrade() -> None: | ||
| op.add_column( | ||
| "emails", | ||
| sa.Column( | ||
| "is_read", | ||
| sa.Boolean(), | ||
| nullable=False, | ||
| server_default=sa.text("true"), | ||
| ), | ||
| ) | ||
| op.execute(_UPGRADE_SQL) | ||
|
github-advanced-security[bot] marked this conversation as resolved.
Fixed
github-advanced-security[bot] marked this conversation as resolved.
Fixed
|
||
|
|
||
|
|
||
| def downgrade() -> None: | ||
| op.drop_column("emails", "is_read") | ||
| op.execute(_DOWNGRADE_SQL) | ||
|
github-advanced-security[bot] marked this conversation as resolved.
Fixed
github-advanced-security[bot] marked this conversation as resolved.
Fixed
|
||
109 changes: 109 additions & 0 deletions
109
backend/alembic/versions/0018_calendar_conflict_judgments.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| """add calendar conflict judgments and corrections | ||
|
|
||
| Revision ID: 0018_calendar_conflict_judgments | ||
| Revises: 0017_merge_newsdom_carddav_heads | ||
| Create Date: 2026-08-30 00:00:00.000000 | ||
| """ | ||
|
|
||
| from alembic import op | ||
| import sqlalchemy as sa | ||
|
|
||
| revision = "0018_calendar_conflict_judgments" | ||
| down_revision = "0017_merge_newsdom_carddav_heads" | ||
|
|
||
| _JUDGMENT_TABLE = "calendar_conflict_judgments" | ||
| _CORRECTION_TABLE = "calendar_conflict_corrections" | ||
|
|
||
|
|
||
| def upgrade() -> None: | ||
| connection = op.get_bind() | ||
| inspector = sa.inspect(connection) | ||
|
|
||
| if not inspector.has_table(_JUDGMENT_TABLE): | ||
| op.create_table( | ||
|
seonghobae marked this conversation as resolved.
|
||
| _JUDGMENT_TABLE, | ||
| sa.Column("calendar_conflict_judgment_id", sa.Integer(), nullable=False), | ||
| sa.Column("judgment_uid", sa.String(length=96), nullable=False), | ||
| sa.Column("user_id", sa.String(), nullable=False), | ||
| sa.Column("organization_id", sa.String(), nullable=True), | ||
| sa.Column("workspace_id", sa.String(), nullable=False), | ||
| sa.Column("proposed_commitment_id", sa.String(length=256), nullable=False), | ||
| sa.Column("source_thread_id", sa.String(), nullable=True), | ||
| sa.Column("source_message_id", sa.String(), nullable=True), | ||
| sa.Column("decision_code", sa.String(length=32), nullable=False), | ||
| sa.Column("reason_code", sa.String(length=64), nullable=False), | ||
| sa.Column("recommended_action", sa.Text(), nullable=False), | ||
| sa.Column("policy_version", sa.String(length=32), nullable=False), | ||
| sa.Column("conflicts_json", sa.JSON(), nullable=False), | ||
| sa.Column("status_code", sa.String(length=32), nullable=False), | ||
| sa.Column("created_at", sa.DateTime(timezone=True), nullable=False), | ||
| sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False), | ||
| sa.PrimaryKeyConstraint("calendar_conflict_judgment_id"), | ||
| sa.UniqueConstraint( | ||
| "judgment_uid", name="uq_calendar_conflict_judgments_uid" | ||
| ), | ||
| ) | ||
|
|
||
| if not inspector.has_table(_CORRECTION_TABLE): | ||
| op.create_table( | ||
| _CORRECTION_TABLE, | ||
| sa.Column("calendar_conflict_correction_id", sa.Integer(), nullable=False), | ||
| sa.Column("correction_uid", sa.String(length=96), nullable=False), | ||
| sa.Column("calendar_conflict_judgment_id", sa.Integer(), nullable=False), | ||
| sa.Column("user_id", sa.String(), nullable=False), | ||
| sa.Column("organization_id", sa.String(), nullable=True), | ||
| sa.Column("workspace_id", sa.String(), nullable=False), | ||
| sa.Column("actor_user_id", sa.String(), nullable=False), | ||
| sa.Column("correction_action", sa.String(length=64), nullable=False), | ||
| sa.Column("before_json", sa.JSON(), nullable=False), | ||
| sa.Column("after_json", sa.JSON(), nullable=False), | ||
| sa.Column("rationale", sa.Text(), nullable=True), | ||
|
seonghobae marked this conversation as resolved.
|
||
| sa.Column("created_at", sa.DateTime(timezone=True), nullable=False), | ||
| sa.ForeignKeyConstraint( | ||
| ["calendar_conflict_judgment_id"], | ||
| ["calendar_conflict_judgments.calendar_conflict_judgment_id"], | ||
| ), | ||
| sa.PrimaryKeyConstraint("calendar_conflict_correction_id"), | ||
| sa.UniqueConstraint( | ||
| "correction_uid", name="uq_calendar_conflict_corrections_uid" | ||
| ), | ||
| ) | ||
|
|
||
| for table_name, indexes in _calendar_conflict_indexes().items(): | ||
| for index_name, column_names in indexes: | ||
| op.create_index( | ||
| index_name, | ||
| table_name, | ||
| column_names, | ||
| if_not_exists=True, | ||
| ) | ||
|
|
||
|
|
||
| def downgrade() -> None: | ||
| connection = op.get_bind() | ||
| inspector = sa.inspect(connection) | ||
|
|
||
| for table_name in (_CORRECTION_TABLE, _JUDGMENT_TABLE): | ||
| if inspector.has_table(table_name): | ||
| for index_name, _column_names in reversed( | ||
| _calendar_conflict_indexes()[table_name] | ||
| ): | ||
| op.drop_index(index_name, table_name=table_name, if_exists=True) | ||
| op.drop_table(table_name) | ||
|
|
||
|
|
||
| def _calendar_conflict_indexes() -> dict[str, list[tuple[str, list[str]]]]: | ||
| return { | ||
| _JUDGMENT_TABLE: [ | ||
| ( | ||
| "ix_calendar_conflict_judgments_scope_thread", | ||
| ["user_id", "organization_id", "workspace_id", "source_thread_id"], | ||
| ), | ||
| ], | ||
| _CORRECTION_TABLE: [ | ||
| ( | ||
| "ix_calendar_conflict_corrections_judgment", | ||
| ["calendar_conflict_judgment_id"], | ||
| ), | ||
| ], | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.