Skip to content

docs(profile): explain why AI participates in judgment - #1780

Open
seonghobae wants to merge 1 commit into
mainfrom
docs/grove-mechanical-prediction-rationale-20260903
Open

docs(profile): explain why AI participates in judgment#1780
seonghobae wants to merge 1 commit into
mainfrom
docs/grove-mechanical-prediction-rationale-20260903

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Grounds the public Contextual Wisdom Lab profile in Grove et al. (2000), Clinical versus mechanical prediction: A meta-analysis.
  • Explains the relevant claim precisely: repeated judgments benefit from explicit, testable, reproducible mechanical procedures; this is not a claim that modern LLMs are universally superior to people.
  • Makes the operational boundary explicit: traceable evidence and lineage, versioned criteria, uncertainty and abstention, cost-sensitive false-positive/false-negative evaluation, governed approval, monitoring, and review or appeal.
  • Adds the full APA reference and DOI to the selected research background.

Evidence boundary

Grove et al. compared clinical judgment with formal statistical, actuarial, and algorithmic prediction across 136 included studies. Mechanical methods were about 10% more accurate on average, substantially outperformed clinical judgment in 33%–47% of studies, and were substantially worse in 6%–16%. The paper supports a general preference for an appropriate mechanical algorithm where available, while also requiring attention to decision and error costs.

This PR does not treat the paper as evidence that generative AI or LLMs are generally superior, nor does it transfer unbounded decision authority to a model. Model output remains evidence until an explicitly governed process grants authority.

Scope and verification

  • Documentation only: profile/README.md.
  • Exact base: 64e67efb4c0414a7db114f67b2a8ea77c6159a6f.
  • Exact head: 595c6ebf762378f449e06290763cf76534c2fd6a.
  • Fresh comparison at creation: one commit ahead, zero behind; one file changed, 18 additions, no deletions.
  • No product/runtime behavior, workflow, permissions, security control, or source-of-truth ownership changes.

Reference: https://doi.org/10.1037/1040-3590.12.1.19


Devin Review

@coderabbitai

coderabbitai Bot commented Sep 3, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 43 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: c7ed4179-f145-4815-8672-4fa24763000e

📥 Commits

Reviewing files that changed from the base of the PR and between 64e67ef and 595c6eb.

📒 Files selected for processing (1)
  • profile/README.md

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@seonghobae
seonghobae enabled auto-merge (squash) September 3, 2026 05:44
@opencode-agent
opencode-agent Bot disabled auto-merge September 3, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant