Skip to content

Accept unmapped traits in long-format datasets - #311

Open
ehwenk wants to merge 2 commits into
developfrom
long-format-unmapped-traits
Open

ehwenk wants to merge 2 commits into
developfrom
long-format-unmapped-traits

Conversation

@ehwenk

@ehwenk ehwenk commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

dataset_test() failed a long-format dataset whose trait column held a trait with no traits entry. The build has always left those rows out (process_parse_data() keeps only rows whose trait is mapped), just as it leaves out a column a wide dataset doesn't map. So the only fix was a filter in custom_R_code, which had to be edited every time another trait was mapped.

This removes that check and adds the reverse one, matching wide data: every var_in in traits must be a value of the trait column, as a wide dataset's must be a column. An entry whose trait isn't in the data builds nothing, usually because the var_in is misspelt or stale.

AusTraits: of 59 long-format datasets, SPRAT_2025_2 and Wenk_2025 fail this check today, and with this change each loses exactly that failure (5 → 4, 7 → 6; the rest predate it). Another 20 filter their trait column in custom_R_code; those filters are now redundant but harmless, and can be removed in austraits.build whenever convenient.

Newly failing in AusTraits: 10 long-format datasets have traits entries for traits not in their data: ABRS_1981 (1), Kubiak_2009 (1), Maslin_2012 (2), NHNSW_2014_2 (1), NHNSW_2024 (1), NTH_2014 (1), SPRAT_2025_2 (1), WAH_2023_1 (8), WAH_2023_2 (17) and Wenk_2025 (3). Their entries need removing or their var_in correcting.

Tests: a new test adds rows for an unmapped trait to a copy of the long-format Test_2023_3: dataset_test() reports the same failures as for the unedited copy, and the build leaves the rows out. It fails on develop. A second test adds an entry for a trait not in the data and checks it is reported. Full suite: 1,790 expectations, 0 failures.

Closes #308

🤖 Generated with Claude Code

ehwenk and others added 2 commits October 23, 2026 18:15
dataset_test() failed a long-format dataset whose trait column held a
trait with no `traits` entry, though the build has always left those
rows out, as it leaves out a column a wide dataset does not map. The
workaround was a filter in custom_R_code that had to be edited whenever
a trait was mapped. The check is removed; the wide-format check that
each `var_in` is a column stays.

Closes #308

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As a wide dataset's var_in must be a column, a long dataset's must be
a value of its trait column; otherwise the entry builds nothing,
usually because the var_in is misspelt or stale (#308).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ehwenk
ehwenk requested a review from dfalster October 23, 2026 07:26
ehwenk added a commit to traitecoevo/austraits.build that referenced this pull request Oct 23, 2026
With traitecoevo/traits.build#311, `dataset_test()` no longer fails a
long-format dataset whose trait column holds traits with no `traits`
entry; the build has always left those rows out. Six datasets filtered
their trait column in `custom_R_code` only to avoid that failure. This
removes those filters.

| Dataset | Change |
|---|---|
| Bell_1995, Bignell_1997 | `custom_R_code` was only the filter; now
`.na` |
| Johnson_1980, Saunders_2024 | filter removed; the rest of the code
stays |
| Steart_2009 | the `trait %in%` condition removed; the `source_id`
filter stays |
| NHNSW_2024 | `filter(!trait_name %in% c("leaf_lamina_division"))`
removed (that trait isn't mapped) |

**Verified:** each dataset builds identical `traits`, `excluded_data`,
`contexts`, `locations` and `methods` with and without its filter, and
`dataset_test()` reports the same failures (0 for all but NHNSW_2024,
which has 1 either way).

**Kept, because they aren't redundant:**
- Judd_1996, Wirthensohn_1998 and Portelli_2023 filter out traits that
*are* mapped: removing them adds 500, 82 and 725 values. Those are
content decisions.
- Maslin_2012's last filter keeps raw `longevity` rows out, and
`longevity` is mapped.
- Hallam_1970: removing the filter builds identically, but
`dataset_test()` then fails its context check, because the unmapped rows
carry a context value (`leaf phase: intermediate`) not in `contexts`.
That check looks at every row of the data, including rows the build
drops.
- ANBG_2019's and Portelli_2023's other trait filters feed reshaping
steps.

**Merge after traitecoevo/traits.build#311**: without it, Bell_1995,
Bignell_1997, Johnson_1980, Saunders_2024 and Steart_2009 fail
`dataset_test()` on their unmapped traits.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stop test failing with unmapped long format traits

1 participant