Skip to content

mask conversion UI dialog (uint64 to bool) - #462

Open
TeunHuijben wants to merge 3 commits into
mainfrom
uint64-mask-conversion
Open

mask conversion UI dialog (uint64 to bool)#462
TeunHuijben wants to merge 3 commits into
mainfrom
uint64-mask-conversion

Conversation

@TeunHuijben

@TeunHuijben TeunHuijben commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Problem:
The geff stores our masks and bounding boxes to display the segmentation. The problem is that previously, these masks were saved as uint64. On disc, this is fine, because zarr compression can do a lot on binary masks. However, when loading the geff into memory (like we do in motile_tracker), it makes memory explode, because the binary masks are loaded as full uint64. Actually, funtracks uses the tracksdata functionality to load the geff, and tracksdata casts to bools upon loading, but the problem is the geff.load which goes through uint64 because that is the type it sees.

Tracksdata has been updated to export the masks as bool, so every future geff will have boolean masks (once tracksdata>funtracks>motile_tracker all use the latest releases). However, in the meantime, all previously saved geffs have the masks as uint64.

This PR:
When loading a geff in which the masks are saved as uint64, the UI pops up a window with the question to convert the masks to booleans. This will duplicate the geff with the masks as booleans and load that one. This is a tracksdata utility that motile_tracker uses, and the conversion is fast, because it works on chunks, so OOM is not a risk.

For small datasets, this won't be a problem, but for larger datasets (>25k nodes) on smaller laptops, the 8-fold memory increase because of uint64 is a major problem.

🔴 currently blocked until royerlab/tracksdata#319 is merged and part of the next tracksdata release

@TeunHuijben
TeunHuijben requested a review from cmalinmayor as a code owner July 24, 2026 22:18
@TeunHuijben TeunHuijben added 🌈 UI User interface only change ⚡ performance Improve speed or scalability blocked PR blocked by something else labels Jul 24, 2026
@TeunHuijben
TeunHuijben requested a review from AnniekStok July 24, 2026 22:21
@codecov-commenter

codecov-commenter commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 22.58065% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.38%. Comparing base (25f0e9d) to head (d06e006).

Files with missing lines Patch % Lines
...otile_tracker/import_export/menus/import_dialog.py 22.58% 24 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #462      +/-   ##
==========================================
- Coverage   87.74%   87.38%   -0.37%     
==========================================
  Files          57       57              
  Lines        5493     5524      +31     
==========================================
+ Hits         4820     4827       +7     
- Misses        673      697      +24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked PR blocked by something else 🌈 UI User interface only change ⚡ performance Improve speed or scalability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants