Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 20 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,20 +152,33 @@ uv run poe run
scan directory (`OMERO_SCAN_DIR` -> `/scan/inbox`) using the first user in
`config/omero/users.yml`.
Imports mirror directory hierarchy in OMERO Folders (folders map to folders,
images map to images). If `shared_group` is set in `config/omero/scan_dirs.yml`,
configured users are joined to that group by default and imported content is
visible to all group members. Set `join_shared_group: false` on restricted
users that should not see shared content. Set `import_mode: inplace` in
images map to images). Configure each real data root with its own per-root
`group` so images are not shared with all users by default. If `shared_group` is
set in `config/omero/scan_dirs.yml`, only roots without a per-root `group` use
that fallback group. Users are not joined to the shared group by default; set
`join_shared_group: true` only for users who should see fallback shared content.
Set `import_mode: inplace` in
`config/omero/scan_dirs.yml` to avoid duplicate storage by importing file
references instead of copying pixel data.
`config/omero/users.yml` is local-only and ignored by Git; commit changes to
`config/omero/users.example.yml` instead, and use `password_env` entries with
real password values in `.env`.
Scan roots may also be configured as mappings with `path` and `group`; imports
for that root run in the configured OMERO group instead of the global shared
group. Set `delete_omero_missing_files: true` to delete tracked OMERO Images
Scan root `group` values are created by `sync-users` with
`scan_group_permissions` so the import owner and intended group members can see
the data. Set `delete_omero_missing_files: true` to delete tracked OMERO Images
when their source files are no longer present after a successful source-root
scan. This only deletes OMERO records, not source files.
Set top-level `import_user: habomero` to keep imported projects owned by one
service account in OMERO.web. Per-root `group` values still control visibility;
per-root `import_user` may be used only when a root should appear under a
different OMERO owner.
With `reimport_legacy_import_state: true`, files tracked by an older owner/group
state format are rechecked and imported into the current configured location.
This repopulates the desired OMERO owner/group after config changes; it does not
delete old OMERO objects that were already imported elsewhere.
With `cleanup_obsolete_duplicate_projects: true`, a root that finishes cleanly
deletes same-named generated scan-root Projects from older placements while
keeping the currently configured Project.

- Production-style full-dataset parallel ingest with periodic rescan:

Expand Down
25 changes: 21 additions & 4 deletions config/omero/scan_dirs.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,25 @@
allow_external_paths: true
scan_directories:
- ~/mnt/bandicoot/RxRx19a
- ~/mnt/bandicoot/CFReT_subtyping_data
- path: ~/mnt/bandicoot/RxRx19a
group: rxrx19a
- path: ~/mnt/bandicoot/CFReT_subtyping_data
group: cfret_subtyping_data
- path: ~/mnt/Way_McKinsey_Cardiac_Fibrosis
group: way_mckinsey_cardiac_fibrosis

# Optional shared OMERO group for imports. All configured users are joined
# to this group by sync-users, and imports use this group context.
# Optional fallback OMERO group for scan roots that do not define a per-root
# group. Users are not automatically joined to this group unless their users.yml
# entry sets join_shared_group: true.
shared_group: lab
# Default OMERO owner used for imports. Keeping one import owner makes OMERO.web
# show imported projects under one user in "All Users"; per-root groups still
# control which users can see each root.
import_user: habomero
# Optional permission profile applied to shared_group by sync-users.
# Typical value for collaboration visibility: read-annotate
shared_group_permissions: read-annotate
# Optional permission profile applied to each per-root scan directory group.
scan_group_permissions: read-annotate

# Optional root folder name in OMERO for mirrored filesystem hierarchy.
omero_folder_root: scan-root
Expand All @@ -24,6 +33,14 @@ import_mode: inplace
# longer found during a successful scan of the configured source root. Source
# files are never deleted by this cleanup.
delete_omero_missing_files: true
# When true, files tracked by the old unscoped imported_files.txt format are
# rechecked/reimported under the current owner/group config so placement config
# changes can repopulate the configured OMERO location.
reimport_legacy_import_state: true
# When true, after a root imports cleanly, same-named scan-root Projects from
# older owner/group placements are deleted while the currently configured
# Project is kept.
cleanup_obsolete_duplicate_projects: true

# Safety controls for high-latency/remote sources:
# Maximum number of files to attempt in a single import-scan run.
Expand Down
4 changes: 3 additions & 1 deletion config/omero/users.example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,16 @@ users:
last_name: Service
group: lab
extra_groups:
- rxrx19a
- cfret_subtyping_data
- way_mckinsey_cardiac_fibrosis
email: habomero@example.org
institution: Local Lab
password_env: OMERO_USER_HABOMERO_PASSWORD
- username: test
first_name: Test
last_name: User
group: lab
group: test_private
email: test@example.org
institution: Local Lab
password_env: OMERO_USER_TEST_PASSWORD
Expand Down
23 changes: 18 additions & 5 deletions docs/src/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,11 +131,24 @@ Then synchronize those users into OMERO:
uv run poe sync-users
```

Users are joined to `shared_group` by default when it is configured in
`config/omero/scan_dirs.yml`. Set `join_shared_group: false` for a restricted
account, and use `extra_groups` for service/import users that need access to
per-root import groups. A scan directory entry may be a mapping with `path` and
`group` to import that root into a separate OMERO group.
Configure each real data root with its own per-root `group` so images are not
shared with all users by default. Imports without a per-root `group` use
`shared_group` when it is configured in `config/omero/scan_dirs.yml`. Users are
not joined to that group by default; set `join_shared_group: true` only for
users who should see fallback shared content. Use `extra_groups` for
service/import users that need access to per-root import groups.
`sync-users` creates scan root groups and applies `scan_group_permissions`.
Set top-level `import_user: habomero` to keep imported projects owned by one
service account in OMERO.web. Use per-root `group` values for access control;
only use per-root `import_user` when that root should appear under a different
OMERO owner.
With `reimport_legacy_import_state: true`, files tracked by an older owner/group
state format are rechecked and imported into the current configured owner/group.
This repopulates the desired OMERO location after config changes; it does not
delete old OMERO objects that were already imported elsewhere.
With `cleanup_obsolete_duplicate_projects: true`, a root that finishes cleanly
deletes same-named generated scan-root Projects from older owner/group
placements while keeping the currently configured Project.

## Backup

Expand Down
Loading
Loading