How it works¶
A short tour of the ideas you will meet in the widgets, the YAML file and the rest of these docs.
The training set and its config¶
Each annotation project is a training set: a fixed selection of images, split into training, validation and test, plus everything annotated for it. It is described by a configuration (YAML) that says which OMERO container (or OME-Zarr images) to take images from, which channel to annotate and how many images go into each split. You can create it with the Jupyter widget or write it by hand; see the configuration reference.
A YAML file you write is a template: TrainingSet.create(config, conn) makes a new set
from it and never changes the file. The set keeps its own config.yaml (with its
identity, seed and the record of every annotation session, including your notes) in
its folder. TrainingSet.open(table_id) or TrainingSet.open(folder) continues a set;
see resume or repeat a run.
Where images come from¶
Images are read directly from an OMERO container: a project, dataset, plate, screen or
a single image. For plates you can keep only wells with certain key-value pairs, for
example treatment: DMSO. Images can also come from OME-Zarr files or URLs
(source: zarr, e.g. the BioImage Archive), without OMERO. Every image is identified by
a URI: omero:image:<id> or the Zarr address.
Annotation units¶
You rarely annotate a whole multi-dimensional image. The pipeline cuts each image into
annotation units: one 2D plane (a chosen channel, timepoint and z-slice), or one 3D
volume if you switch on 3D mode. Timepoints and z-slices are each chosen as a list, all
of them, or n at random per image. With patches switched on, each plane is further cut
into smaller crops (for example 512 × 512 pixels), which are quicker to annotate and to
process on a CPU.
flowchart LR
I[Image<br/>channels · time · z] --> P1[Plane<br/>t=0, z=5]
I --> P2[Plane<br/>t=3, z=5]
P1 --> A[Patch 1]
P1 --> B[Patch 2]
Every unit gets an ID such as 4ba674d4_0_5 (a short key of the image URI, timepoint, z-slice), and all files for
that unit are named after it.
Train, validation and test sets¶
Images are picked at random from the container and assigned to a training,
validation or (optionally) test set. You either give counts (train_n: 10,
validate_n: 3) or annotate everything and give fractions. Counts are in images, not
units: one image with four patches gives four units.
The two storage modes: OMERO or local¶
storage: omero (default) |
storage: local |
|
|---|---|---|
| Writes to OMERO | yes: masks, ROIs, tracking table, config | nothing |
| Local folder | images, masks and tracking_table.csv |
the same; it is the only copy |
| Typical use | your own data on your institute's server | OME-Zarr data (e.g. public IDR studies), servers you cannot write to, or a trial run |
Set it with storage in the config or the Storage choice in the widget. It is fixed
when the training set is created; OME-Zarr sets are always local.
What ends up in OMERO¶
For a set with storage: omero, for each annotated unit:
- a label mask (TIFF, compressed losslessly) attached to the image,
- polygon ROIs on the image, visible in OMERO.web and OMERO.iviewer,
both tagged omero-annotate-ai:<table id>:<unit id> in their description, and for the
set as a whole:
- a tracking table on the container listing every unit, its split, whether it is done, the IDs of its mask and ROIs, and a pixel checksum of image and mask,
- the set's config YAML next to the table, updated after each session.
The tracking table is updated in place, so its ID never changes: that ID is the
training set's identity. The checksums let a resume or an export notice when a local
mask no longer matches the one on OMERO. With the provenance extra each unit also
gets an ISCC Image-Code of image and mask, which tells how much
a file changed: a touched-up mask is reported as a near-duplicate, a replaced one as
different content.
Batches¶
workflow.batch_size sets how many units napari shows at once. Each mask is saved to
the set's folder and uploaded as soon as the batch is closed; the OMERO table is updated
once per batch. With 0 everything is opened in one go. If something crashes, resuming
the set repairs what was half done (see
Resume or repeat a run).
Sessions, notes and review¶
Every time you annotate, a session is recorded in the set's config: who, when, which
model, how many units were saved, and an optional note ("nuclei only, small debris
skipped"). The note can be edited later, also for earlier sessions. ts.review() (widget:
Review annotated) opens annotated units with their saved masks so you can correct them;
a changed mask replaces the label file and ROI on OMERO, an unchanged one is left alone.
Several computers, one set¶
With storage: omero the OMERO table is the shared record, so a set can be opened on any
computer. Before napari shows a batch, the session claims its units in the table;
other sessions skip claimed units, and a unit someone else saved first is never uploaded
again (your mask is kept in annotation_output/conflicts/). Claims from your own earlier
runs on the same computer (after a crash or a dropped connection) never block you; other
computers' claims expire after workflow.claim_hours (default 6).
Local folders¶
Each training set has one folder on your disk, with the same layout in every mode:
~/omero_annotate_ai/ ← home folder
├── <server>/<table id>_<name>/ a set tracked in an OMERO table
└── local/<id>_<name>/ a local (or OME-Zarr) set
├── config.yaml the config and progress, used to resume
├── tracking_table.csv local copy of the progress table
├── annotation_input/ images as annotated: <unit id>.tif
├── annotation_output/ accepted masks: <unit id>_mask.tif
├── external_output/ masks from Cellpose or other tools, until collected
├── model_input/ training channel(s), if different from the annotated one
├── ro-crate-metadata.json RO-Crate description, written by ts.write_ro_crate()
└── .scratch/ micro-SAM's files for the current batch only
Choose another home folder with the OMERO_ANNOTATE_AI_HOME environment variable or
home= on TrainingSet.create / TrainingSet.open. Files are written atomically (a crash never
leaves half a file), and micro-SAM works in a fresh folder for every batch, so masks
always land on the image they were drawn on.
Training data is written to a separate folder; see Prepare training data.