Classification

A classification links an existing Machine Learning class to an element, with a confidence score.

It can also be used for manual classification by human annotators.

Rules¶

Machine learning classes can be configured by instance administrators, project administrators, or workers.
A classification has a state attribute:
default to Pending, when created by a workflow,
✅ Validated when approved by a human annotator,
❌ Rejected when invalidated by a human annotator.
A classification has a confidence score, set between 0.0 and 1.0 (percentage). It defaults to 1.0 when the classification is created by a human (we always trust human annotators)
A machine learning workflow can set the high_confidence flag to True on the classification, when it judges that the confidence score is high enough (that score would be dependent on tools, models and context).

Web interface¶

Classes and classifications can be viewed and modified on multiple pages of the web interface.

Element lists¶

When browsing elements, you can apply classification filters using the filter bar:

Class name: Select elements that have a specific ML class.
Classification confidence: Select elements that have a classification with a confidence greater than, equal to, or less than a specified threshold.
High confidence classification: Select elements that have or do not have a classification marked as high confidence by a worker.

When any of those filters are combined, then only elements that have a classification that matches all the specified criteria will be selected. An element with multiple classifications that match only one of each criteria would not be displayed.

Filtering elements by class, confidence score, and high confidence state.

Element details¶

When viewing the details of a single element, the panel on the right side has a Classifications section where you can:

View all current classifications, grouped by worker, or Manual (created by a human annotator)
Moderate the classifications (either validate or reject)
Add a new classification (which will then appear in the Manual section, and already validated)

You cannot create a new Machine Learning class from this interface, but you can apply an existing one (provided it’s not already applied).

The classifications panel with a classification produced by a worker

If a Machine Learning worker produces a classification with a high confidence (known high score), a gold medal icon will appear next to the class name:

A medal icon appears on classifications with high confidence

Class management¶

Classes can be managed by project administrators by going to Project information under the Project menu in a project, then going to the Classes tab.

The Project information button under the Project menu

The Classes tab on the project information page

From the class management page, it is possible to add, rename and delete classes. Classes can only be deleted if they are not used by any classification on any element.

API Endpoints¶

These endpoints are the most useful to handle Element classifications:

In order to retrieve the classifications within a list of elements, you can use with_classes=true query parameter on these endpoints: