Classification

    A classification links an existing Machine Learning class to an element, with a confidence score.

    It can also be used for manual classification by human annotators.

    Rules🔗

    • Machine learning classes are set by instance administrator (should be available to project administrator in S2 2021), and Machine learning tools.
    • A classification has a state attribute:
      • default to Pending, when created by a workflow,
      • Validated when approved by a human annotator,
      • Rejected when invalidated by a human annotator.
    • A classification has a confidence score, set between 0.0 and 1.0 (percentage). It defaults to 1.0 when the classification is created by a human (we always trust human annotators)
    • A machine learning workflow can set the high_confidence flag to True on the classification, when it judges that the confidence score is high enough (that score would be dependent on tools, models and context).

    Best classes🔗

    Best classes represents significant classifications for an element. They are used by the web interface and API endpoints to filter the most important elements.

    A classification is known has a best class:

    • When ✅ validated by a user
    • or when the high_confidence flag is set by a Machine Learning workflow (and has not been rejected)

    Web interface🔗

    When browsing elements, you can select a specific class to view which elements have that class with a high confidence, or already validated. You can also filter all elements that have any best class, or no best class at all.

    Filter elements with a specific best class.
    Filter elements with a specific best class.

    When viewing the details of a single element, the panel on the right side has a Classifications section where you can:

    • View all current classifications, grouped by worker, or Manual (created by a human annotator)
    • Moderate the classifications (either validate or reject)
    • Add a new classification (which will then appear in the Manual section, and already validated)

    You cannot create a new Machine Learning class from this interface, but you can apply an existing one (provided it's not already applied)

    The classifications panel with a classification produced by a worker
    The classifications panel with a classification produced by a worker

    If a Machine Learning worker produces a classification with a high confidence (known high score), a gold medal icon will appear next to the class name:

    A medal icon appears on classifications with high confidence
    A medal icon appears on classifications with high confidence

    API Endpoints🔗

    These endpoints are the most useful to handle Element classifications:

    In order to retrieve the best classifications within a list of elements, you can use with_best_classes=true query paramenter on these endpoints: