Advanced documentation
Workers and Git repositories¶
Workers can, but do not have to, be linked to a Git repository through a repository_url attribute. When a worker is linked to a Git repository, its worker versions must in turn be linked to commits on this repository through a revision_url attribute. However it is also possible to create workers with no link to a Git repository: to be usable in processes, worker versions only need a Docker image reference.
Info
It is only possible to create worker versions from the frontend for workers that are not linked to a Git repository. To create a new worker version for a worker linked to a Git repository, you must follow the worker developer documentation.
Worker types¶
In Arkindex, you need to assign a type to any worker you create. There is no preset list of available types. You can use worker types to group together workers that perform similar tasks; for example you could use types such as recognizer, classifier, ner etc. You can use worker types to filter workers in the list of workers when selecting workers for a process or browsing available workers.
Worker versions¶
Version or Git revision URL¶
Depending on how it is created, and whether or not its parent worker is linked to a Git repository, a worker version can have either a version attribute or a revision_url linking it to a Git revision. Worker versions created from the frontend can only have a version, worker versions published to Arkindex following the worker developer documentation can have a revision_url.
A version is an integer that can be incremented each time a new worker version is published in order to keep track of the worker’s evolutions.
Branch¶
Worker versions published to Arkindex following the worker developer documentation can have a branch attribute linking them to a branch on a Git repository. Branches are unique per worker version: when a new worker version is published with a branch attribute, if a worker version already exists with the same branch attribute, then it is removed on the old version.
Tag¶
Worker versions published to Arkindex following the worker developer documentation can have a tag attribute. Tags can be used to easily identify a worker version.
Docker image IID¶
In order to be in the available state and usable in a process, a worker version must have a Docker image IID set. This Docker image IID is the link to the Docker image containing the code that will be executed when the worker version is used in an Arkindex process.
States¶
Worker versions can be in three different states:
created: the worker version has been created, but not validated.available: the worker version has been created and its Docker image IID is valid.error: something went wrong when validating the worker version.
Only worker versions in the available state can be used in processes.
Feature usage¶
Every worker version either supports, requires or does not support these two features: model usage and GPU usage.
Model usage¶
required: to run a process using this worker version, you have to use a model.supported: when using this worker version in a process, you can use a model but you do not have to. This can be the case for example if the worker version falls back to a default model.disabled: it is impossible to use a model when running a process with this worker version.
GPU usage¶
required: this worker version requires a GPU to run. This means that any process that uses this worker version will wait for a Ponos agent with a GPU to be available to start.supported: this worker version can be executed either on CPU or on GPU. You can choose whether or not to use a GPU when configuring a process.disabled: this worker version cannot be executed on a GPU.
Feature¶
See the system workers documentation.
User configuration¶
When you develop your own worker you can define in its configuration parameters that users will be able to set when using this worker version in a process. These parameters make up the user configuration.
For example, a worker can define a parameter called element_prefix: all the elements that will be created when running this worker will be prefixed with this parameter. When using the corresponding worker version in a process, users will be able to set this element_prefix to a string of their choice.
Warning
Unlike the frontend, the Arkindex backend does not validate configurations against an existing user configuration. This means that through the API, any configuration can be set. This also means that old configurations can still be used even if the user configuration has changed. To safeguard against this, you can archive obsolete configurations.
See also the user configuration section of the process documentation.
Worker access rights¶
Info
This only concerns Arkindex Enterprise Edition: in the Community Edition, there are no access rights.
If you have an administrator access to a worker, you can manage a worker’s memberships from its details. An administrator access also allows a user to archive or delete a worker, and to create new worker versions for that worker.
A contributor access to a worker is required to use it in a process.
See the rights documentation for more information.
Public workers¶
Info
This only concerns Arkindex Enterprise Edition: in the Community Edition, there are no access rights.
Some workers do not require access rights and can be executed by any user on an Arkindex instance. However, users still need administrator rights on these workers to perform some actions, such as archiving workers or managing their memberships.
See the rights documentation for more information.
Only instance administrators can make workers public.
Archived workers¶
Deleting workers that will not be used anymore is not always possible, because Arkindex does not allow deleting workers if objects (elements, transcriptions, entities etc) that they have produced still exist on the instance. It is however possible to archive a worker.
Warning
Only users with administrator access to a worker, or instance administrators, can archive workers.
Archived workers can no longer be used in processes, and are displayed in their own tab in the workers list. It is possible to archive a worker from the Arkindex frontend, but an archived worker can only be made available again by an instance administrator from the admin interface.
Secrets¶
Some workers may require the use of Secrets, like for example an authentication token to interact with an external API. See the dedicated secrets documentation.