Administration

This page is intended for system administrators, to document common tasks and tips needed to efficiently manage an Arkindex instance.

File storage

Arkindex requires the following S3-API buckets to store various files:

Configuration	Default name	Description
s3.staging_bucket	staging	Staging area where user uploaded files are stored before validation
s3.export_bucket	export	SQLite databases generated by the project export task
s3.thumbnails_bucket	thumbnails	Folder thumbnails generated by the thumbnails generation worker
s3.training_bucket	training	Machine Learning models
s3.ponos_artifacts_bucket	ponos-artifacts	Artifact files produced by processes
s3.ponos_logs_bucket	ponos-logs	Log files for all tasks of all processes
N/A	iiif-cache	(Optional) Bucket used by the Cantaloupe IIIF server to cache image renderings
N/A	uploads	(Optional) Bucket used by the Cantaloupe IIIF server to expose locally uploaded images

You can create these buckets on MinIO or any other S3-API compatible service by using the MinIO Client (open-source):

# Login on your provider by creating an alias
mc alias set arkindex-s3 <URL> <LOGIN> <PASSWORD>

# Create required buckets by prefixing them with the alias name
mc mb arkindex-s3/staging
mc mb arkindex-s3/export
...

Administration interface

As an administrator, you can use the administration interface on your Arkindex instance, available at https://<INSTANCE_URL>/admin/ (for example, if your instance lives at ark.localhost, you’ll be able to use the interface here).

Administration modules

Warning

Beware of your actions on this interface: you can delete most items in the Arkindex database, without any rights checks. You are an administrator: with great power comes great responsibility.

User creation and management

By clicking on the Users > Users link from the main page, you’ll reach the user management. From there you can:

• create new users
• promote an existing user to administrator
• change user email & display name
• change their password
• disable their account
• delete their account

Users management

Project management

By clicking on the Documents > Corpora link from the main page, you’ll reach the project management (projects are internally named corpus). From there you can:

• create a new empty corpus
• change corpus attributes (name, description, …)
• make the corpus fully public
• make the corpus search indexed
• delete a corpus

Asynchronous tasks

This page is not linked from the main administration page, and is only reachable by using the link https://<INSTANCE_URL>/rq/

From that page, you can view all the asynchronous queues, current tasks and active workers.

Asynchronous queues and workers

System worker management

While the recommended way to manage Arkindex system workers is to let the arkindex update_system_workers command configure the recommended worker versions, it is possible to manage them manually through the administration interface.

Under Process → Worker versions, the worker version management page includes a feature filter, which makes it easier to find the worker version that implements a specific feature.

Filtering worker versions by feature

To assign or unassign a feature on a worker version:

1. Click on the UUID of the worker version in the list.
2. Set or unset a feature in the Feature field.
3. Click on Save.

Assigning a feature on a worker version

Only one worker version can provide an Arkindex feature at once. To assign an already assigned feature to another version, you will need to first unassign the feature from the current version, then assign it to the new version. If a version was already assigned to a feature, saving will show an error.

Warning

Arkindex expects that the worker versions providing features meet the following conditions:

• Their state must be Available.
• They must not require a GPU.
• They must not require a model version to be set.
• They must not have required fields in their user configuration.

When those conditions are not met, imports and Workers processes may not work correctly.

arkindex update_system_workers verifies those conditions automatically.

Warm shutdowns of Ponos agents

Info

This feature is only available for Enterprise Edition customers.

Any Ponos agent that is currently running and has enough system resources available will automatically get new tasks assigned for it to run. This can make it difficult to stop an agent cleanly. While shutting an agent down will not stop the tasks themselves, they will no longer be tracked on Arkindex, and some data may be lost if they finish and get cleaned up.

Through the administration interface, it is possible to cause a warm shutdown of an agent:

1. Under Ponos: tasks manager → Agents, select an agent.
2. Toggle the Accept tasks checkbox to enable or disable assigning any new tasks to this agent.
3. Click on Save.

When Accept tasks is disabled, no new tasks will be assigned, but existing tasks will continue to run on the agent. When those tasks have finished, it becomes possible to shut an agent down without any impact on any process.