How does Tovi work?
Tovi is a distributed application which uses a client-server architecture to connect remote users with shared resources.
The Tovi Server application is deployed to a Kubernetes cluster and handles job scheduling by efficiently tracking and allocating processes as the required compute resources become available.
The Tovi Client can be used as a command line interface (CLI) tool, or via a web application. Remote users can create and manage projects, upload data to a shared directory, and submit jobs to be run on the server.
Tovi Server
The Tovi Server has three major components: the receiver, the manager, and the scheduler.
The receiver listens for requests from a Tovi Client application, authenticates the user, and forwards the request to the manager. Types of requests include (but aren’t limited to) job submissions, data uploads, and information queries.
The manager processes the request by type, copying any submitted files to the appropriate locations. Job submissions are added to a queue which is managed by the scheduler.
The scheduler scans cluster machines at regular intervals, allocating jobs from the queue as resources become available. The scheduler is highly customizable, so this behavior can be set as desired.
Tovi Client
The Tovi Client is a simple CLI tool which allows users to connect to and query a cluster of shared resources managed by a Tovi Server. This allows users to accomplish two primary tasks: (1) obtain information on different projects, available datasets, and the current state of the job queue; and (2) submit jobs i.e., code they want to run on the server.
The web app expands this functionality by providing a graphical user interface with which users can browse libraries of datasets and results.
tovi.yaml
Requests are defined by a special tovi.yaml file. This user-friendly and simple YAML file can encode complex experiment pipelines and is the only change required to switch from running code locally to running on Tovi.
