Tutorial

This is a workshop on how to use this package. The main components of the orchestrator is shown in the following image:

Short summary: - Tasks: Something that can be executed and gives back a result - Parameters: Input to tasks - Tools: Wrappers around external tools (like Virtuoso, Calibre, etc.) - PDK: API to access process dependend information - Assets: Additional files that can be delivered and packaged together with Tasks in a python package - Caching: Cache results of tasks to speed up execution - Logging: Standardized logging infrastructure

Let’s go through each of the topics step by step.

Tasks

Tasks build the fundamental idea of the orchestrator. You can think of a task as a function: - It takes arguments/parameters as an input - It does calculations based on the inputs - It may call other tasks/functions - It returns a final result

These tasks can be as complex or simple as you want. However, instead of using plain python methods to do this job, we want to add more information to tasks, like: - What parameters are required? - Are the provided parameters valid? - What dependencies does it have? - Where to store the log? - Does it need specific information from the PDK? - If a task shall be cachable (more on that later), is the cache still valid? - Where to store the results and intermidate files? - How to visualize and present the task to a “non-programmer”? - How to profile the task? - …

As you can see, there are many additional topics that eventually each and every framework would have to reinvent. By providing a common way of structure the code, we can increase reuse, readability, maintainability and number of features.

How to define a task?

A task is a simple class.

from mosaic_orchestrator.task import Task

class TaskA(Task):
    def run(self):
        return "Hello"

How to run a task?

To run the task TaskA we have to use the Mosaic class and create a builder (we are using the common builder design pattern, see https://en.wikipedia.org/wiki/Builder_pattern).

Simply speaking, we are creating an instance of a Task and then we can run it.

from mosaic_orchestrator.mosaic import Mosaic

instance = Mosaic.create(TaskA).build()
instance.run()

'Hello'

Such kind of tasks are not very helpful yet. Let’s find out how to include parameters. # Inputs

All inputs inherit from the base class Input.

from mosaic_orchestrator.task import Task, Input

class TaskB(Task):
    some_arg : Input

    def run(self):
        return self.some_arg.value

from mosaic_orchestrator.mosaic import Mosaic

instance = Mosaic.create(TaskB).with_inputs({
    "some_arg": "Hello"
}).build()
instance.run()

'Hello'