Repositories#

A repository is a collection of jobs, schedules, and sensor definitions that the Dagster CLI, Dagit and the Dagster Daemon can target to load them.

Relevant APIs#

Name	Description
`@repository`	The decorator used to define repositories. The decorator returns a `RepositoryDefinition`
`RepositoryDefinition`	Base class for repositories. You almost never want to use initialize this class directly. Instead, you should use the `@repository` which returns a `RepositoryDefinition`

Overview#

A repository is a convenient way to organize your job and other definitions. Each repository:

Includes various definitions: Jobs, Schedules, Sensors.
Is loaded in a different process than Dagster system processes like Dagit. Any communication between the Dagster system and repository code occurs over an RPC mechanism, ensuring that problems in repository code can't affect Dagster or other repositories.
Can be loaded in its own Python environment, so you can manage your dependencies (or even your own Python versions) separately.

You can set up multiple repositories and load them all at once by creating a workspace.yaml file. This can be useful for grouping jobs and other artifacts by team for organizational purposes. See Workspace to learn more about setting up multiple repositories.

Defining a Repository#

Repositories are typically declared using the @repository decorator. For example:

from dagster import RunRequest, ScheduleDefinition, job, op, repository, sensor


@op
def hello():
    pass


@job
def job1():
    hello()


@job
def job2():
    hello()


@job
def job3():
    hello()


job1_schedule = ScheduleDefinition(job=job1, cron_schedule="0 0 * * *")


@sensor(job=job2)
def job2_sensor():
    should_run = True
    if should_run:
        yield RunRequest(run_key=None, run_config={})


@repository
def my_repository():
    return [
        job1_schedule,
        job2_sensor,
        job3,
    ]

The repository specifies a list of items, each of which can be a JobDefinition, ScheduleDefinition, or SensorDefinition. If you include a schedule or sensor, the job it targets will be automatically also included on the repository.

Using a Repository#

If you save the code above as repo.py, you can then run the Dagster command line tools on it. Try running:

dagit -f repo.py

Now you can see that all jobs in this repository are on the left:

You can also use -m to specify a module where the repository lives (See dagit for the full list of ways to locate a repository).

Loading repositories via the -f or -m options is actually just a convenience function. The underlying abstraction is the Workspace, which determines all of the available repositories available to Dagit. See Workspace for more details.