Using jobs in Dagster projects

info

Dagster Components is currently in Release Candidate status. APIs are stable, with broader integration coverage and full feature parity in active development and coming soon. Check it out and let us know what you think in the #dg-components channel in the Dagster Community Slack!

Prerequisites

Before following this guide, you will need to create a project with the create-dagster CLI.

Assets frequently use jobs that are instantiated elsewhere in the project.

For example, if you have created a new Dagster project with dg called my_project, you can define the jobs at src/my_project/defs/jobs.py:

Job binding can happen at any level of the defs hierarchy. If you moved asset_one in this example to a subdirectory, you could leave the existing jobs.py file at src/defs/jobs.py:

src
└── my_project
    └── defs
        ├── assets
        │   └── asset_one.py # contains def asset_one():
        └── jobs.py # contains dg.job

Scaffolding jobs

To create a job dictionary like the above, you can run the following:

dg scaffold defs dagster.job path/to/jobs.py

which will create

src/<project_name>/defs/jobs.py

import dagster as dg


@dg.job
def jobs():
    pass

and you can fill out the job dictionary as needed.