Get familiar with the concepts around scheduled jobs and managing tasks with Prefect
Scheduled Jobs and Prefect
Prefect is an open source workflow orchestration tool made for data-intensive workloads. This allows you to schedule and organize jobs to run any time, in your chosen order. It can accommodate dependencies between jobs, and is very useful for data pipelines and machine learning workflows.
Saturn Cloud supports two different tools in the Prefect ecosystem:
- Prefect Core, which is an open source library users can install, and
- Prefect Cloud, which is a fully hosted cloud service which you can purchase. The same team that maintains the
prefectcore library runs Prefect Cloud. Prefect Cloud is a hosted, high-availability, fault-tolerant service that handles all the orchestration responsibilities for running data pipelines.
If you’re already feeling confused, or would like an introduction to the concepts behind job scheduling with Prefect, you can start with our Introduction to Prefect Concepts.