Hacker News Example Project#
The Hacker News example project takes advantage of a wide range of Dagster features and can be useful as a point of reference for how we imagine structuring Dagster projects and using different APIs. At a high level, it consists of three jobs, all centered around a (contrived) use case of downloading Hacker News user activity data, loading it into a database, and doing analysis / training an ML model on the aggregated data.
Beyond the dagster libraries that it demonstrates the use of (listed below), this example shows useful patterns for many Dagster concepts: schedules, sensors, IOManagers, resources, and unit tests (to name a few).
As time goes on, this project will be kept up to date, taking advantage of new Dagster features and interfaces. If you have anything you'd like to add, or like to see an example of, don't hesitate to reach out!
Integrations Used#
This project uses several dagster integration libraries: