DataOps is an automated, process-oriented methodology, used by analytic and data teams, to improve the quality and reduce the cycle time of data analytics. While DataOps began as a set of best practices, it has now matured to become a new and independent approach to data analytics. DataOps applies to the entire data lifecycle from data preparation to reporting, and recognizes the interconnected nature of the data analytics team and information technology operations.
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Turns Data and AI algorithms into production-ready web applications in no time.
Snowflake CLI is an open-source command-line tool explicitly designed for developer-centric workloads in addition to SQL operations.
Opt-Out tool to check Copyright reservations in a way that even machines can understand.
Open Source Data Quality Monitoring.
funsies is a lightweight workflow engine 🔧
A data lineage tool detects table dependencies from rendered SQL statements.
Efficient streaming data ingestion, transformation & activation
Data analytics library for Python and suite of open source, command line based data ops tools.
A next-generation open source orchestration platform for the development, production, and observation of data assets.
A prefect extension that builds on top of the task decorator to reduce negative engineering!
The Open Platform for building Data Platforms. Ship faster. Stay compliant. Scale to Data Mesh.
Local-first spreadsheet operations workbench with isolated site setup, adaptive profiles, versioned runs, dashboards, and release safety gates.
🎯 DBT Masterclass with CI/CD | Hands-on Data Transformation Guide A comprehensive tutorial repository for mastering Data Build Tool (DBT) from scratch to deployment! Includes Bronze-Silver-Gold architecture, testing, snapshots, macros, and production CI/CD workflows. Follow along with the 5-hour video tutorial! 🚀
A schema contract guard for data pipelines. Detects, classifies, and reports schema changes in PostgreSQL databases.
Production-grade E-Commerce analytics engineering pipeline featuring Python ETL, dbt Medallion modeling, schema quality assertions, and DuckDB warehousing.
Production-style ML incident response system. Runbooks, Airflow DAGs, dbt models, drift detection, postmortems, Docker, and CI — built for MLOps operational portfolio demonstration.
An autonomous data-ops analyst on Google ADK that predicts a KPI miss, finds the cause, and opens the pull request that fixes it.
Airflow demo pipeline for validating, transforming, and incident-reporting on daily paid media extracts.