Azure Data Factory or Apache Airflow: Which Orchestration Tool Reigns Supreme?
Compare Azure Data Factory vs Apache Airflow: key features, strengths, and limitations for data orchestration. Choose the best tool for your workflow needs.
DevOps
Dealing with code vulnerabilities with SNYK
How Snyk finds vulnerabilities in source code, external libraries and configuration, and why automated security testing belongs in every software project.
5 Best Practices for Data Validation
Data validation keeps bad records out of business decisions. Five techniques, from Python scripts to enterprise tools, to keep data accurate and complete.
A more efficient scheduler to improve performance in Airflow 2.0
Airflow 2.0 ships a faster, scalable scheduler with an active-active setup and zero downtime, plus fast-follow task execution for complex pipelines.
Arkadiusz Bach
How to improve Airflow 2.0 performance with Smart Sensors?
Compare Azure Data Factory and Apache Airflow: source integration, workflow management, data transformation, and scheduling flexibility for your needs.
How to simplify Airflow 2.0 setup with the DAG Versioning and Serialization
Airflow 2.0 added DAG serialization and versioning. See how these features simplify setup and deployment and keep a history of changes to your pipelines.
How to spin up to 1000 parallel Airflow 2.0 tasks in 5 minutes from nothing with the CeleryExecutor and Kubernetes
Run up to 1000 parallel tasks in Apache Airflow 2.0 within 5 minutes by combining the CeleryExecutor with Kubernetes. Setup steps from a clean start.
Powerful REST API in Airflow 2.0 — what do you need to know?
Discover the benefits of Apache Airflow 2.0, including REST API, efficient scheduling, and smart sensors. Learn how it improves workflow management.
Python „Code check” implementation in IDE + Github action
Use flake8 to catch Python errors and style issues before the code even runs, both in your IDE and in a GitHub Actions workflow that checks every push.
Michał Milik
7 Vs of Big Data: What They Are & Why They Matter
The 7 Vs of Big Data explained: volume, velocity, variety, and beyond. See what each one means and why they matter for your data strategy.
Advanced Kubernetes for FMCG: Automating Deployment, Scaling, and Management of Containerized Applications
How Kubernetes automates deployment, scaling and management of containerized applications for FMCG companies, plus its architecture and best practices.
Kamil Leduchowski
Airflow Celery Executor – what do you need to know about it?
Airflow Celery Executor explained: how it distributes tasks across workers, when to use it, and how to configure it for scalable data pipelines.
Tomasz Stachera