Databricks Lakebase: The Reality of Postgres inside the Lakehouse
What Databricks Lakebase really delivers: Federation vs Lakehouse Sync, copy-on-write branching for CI/CD, cold starts, token expiry and governance.
Author
Michal Milosz
Michał is a Data Engineer specializing in creating and managing data pipelines using tools like Databricks and Azure. He enjoys tackling challenges related to data processing and continuously develops his skills to keep up with the latest advancements in the IT field. Beyond work, Michał has a strong interest in football, tennis, and history, and he loves to travel, exploring new places and cultures.
- 20
Articles
First article
Latest article
Areas of expertise
- Data Engineering
- Data Science
- DevOps
- Databricks
Articles by Michal Milosz
Breaking the Last Barrier: Why We Are Building Full-Stack Web Apps Directly Inside Databricks
Databricks Apps lets data engineers build and deploy secure full-stack internal web apps inside the lakehouse, without the usual infrastructure friction.
"SQL is the New Regex": Cleaning Messy Data with Databricks ai_query
Replace brittle regex and manual string parsing with Databricks ai_query, using SQL and PySpark AI functions to clean and standardize messy text data.
The Orchestration Dilemma: Driving the Engine vs. Designing the Factory
Databricks Workflows or Delta Live Tables in 2026? Compare driving the engine with designing the factory, and stop hand-wiring fragile data jobs.
The "Shift Left" Revolution: Why Your PySpark Pipelines Need Unit Tests (and How to Do It)
Shift left on data quality: unit test PySpark pipelines with modular code, local tests and CI/CD so bugs surface before they ever reach production.
The Orchestration Dilemma: Declarative vs. Imperative Patterns in Modern Data Engineering
Declarative Delta Live Tables or imperative Databricks Workflows? How to choose an orchestration pattern that fits your data platform and your team.
Streamlining CI/CD for Databricks Workflows with DAB Templates and Azure DevOps/GitHub Actions
Automating Databricks deployments with DAB templates and CI/CD tools ensures fast, error-free pipelines - read to learn how.
Unity Catalog and Volumes in Databricks Explained
Unity Catalog and Volumes: a data engineer's guide to modern data governance in Databricks, covering access control, lineage, and managed storage.
Designing Scalable Data Pipelines: A Practical Guide
Scalable data pipelines by design: learn batch, streaming, and layered architectures, with patterns and trade-offs for building reliable data systems.
The Power of Orchestration: Managing Complex Workflows in Databricks
Master Databricks workflow orchestration – automate tasks, integrate Airflow & ADF, and optimize pipelines. Boost efficiency with dynamic scheduling!
Azure Data Factory or Apache Airflow: Which Orchestration Tool Reigns Supreme?
Compare Azure Data Factory vs Apache Airflow: key features, strengths, and limitations for data orchestration. Choose the best tool for your workflow needs.
Managing Large Data Sets in Databricks
Optimize large datasets in Databricks with Partitioning, Z-Ordering, Auto Optimize, Delta Lake Vacuum, Caching, and Cost Monitoring for better performance.
Configuring the Celery Kubernetes
Configure the Celery Kubernetes Executor in Airflow 2.0 to combine Celery's scalability with Kubernetes' resource optimization for efficient workflows.
The Future of Data Engineering – Trends to Watch in 2025
Data engineering trends for 2025: AI-driven automation, Lakehouse architecture, serverless and edge computing, plus more sustainable data practices.
Data Warehouse vs Data Lake vs Lakehouse: How to Choose
Data warehouse vs data lake vs lakehouse: how the three architectures differ in structure, limitations and use cases, and when each one is the right fit.
How to spin up to 1000 parallel Airflow 2.0 tasks in 5 minutes from nothing with the CeleryExecutor and Kubernetes
Run up to 1000 parallel tasks in Apache Airflow 2.0 within 5 minutes by combining the CeleryExecutor with Kubernetes. Setup steps from a clean start.
From Excel to Data Lake: The Evolution of Data Storage in a Modern Organization
Explore the shift from Excel to data lakes, modern storage solutions, and best practices for scalable, efficient, and innovative data management strategies.
Lakehouse Federation in Databricks: A Practical Guide
Lakehouse Federation in Databricks: query external data sources without moving data. See how it works and when to use it in your data platform.
Microservices in Data Engineering: How to Break a Monolith into Smaller Parts
Discover how microservices revolutionize data engineering by replacing monolithic systems, boosting scalability, flexibility, and deployment efficiency.
Stream Processing vs. Batch Processing - A Practical Guide to Data Processing
Explore stream vs batch processing, their benefits, use cases, and technologies. Learn how to choose the right data strategy for your business needs.