DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
QN : Ingest and transform data in a lakehouse
pauletart profile

QN : Ingest and transform data in a lakehouse

#architecture #database #dataengineering #sql
Comments
2 min read
Apache Kafka Explained: A Practical Beginner Guide for Data Engineers
anthony-gicheru profile

Apache Kafka Explained: A Practical Beginner Guide for Data Engineers

#kafka #dataengineering #beginners #distributedsystems
3
Comments
9 min read
Honest Memory: What Production Accuracy Data Actually Shows About AI Agent Memory
labyrinthanalytics profile

Honest Memory: What Production Accuracy Data Actually Shows About AI Agent Memory

#ai #machinelearning #dataengineering #llm
Comments
4 min read
Building on Brazilian public data: a developer's field guide (CNPJ, CEP, Congress, BACEN)
rodrigo_dasilva_c90c2024 profile

Building on Brazilian public data: a developer's field guide (CNPJ, CEP, Congress, BACEN)

#api #data #dataengineering #resources
Comments
2 min read
I let an AI agent set up my entire Kafka platform. Here's what actually happened.
sderosiaux profile

I let an AI agent set up my entire Kafka platform. Here's what actually happened.

#kafka #dataengineering #devops #tutorial
Comments
4 min read
LINUX FUNDAMENTALS FOR DATA ENGINEERING
karen_wangui_ profile

LINUX FUNDAMENTALS FOR DATA ENGINEERING

#dataengineering #linux #tutorial
Comments
5 min read
Advanced Kubernetes Patterns for Data Engineers
victor_kipruto_0f44ad2a3b profile

Advanced Kubernetes Patterns for Data Engineers

#kubernetes #dataengineering #docker #devops
5
Comments
1 min read
Extract data from Databases into DuckLake
flarco profile

Extract data from Databases into DuckLake

#duckdb #ducklake #dataengineering #etl
Comments
4 min read
How I forced Python standard libraries to process and serialize production server crashes into Parquet locally
nexus_os profile

How I forced Python standard libraries to process and serialize production server crashes into Parquet locally

#python #dataengineering #devops #opensource
Comments
1 min read
From Clean CSVs to Production‑Shaped Data: A Practical Guide for Academic ML and Data Engineering
jitendra_devabhaktuni_0f1 profile

From Clean CSVs to Production‑Shaped Data: A Practical Guide for Academic ML and Data Engineering

#ai #database #datascience #dataengineering
Comments
5 min read
I Built a Write-Ahead Log in Pure Python and Finally Understood How Databases Survive Crashes
hajirufai profile

I Built a Write-Ahead Log in Pure Python and Finally Understood How Databases Survive Crashes

#python #database #dataengineering #programming
Comments
7 min read
Architecture Over Alerts: How We Cut BigQuery Costs by 57%($12M) for a Fortune 500
pratikdhanave profile

Architecture Over Alerts: How We Cut BigQuery Costs by 57%($12M) for a Fortune 500

#bigquery #finops #cloudarchitecture #dataengineering
Comments
4 min read
I Built a Columnar File Format in Pure Python — a tiny, readable Parquet
hajirufai profile

I Built a Columnar File Format in Pure Python — a tiny, readable Parquet

#python #dataengineering #database #programming
Comments
6 min read
Running Apache Airflow + Docker for Free Using GitHub Codespaces
tanmay_bhurkunde profile
Tanmay

Running Apache Airflow + Docker for Free Using GitHub Codespaces

#dataengineering #docker #python #apacheairflow
Comments
1 min read
Why Big Tech is Migrating from Traditional Databases to NewSQL
dinhphu2k1gif profile

Why Big Tech is Migrating from Traditional Databases to NewSQL

#bigdata #dataengineering #database #sql
1
Comments
1 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.