Back to Learning: Starting Data Engineer in Python
APIs, data cleaning, Git, ETL/ELT and Airflow — why I'm rounding out my SQL foundation with the parts of the stack SQL can't reach.
Shaharier Shourov
Data Analyst & Aspiring Data Engineer
Why I'm going back to Python
My last few months were SQL-heavy — PostgreSQL ETL pipelines, MySQL analysis, Snowflake concepts. That's given me a strong foundation, but there's a category of data engineering work SQL alone can't cleanly handle: pulling data from APIs, complex programmatic cleaning, and full pipeline orchestration.
"SQL gets you 80% of the way to a working data pipeline. Python covers the other 20% that actually makes it production-grade."
What the track covers
- Working with APIs — authentication, pagination, and pulling structured data from REST endpoints
- Data cleaning in Python — pandas-based transformation for the messier cases SQL struggles with
- Git — proper version control for data pipeline code, not just scripts
- ETL/ELT patterns — designing pipelines that separate extraction, transformation, and loading cleanly
- Airflow — DAG-based orchestration for scheduling and monitoring multi-step pipelines
How this connects to my SQL projects
My PostgreSQL ETL pipeline proved I could build a scheduled, self-auditing pipeline entirely in SQL. The Python track is about extending that same discipline to sources SQL can't reach directly — APIs, unstructured files, and orchestration across multiple systems.
Following along
I'll be posting updates as I work through this track, the same way I did with the SQL certification. If you're on a similar journey from a non-CS background into data, feel free to connect on LinkedIn.
