Senior Data Platform Engineer
avathon
Job Description
- Work on engineering projects to design, construct, install, test and maintain highly scalable data management systems (data lakes, warehouses, pipelines) to ensure secure, efficient data flow and accessibility for analytics and AI/ML
- Building robust ETL pipelines and automating data workflows. Focus on automating data management, implementing DevOps principles (CI/CD, monitoring), and collaborating on data architecture.
- Utilise Key tools include Python, SQL, Cloud platforms (AWS/Azure/GCP), Kafka, Spark, and Docker.
- Developing and maintaining scalable ETL and MLOps pipelines and workflows.
- Enforcing data quality rules to ensure consistency, completeness, and accuracy across ETL processes.
- Designing, constructing, testing, and maintaining highly scalable data management systems.
- Mentoring a team of Data Analysts, Data Engineers, and Data Quality Specialists.
- Fostering a culture of continuous learning, collaboration, and data-driven decision-making.