Data Engineer III

pearson

Chennai, Tamil Nadu, India 6 Years Exp Posted 47d ago

Job Description

  • Design, develop, and maintain scalable ETL/ELT data pipelines supporting enterprise data products and analytics.

  • Build and optimize data solutions using Google BigQuery, GCS, Cloud Composer/Airflow, dbt, Python, and SQL .

  • Develop robust data ingestion and transformation pipelines from databases, SaaS applications, APIs, files, and other enterprise data sources.

  • Implement and maintain API integrations and data ingestion frameworks for cloud and enterprise applications.

  • Develop reusable dbt models, transformation frameworks, and data quality processes .

  • Build and manage workflow orchestration using Apache Airflow / Google Cloud Composer .

  • Work with CDC and data ingestion technologies such as Informatica CDC, Airflow, Composer, dbt, or similar platforms .

  • Design and implement modern data lakehouse architectures using BigQuery and related cloud technologies.

  • Optimize data pipelines and BigQuery workloads for performance, scalability, reliability, and cost efficiency .

  • Implement engineering best practices including CI/CD, version control, automated testing, deployment automation, and DevOps practices .

  • Contribute to data observability and monitoring frameworks , including pipeline health, data quality, SLA monitoring, and operational metrics.

  • Partner with data architects, analysts, data scientists, product teams, and business stakeholders to deliver high-quality data solutions.

  • Contribute to data governance, metadata management, lineage, and data discovery capabilities.

  • Explore and implement AI/ML and GenAI capabilities to improve data engineering workflows, automation, data discovery, and developer productivity.

  • Troubleshoot complex data pipeline and platform issues and drive root-cause analysis and long-term improvements.

  • Help establish engineering standards, reusable frameworks, and best practices across the Global Data organization.

    • Champion self-service data capabilities that improve the experience of data consumers and engineering teams.

Similar Openings for You