Senior Data Engineer

amgen

Hyderabad 9 Years Exp Posted 1h ago

Job Description

  • Lead the design and development of scalable batch and real-time ETL/ELT pipelines. 

  • Own complex data solutions from requirements and design through deployment and production support. 

  • Build cloud-based lakehouse and data-platform solutions using Databricks and AWS. 

  • Develop reusable, metadata-driven data integration frameworks. 

  • Integrate structured, semi-structured, and unstructured data from enterprise, manufacturing, API, and third-party sources. 

  • Optimize Spark workloads, Databricks compute, SQL queries, partitioning, storage, and caching for performance and cost. 

  • Implement workflow orchestration, monitoring, alerting, data-quality controls, and recovery processes. 

  • Develop CI/CD pipelines and automated testing for data solutions. 

  • Implement metadata management, lineage, cataloging, governance, RBAC, and data-security controls. 

  • Define data models, data contracts, integration patterns, and reusable engineering standards. 

  • Lead architecture reviews, code reviews, troubleshooting, and root-cause analysis. 

  • Mentor engineers and provide technical guidance across delivery teams. 

  • Collaborate with architects, analysts, data scientists, product teams, and DevOps teams. 

  • Support estimation, sprint planning, technical roadmaps, and delivery-risk management. 

  • Evaluate emerging technologies and recommend solutions based on scalability, security, maintainability, and business value. 

  • Participate in operational support, including occasional off-hours support.