Senior Data Engineer
amgen
Job Description
-
Lead the design and development of scalable batch and real-time ETL/ELT pipelines.
-
Own complex data solutions from requirements and design through deployment and production support.
-
Build cloud-based lakehouse and data-platform solutions using Databricks and AWS.
-
Develop reusable, metadata-driven data integration frameworks.
-
Integrate structured, semi-structured, and unstructured data from enterprise, manufacturing, API, and third-party sources.
-
Optimize Spark workloads, Databricks compute, SQL queries, partitioning, storage, and caching for performance and cost.
-
Implement workflow orchestration, monitoring, alerting, data-quality controls, and recovery processes.
-
Develop CI/CD pipelines and automated testing for data solutions.
-
Implement metadata management, lineage, cataloging, governance, RBAC, and data-security controls.
-
Define data models, data contracts, integration patterns, and reusable engineering standards.
-
Lead architecture reviews, code reviews, troubleshooting, and root-cause analysis.
-
Mentor engineers and provide technical guidance across delivery teams.
-
Collaborate with architects, analysts, data scientists, product teams, and DevOps teams.
-
Support estimation, sprint planning, technical roadmaps, and delivery-risk management.
-
Evaluate emerging technologies and recommend solutions based on scalability, security, maintainability, and business value.
-
Participate in operational support, including occasional off-hours support.