Data Engineer
amgen
Job Description
Data Engineering & Pipeline Development
- Build and optimize databricks pipelines using modern frameworks (Databricks, Spark)
- Implement reliable, scalable, and production-ready data pipelines using engineering best practices, monitoring, and automated validation frameworks
- Integrate structured and unstructured legal data into the enterprise data fabric
- Ensure reliability, scalability, and performance of data pipelines
Databricks & Modern Data Platform
- Develop pipelines using Databricks (Delta Lake, Spark, notebooks)
- Implement data transformation and orchestration workflows
- Support migration and modernization of legacy data solutions to cloud-native platforms
- Contribute to reusable data engineering patterns and components
- Optimize Delta Lake and Spark workloads for scalable, cost-efficient, and high-performance enterprise data processing
Data Quality, Governance & Compliance
- Implement data quality checks, validation rules, and monitoring
- Implement governance, lineage, and security controls for sensitive legal and compliance datasets
- Ensure compliance with data governance, privacy
Collaboration & Delivery
- Work with Legal stakeholders to understand data needs and translate into technical solutions
- Partner with Data Architects to align with enterprise data fabric strategy
- Participate in Agile development processes (sprint planning, estimation, delivery)
- Document pipelines, models, and technical decisions