Data Engineer
amgen
Job Description
- Design, develop, and optimize data pipelines/workflows using Databricks (Spark, Delta Lake) for ingestion, transformation, and processing of large-scale data
- Build ETL pipeline with Informatica or other ETL tools.
- Support data governance and metadata management.
- Collaborate with Data Architects, Business SMEs, and Data Scientists to design and develop end-to-end data pipelines to meet fast paced business needs across geographic regions
- Identify and resolve complex data-related challenges
- Adhere to best practices for coding, testing, and designing reusable code/component
- Analyze business and technical requirements and begin translating them into simple development tasks
- Execute unit and integration tests, and contribute to maintaining software quality
- Identify and fix bugs and defects during development or testing phases
- Contribute to the maintenance and support of applications by monitoring performance and reporting issues
- Use CI/CD pipelines as part of DevOps practices and assist in the release process