Associate III - Data Engineering

ripplehire

Hyderabad 3 Years Exp Posted 2h ago

Design, develop, and maintain scalable and reliable data pipelines for processing large volumes of structured and unstructured data.
Build and optimize ETL/ELT workflows using PySpark and Python.
Develop and manage data lake and data warehouse solutions on AWS Cloud.
Implement data ingestion frameworks from multiple data sources, including databases, APIs, files, and streaming platforms.
Leverage AWS services such as S3, Glue, EMR, Lambda, Redshift, Athena, CloudWatch, and IAM for data engineering solutions.
Perform data transformation, cleansing, validation, and enrichment to ensure high data quality.
Optimize Spark applications for performance, scalability, and cost efficiency.
Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver solutions.
Monitor data pipelines and resolve production issues to ensure data availability and reliability.
Implement security best practices, governance, and compliance standards across data platforms.
Participate in code reviews, unit testing, and deployment activities.
- Create technical documentation and maintain operational runbooks.