Data Engineer
idfcfirst.bank
Job Description
- Build and maintain data engineering pipelines, with a focus on unstructured data.
- Conduct requirements gathering and scoping sessions with business users and stakeholders to define GenAI-related data needs.
- Design, build, and optimize data architecture and ETL pipelines for accessibility by Data Scientists and GenAI products.
- Manage the full data lifecycle: ingestion, transformation, and consumption.
- Ensure high standards of data reliability, integrity, and governance.
- Work with APIs to enable seamless data integration and usability.
- Create technical design documentation for data pipelines and projects.
- Debug technical issues and manage code versioning using Git.
- Demonstrate experience with big data infrastructure such as MapReduce, Hive, HDFS, YARN, HBase, MongoDB, DynamoDB, etc.
Secondary Responsibilities
- Apply machine learning and predictive analytics techniques where relevant.
- Leverage domain knowledge in banking or financial services to enhance data solutions.
- Present data insights using effective storytelling and visualization techniques.
What We Are Looking For
Education
- Bachelor’s or master’s degree in computer science or Data Engineering or Information Systems or a related field.
Experience
- 2+ years of Proven experience in building and managing data pipelines and architectures.
- Hands-on experience with big data technologies and cloud platforms (AWS, GCP, or Azure).
- Exposure to GenAI applications and working with unstructured data formats.
Skills and Attributes
- Strong programming and debugging skills.
- Proficiency in data architecture, ETL design, and pipeline optimization.
- Familiarity with API integration and cloud services.
- Ability to work collaboratively with cross-functional teams.
- Excellent documentation and communication skills.
- Strong problem-solving mindset and attention to detail.
- Ability to deliver high-quality outputs under tight timelines.