Data Engineer

Direct Responsibilities

• Design & Development: Design and implement robust, scalable data pipelines and backend services using Java -Spring Boot, Spark.
• Java & Spring boot Aspects: Design robust, high performance, easily maintainable backend applications using Java and Spring boot.
• Spark Aspects: Design Spark based ETL pipelines to process heavy volume of data residing on On-Prem as well as Cloud clusters with a high performance 
• Kafka Aspect: Ability to design solutions integrated with Kafka as one of the source & target end points
• Database Aspects: Design MongoDB pipelines, optimize complex queries, and manage indexing strategies to ensure high availability and low latency.
• API Integration: Develop and maintain secure, high-performance RESTful APIs to facilitate seamless data exchange between internal systems and external partners.
• Cloud Infrastructure: Deploy and manage data workloads on cloud platforms (e.g., IBM Cloud, AWS, Azure, or GCP), leveraging managed services for compute and storage.
• Performance Tuning: Conduct deep-dive analysis of system bottlenecks and implement performance optimizations across the data stack.
• Code Quality: Perform rigorous code reviews and implement automated testing frameworks (JUnit, Mockito) to maintain high software quality standards.
• Able to communicate effectively with IT teams as well as Business Teams
• Closely work with Tech Leads/BA to resolve technical & business issues and to develop a good understanding of the project objectives.
• Must have good analytical skills.
• Excellent verbal and written communication skills. Should be able to communicate effectively with IT teams as well as Business Teams

Contributing Responsibilities

• Team player
• Effectively communicate work and delivery status
• Adhere to timelines set for delivery
• Open to adaptation of new technology, learning and implementation within the project.
• Able to integrate in a French-Indian agile extended team

Technical & Behavioral Competencies

• Experience of 7-9 years in Java Development field with working experience in Java, Spring boot, API Integration and cloud infrastructure
• Strong experience in translating requirements to an enterprise solution meeting all architectural and design constraints with robust implementation
• Expert proficiency in Java and experience with modern frameworks (Spring Boot)
• Expert proficiency in designing fault tolerant, robust Spark Pipelines to process Batch and real time huge volume of data
• Working knowledge of Kafka to create and manage topics, read/write data with high throughput 
• Working knowledge of MongoDB (Aggregation framework, Sharding, Replication, and Atlas).
• Experience with cloud environments.
• Proficiency with Git, Maven/Gradle, and containerization tools like Docker and Kubernetes.
• Good knowledge and experience of writing complex SQL queries.
• Knowledge of REST API design principles, OAuth2, and API Gateway management will be added advantage
• Proficient in analytical skills (required)
• Excellent oral and writing communication skills (required)
• Experience in working in Agile environment
• Good knowledge of CI/CD and automated Unit & Integration testing using pipelines will be an added advantage
  • Experience with Event driven architecture will be an advantage