Kafka + Java + Data Engineer

- Extensive experience designing and developing streaming pipelines, data platform to handle high volume data processing, distributed system for both batch and realtime

- Strong Expertise in– Java, Kafka and Hadoop ecosystem (Spark, Hive, YARN, HDFS), Scala, Cassandra, Linux (on prem), CI-CD (Jenkins, Github), Obs (Dynatrace).

- Design and implement real-time streaming pipelines using Kafka

- Develop producers and consumers for high-throughput, fault-tolerant systems

- Responsible for both development and strong expertise to debug and fix any post production issues

- Implement Kafka-based event-driven architectures

- Experience with Databricks, Azure Cloud PaaS Services, Azure Data Factory, ADLS, Kubernetes, GitHub, Docker, and CI/CD workflows

- Experience in scripting, ETL (Pandas, Pola.rs, PySpark, Ibis), SQL, and NoSQL databases (Cassandra, MongoDB)

- Understanding of system architecture concepts: distributed systems, caching, replication, data modeling