Data Engineer

Primary Responsibilities

• Minimum 2 to 3 years of hands-on experience in data engineering.
• Proven expertise in SQL, Spark, and the Hadoop ecosystem.
• Experience handling multiple terabytes of data from ingestion to consumption.
• Collaborate with business stakeholders to identify and document high-impact business problems and potential solutions.
• Strong understanding of Data Lake/Lakehouse architecture and exposure to Hadoop (Cloudera, Hortonworks) and/or AWS.
• Manage the full data lifecycle: ingestion, transformation, and consumption.
• Proficient in Spark, Spark Streaming, Hive, and SQL.
• Experience with big data infrastructure including MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc.
• Deep knowledge of relational databases and data modeling.
• Technical debugging skills and experience with Git for version control.
• Create and maintain technical design documentation for data pipelines and projects.

Secondary Responsibilities

• Ability to work independently and manage personal development efforts.
• Strong oral and written communication skills.
• Learn and apply internal analytics technologies.
• Identify key performance indicators and develop strategies to achieve them.
• Use educational background in data engineering to perform data mining and analysis.
• Collaborate with BI analysts/engineers to build prototypes.
  • Participate in the delivery and presentation of data solutions.