Senior Data Engineer

• Understanding Business Requirements and translating them into low level design for implementation
• Estimate the effort required to execute the business requirements
• Development of intake pipelines to ingest relevant data into Datalakes, Datawarehouses to support analytics and reporting
• Track and report status on progress of development activities in team forums such as stand-up meetings
• Participate in planning activities such as Feature Grooming and PI Planning
• Driving innovation through delivery focused on reducing processing costs, automation of CI/CD pipelines and effective utilization of Azure/AWS resources
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• 4+ years of experience in Software development
• 2+ years of experience Python & SQL
• 2+ years of experience in SPARK or Pyspark
• Experience with databases such as Postgres, Hive etc.
• Experience with Delta tables and SQL Querying
• Experience with Streaming Platforms - Kafka Preferable
• Programming Languages - Solid experience with languages like Python for data analysis and application development
• Orchestration - Solid experience in Databricks for orchestrating data workflows, data movement, and transformation
• Cloud Computing - Solid understanding of Cloud platforms & Services (Preferably AWS)
• Solid understanding of data strategy and data pipelines
• Solid understanding of data quality strategies
• Proficiency in CI/CD pipeline tools and agile methodologies
• Proven excellent problem-solving skills and the ability to think critically and creatively
  • Demonstrated solid communication and interpersonal skills, with the ability to work