Data Engineer

barclays

pune NM Years Exp Posted 2h ago

Job Description

  • Hands on experience in pyspark and strong knowledge on Dataframes, RDD and SparkSQL

  • Hands on Experience in developing, testing and maintaining applications on AWS Cloud.

  • Strong hold on AWS Data Analytics Technology Stack (Glue, S3, Lambda, Lake formation, Athena)

  • Design and implement scalable and efficient data transformation/storage solutions using Snowflake.

  • Experience in Data ingestion to Snowflake for different storage format such Parquet, Iceberg, JSON, CSV etc

  • Experience in using DBT (Data Build Tool) with snowflake for ELT pipeline development.

  • Experience in Writing advanced SQL and PL SQL programs.

  • Hands On Experience for building reusable components using Snowflake and AWS Tools/Technology

  • Should have worked at least on two major project implementations.

  • Exposure to data governance or lineage tools such as Immuta and Alation is added advantage.

  • Experience in using Orchestration tools such as Apache Airflow or Snowflake Tasks is added advantage.

  • Knowledge on Abinitio ETL tool is a plus

 

Some other highly valued skills may include:

  • Ability to engage with Stakeholders, elicit requirements/ user stories and translate requirements into ETL components

  • Ability to understand the infrastructure setup and be able to provide solutions either individually or working with teams.

  • Good knowledge of Data Marts and Data Warehousing concepts.

  • Resource should possess good analytical and Interpersonal skills.

    • Implement Cloud based Enterprise data warehouse with multiple data platform along with Snowflake and NoSQL environment to build data movement strategy.