Data Engineer
vanguardjobs
Job Description
-
Develop and maintain scalable ETL processes using Python and SQL to support data integration and analytics requirements.
-
Design and implement data models, database structures, and data pipelines to ensure high performance, scalability, and data quality.
-
Integrate data from multiple sources, including APIs, third-party services, and on-premises/cloud databases, to create reliable datasets for business consumption.
-
Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver effective data solutions.
-
Develop automated workflows and data processing solutions using Python and SQL to streamline data ingestion and transformation.
-
Support cloud-based data platforms and contribute to building scalable and efficient data solutions.
-
Maintain and improve data quality through validation, monitoring, and data governance best practices.
-
Thoroughly test code to ensure accuracy, reliability, and alignment with business requirements.
-
Provide tool and data support to business users and team members to ensure effective use of data solutions.
-
Participate in continuous improvement initiatives, adopting new technologies and approaches to solve business problems.
Qualifications and Skills
-
Minimum 3 years of relevant work experience, with at least 1 year in designing and maintaining data pipelines, ETL processes and database architectures.
-
Bachelor's degree (B.E./B.Tech) in Computer Science or IT, or Diploma in Data Science, Statistics, or related field.
-
Strong software engineering background with Python programming experience preferred.
-
Strong proficiency in SQL and experience working with relational databases such as PostgreSQL and/or MySQL.
-
Good understanding of data modeling concepts and database design principles.
-
Demonstrated ability to learn quickly, solve problems, and work with a curious, passionate, and go-getter mindset.
-
Good to Have: Experience with Java, C++, Apache Spark, AWS Glue, or other data engineering technologies and exposure to cloud platforms and modern data processing frameworks.
-