Data Engineer

Translate data requirements for new data quality controls and thresholds into technical designs and specifications
• Create automated processes to measure data quality, failures, and errors
• Analyze existing systems and data pipes to identify areas for improvement or opportunities to enhance
• Continue to help evolve and improve the data quality processes and templates
• Profile data sources to understand existing quality issues in the ETL processes
• Collaborate on data modeling tasks including recommending and evaluating designs based on Kimball dimensional data modeling practices
• Aggregate and combine raw information from different data sources
• Explore ways to enhance data quality and reliability including helping with the development of support and remediation processes
• Identify opportunities for data acquisition
• Develop analytical tools and processes to measure and report data quality
• Help adjust and tune existing CI/CD framework and processes (Azure DevOps) as needed to improve data quality
• Assist Senior BI Engineer and Senior Data Engineer on innovative technology POCs

Skills

• Bachelor's Degree or higher in a technical field such as Engineering, Computer Science, etc.
• Minimum 5+ years of experience designing, building, and supporting data pipes
• Minimum 2+ years of experience creating automated solutions for measuring data quality for ETL processes
• Minimum 1+ year of recent experience working with the following Azure Data Services: Azure Data Catalog, Azure Blob Storage, Azure Data Lake Storage, Azure Synapse Analytics, Azure Analysis Services, Azure Data Factory
• Strong background working with Microsoft SQL Server databases (preferably 2016)
• Understanding of Power BI and DAX functions (do not need to be a report developer or data visualization expert)
• Able to read and edit PowerShell scripts
• Prior experience working with CI/CD processes using Azure DevOps
• Strong interpersonal and customer relationship skills
• Ability to communicate to multiple levels of the organization and with non-technical partners
• Experience reading and editing Python scripts