Data Engineer
micro1
Job Description
- Design, develop, and maintain scalable ETL pipelines.
- Collect, clean, transform, and organize structured and unstructured data.
- Conduct exploratory data analysis to identify trends, patterns, anomalies, and data-quality issues.
- Collaborate with researchers, data scientists, and engineering teams to prepare datasets for AI and machine-learning initiatives.
- Develop and maintain data models, database schemas, and data-storage solutions.
- Write and optimize SQL queries for data extraction, transformation, and analysis.
- Ensure data quality, integrity, consistency, and security across pipelines and databases.
- Automate data validation, reporting, and recurring data processing workflows.
- Document data pipelines, processes, and technical decisions.
- Troubleshoot pipeline failures and resolve data-related issues.