Clinical Data Engineer
hirist
Job Description
- Design, build and maintain production ETL pipelines in Databricks / Delta Lake to ingest RWD (registries, claims, EHR extracts) and transform into standard models.
- Implement harmonisation workflows to map incoming RWD to OMOP and to the internal CDISC SDTM canonical model; handle vocabulary mapping, units normalization and provenance.
- Extend the medallion architecture (bronze / silver / gold) patterns with robust validation, lineage, partitioning and performance tuning.
- Develop configurable, input - driven transformation frameworks so clinical experts can drive mapping rules via config files and catalogs.
- Integrate AI / automation components (e.g., model - assisted mapping, NLP for free text) with human - in - the - loop review and confidence scoring.
- Establish testing, CI / CD, monitoring and alerting for ETL jobs and automations; ensure reproducibility, versioning and governance.
- Collaborate with clinical data scientists, data stewards and stakeholders to define requirements, data contracts and success metrics.