Data Engineer
bt
Job Description
Create, develop, and maintain end‑to‑end data pipelines using Ab Initio (GDE, PDL, Conduct>IT, Express>IT).
Build scalable batch and streaming workloads using Apache Spark (PySpark/Scala).
Develop and orchestrate real-time/near‑real‑time data ingestion flows using Apache NiFi.
Optimize ETL/ELT pipelines for performance and efficiency.
Perform tuning and bottleneck analysis for Spark, NiFi, and Ab Initio.
Deliver demos and knowledge sharing to stakeholders.
Enhance automated testing and deployments through CI/CD.
Continuously learn through internal and external training.