Data Engineer
globallogic
Job Description
Lead the data engineering team, owning the development and maintenance of data pipelines, data lakes, and data warehouses across structured and unstructured data sources.
Architect and implement scalable, fault-tolerant, and secure data platforms in cloud environments (AWS, Azure, or GCP).
Design and optimize ETL/ELT processes using modern tools and technologies (e.g., Apache Spark, Apache Airflow, dbt, Kafka, Glue).
Build and maintain real-time and batch data processing systems, ensuring data quality, accuracy, and lineage.
Partner with data scientists, analysts, and business teams to enable data democratization and self-service analytics.
Implement data governance, security, compliance (GDPR, HIPAA, etc.), and metadata management frameworks.
Ensure platform performance, scalability, and cost-efficiency through infrastructure optimization and capacity planning.
Lead technical design reviews, provide architectural guidance, and mentor data engineers on best practices and emerging technologies.
Drive innovation by evaluating and adopting new data tools, cloud services, and architectural patterns.
Collaborate with DevOps, Product, and Security teams to establish CI/CD pipelines, data access controls, and incident response strategies.
Document architecture, data flows, and processes to support knowledge sharing, onboarding, and auditing.
Track key data engineering KPIs such as pipeline latency, data freshness, system availability, and throughput.