Lead Data Engineer

reczee

Bengaluru 7 Years Exp Posted 24d ago

Design and scale end-to-end data platforms supporting batch and real-time workloads.
Build scalable ETL/ELT and CDC pipelines using Kafka, Debezium, Spark/Flink, and Airflow.
Architect data lakes/lakehouses using S3, Parquet, Iceberg/Delta Lake.
Design and optimise ClickHouse/Redshift/BigQuery architectures, including schema design, aggregation, sharding, and replication.
Build multi-tenant data architectures with strong isolation, security, and compliance.
Drive data observability, lineage, quality, and disaster recovery across critical pipelines.
Define infrastructure using Terraform and drive compute/storage and cost optimisation.
Partner with platform and security teams on AWS, VPC, IAM, encryption, and data security.
Mentor engineers and establish data architecture standards and best practices.