Senior Software Engineer
cgi
Job Description
Design, develop, and maintain scalable **data pipelines** using PySpark, Hadoop, Hive, Python, and SQL.
* Develop and optimize **PySpark applications** for large-scale data processing.
* Work with **Hadoop/HDFS and Hive** for distributed data storage and processing.
* Write complex and optimized **SQL queries** for data extraction, transformation, and validation.
* Perform data cleansing, transformation, aggregation, and validation.
* Troubleshoot data pipeline failures and performance issues.
* Participate in code reviews and follow established coding and development standards.
* Use **GitHub** for source-code management, version control, branching, and collaboration.
* Work closely with Business Analysts, Technical Leads, and other engineering teams to understand requirements and deliver solutions.
* Follow banking-domain data security, compliance, and governance standards.
* Support production deployments, incident resolution, and continuous improvement of data pipelines.