Job Description:
Required a results-driven Senior Data Engineer with experience in designing, developing, and delivering enterprise-scale data engineering solutions across data warehousing, big data, and cloud platforms.
Key Responsibilities & Leadership -
- Design and implement enterprise-scale data architectures, including Data Warehouses and Data Lakes.
- Build and maintain scalable ETL/ELT pipelines for analytics, reporting, and operational workloads.
- Lead end-to-end data migration and modernisation programs from legacy systems to Hadoop, Spark, Databricks, and AWS platforms using Agile methodology.
- Define and execute migration strategies, including data analysis, mapping, transformation, integration, validation, reconciliation, and production deployment.
- Design and optimise data models, SQL queries, and Spark workloads to improve performance, scalability, and cost efficiency.
- Develop automation solutions to reduce manual effort and improve operational efficiency.
- Ensure data quality, consistency, governance, and compliance across enterprise data platforms.
- Perform data profiling, validation, and reconciliation to ensure accuracy and reliability of data assets.
- Lead technical design reviews, architecture discussions, and code reviews to ensure adherence to best practices and standards.
- Manage full project lifecycle delivery, including requirements gathering, design, development, testing, deployment, and production support.
- Collaborate with architects, business analysts, product owners, and clients to translate business requirements into technical solutions.
- Lead and mentor data engineering teams, including task allocation, skill development, and performance management.
- Identify and resolve technical and delivery blockers to ensure timely project execution.
- Manage stakeholder expectations and communicate technical solutions clearly to both technical and non-technical audiences.
- Deliver both new development and production support projects in enterprise environments.
- Continuously improve systems by identifying opportunities for performance, scalability, and cost optimisation.
- Adapt quickly to new technologies, including AI-enabled data platforms, modern data engineering tools, and evolving coding methodologies.
- Apply modern coding practices, including modular design, reusable components, version control standards, and CI/CD-aligned development approaches.
- Maintain a strong continuous learning mindset and ability to work across evolving data and AI-driven ecosystems.
- Strong domain expertise in banking domain.
Technical Skills -
- Data Engineering & Platforms: Data Warehousing, Data Lakes, ETL/ELT, Data Pipelines, Data Modelling, Data Profiling, Data Migration, Data Modernisation, Data Quality, Data Governance
- Databases: SQL, Teradata, Oracle, DB2, Vertica, PostgreSQL, Amazon Redshift
- Big Data Technologies: Hadoop ecosystem (HDFS, Hive, Hive SQL), Apache Spark, Spark SQL, PySpark
- Cloud & Modern Data Platforms: AWS/Azure/GCP, Databricks/Snowflake/DBT
- Programming & Scripting: Python, UNIX Shell Scripting
CI/CD: GitHub/ Jenkins
Pay: $100,000.00 – $150,000.00 per year
Work Location: In person
