Role Summary:
We are seeking an experienced Automation Engineer with a strong Data Engineering background to join our growing Data Platform team. This is a hands-on engineering role focused on building automation frameworks, validating cloud-based data pipelines, and ensuring data quality across modern AWS data platforms. The ideal candidate will have expertise in AWS Data Services, Snowflake, Apache Airflow, Python, Java, and CI/CD, with proven experience in data pipeline automation and testing.
Key Responsibilities
- Design, develop, and maintain automation frameworks for cloud-based data platforms and data pipelines.
- Build automated validation solutions for data ingestion, transformation, ETL/ELT, and reporting workflows.
- Validate data quality, integrity, and consistency across AWS and Snowflake environments.
- Develop automation scripts using Python and Java for testing data pipelines and APIs.
- Perform API testing, data validation, and SQL-based testing for enterprise data platforms.
- Support CI/CD pipelines using GitLab and implement DevOps best practices.
- Test and validate Kafka-based data streaming and event-driven architectures.
- Implement BDD automation frameworks using Cucumber.
- Collaborate with Data Engineers, Developers, Architects, and Business stakeholders to ensure platform reliability and high-quality data delivery.
- Troubleshoot automation issues and continuously improve testing processes and framework performance.
Required Skills & Experience
- 7+ years of experience in Automation Engineering with a strong Data Engineering background.
- Hands-on experience with AWS Data Services, including:
- Amazon S3
- AWS Glue
- AWS Lambda
- Amazon Redshift
- Amazon Athena
- Strong experience with Snowflake and Apache Airflow.
- Proficiency in Python and Java.
- Experience developing API automation frameworks.
- Strong knowledge of ETL/ELT testing, data validation, and SQL.
- Experience with Kafka and data pipeline testing.
- Hands-on experience with GitLab CI/CD and DevOps practices.
- Experience implementing BDD frameworks using Cucumber.
- Strong analytical, troubleshooting, and problem-solving skills.
Preferred Skills
- Experience with enterprise cloud-native data platforms.
- Knowledge of distributed data processing and modern data architectures.
- Experience working in Agile/Scrum environments.
- Familiarity with data governance and data quality best practices.
Pay: From $120,000.00 per year
Work Location: Hybrid remote in Sydney NSW
