Role Summary
We are looking for a Python Data Integration Engineer to design, build, and maintain scalable data pipelines and cloud-native services supporting AI-driven applications. You will work across Azure, Snowflake, and enterprise integrations to deliver reliable, production-ready solutions while enhancing observability, automation, and platform performance.
Key Responsibilities
- Design, develop, and maintain data integration pipelines across multiple systems and platforms.
- Build and enhance Azure Durable Functions for workflow orchestration and pipeline automation.
- Develop resilient, scalable, and production-ready cloud services.
- Monitor, troubleshoot, and optimise AI and data processing pipelines.
- Build and maintain ML evaluation infrastructure and automation workflows.
- Develop operational dashboards, logging, and monitoring using Application Insights and Grafana.
- Implement automated testing, CI/CD pipelines, and engineering best practices.
- Integrate enterprise systems, cloud storage, and data warehouses.
- Maintain technical documentation and support continuous platform improvements.
Required Skills & Experience
- Strong Python development experience with type hints, testing, and modern development practices.
- Experience building cloud-native applications on Microsoft Azure (Functions, Durable Functions, Managed Identity).
- Strong SQL skills and experience with data warehouses such as Snowflake.
- Experience building ETL/data integration pipelines.
- Knowledge of CI/CD, Git workflows, logging, monitoring, and observability.
- Experience working with production systems requiring reliability, auditability, and security.
- Strong analytical and problem-solving skills.
Desirable Skills
- Azure OpenAI, Document Intelligence, OCR, or AI/LLM application development.
- SAP or enterprise ERP integration (OData, OAuth2).
- Infrastructure as Code, Azure Key Vault, and workload identity.
- Power BI, MLflow, Azure ML, and MLOps practices.
- Experience with AI-assisted development tools such as GitHub Copilot.
Technology Stack
- Languages & Frameworks: Python 3.12, Pydantic, pytest, mypy, Ruff
- Cloud: Microsoft Azure Functions, Durable Functions, Azure OpenAI, Azure Document Intelligence, Cosmos DB, Blob Storage, Key Vault
- Data: Snowflake, SQL, Cosmos DB, SAP BTP OData
- Observability: OpenTelemetry, Application Insights, Grafana LGTM
- DevOps: GitLab CI/CD, Git, DevContainers
- Methodologies: Agile, CI/CD, Data Engineering, AI-Assisted Development
Pay: From $100,000.00 per year
Work Location: Hybrid remote in Perth WA
