- Build real-time data platforms.
- Develop AWS SageMaker solutions.
- Deliver AI and ML-ready pipelines.
Company Description
Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, data, DevSecOps, platform engineering, artificial intelligence, and digital transformation programs.
We partner with major organisations to build high-performing technology teams and deliver innovative cloud-native solutions that improve scalability, operational efficiency, data intelligence, and business outcomes.
The Role
We are seeking an experienced Senior Data Engineer to join a high-performing cloud and data engineering team.
This is a hands-on role focused on designing, building, and optimising enterprise-scale data platforms, ingestion frameworks, and analytics solutions within AWS. You will play a key role in developing real-time and batch data pipelines, enabling advanced analytics and machine learning capabilities, and supporting enterprise data governance initiatives using AWS SageMaker Unified Studio.
The ideal candidate will bring deep expertise across data engineering, AWS cloud services, workflow orchestration, and data integration technologies, with proven experience delivering scalable, secure, and high-performance data solutions in complex enterprise environments.
Key Responsibilities
- Design, build, and maintain enterprise-scale data ingestion, transformation, and processing solutions.
- Develop and support data platforms leveraging AWS SageMaker Unified Studio.
- Configure and manage SageMaker Unified Studio environments, including:
- Develop solutions using SageMaker IDE, JupyterLab, Spaces, and Partner AI applications.
- Design and implement reusable and scalable data ingestion frameworks supporting multiple source systems, APIs, databases, message queues, and file-based integrations.
- Build low-latency, real-time data pipelines using AWS streaming technologies.
- Develop and manage batch data processing solutions for large-scale datasets.
- Design and implement efficient data transformation and enrichment processes for both streaming and batch workloads.
- Develop data processing solutions using Python and AWS SDKs.
- Create and manage workflow orchestration and automation using AWS Managed Apache Airflow.
- Design, deploy, and optimise complex Airflow DAGs supporting enterprise data pipelines.
- Support machine learning and Generative AI initiatives through integration with AWS SageMaker and associated services.
- Implement monitoring, observability, alerting, and operational support capabilities across data platforms.
- Collaborate with data scientists, analysts, architects, DevOps engineers, and business stakeholders to deliver data-driven solutions.
- Ensure solutions align with enterprise security, governance, compliance, and operational standards.
What You'll Need
- Extensive experience as a Senior Data Engineer or Cloud Data Engineer.
- Strong expertise with AWS SageMaker Unified Studio (Discover, Build, Govern).
- Hands-on experience with SageMaker IDE, JupyterLab, and AI-enabled development environments.
- Strong experience building real-time and batch data pipelines using AWS services.
- Expertise with AWS Kinesis, Glue, Spark, Lambda, S3, Redshift, RDS, DynamoDB, and related services.
- Advanced Python development skills for data engineering, automation, and API integration.
- Experience with AWS Managed Apache Airflow and workflow orchestration.
- Strong knowledge of AWS networking, IAM, security, and access controls.
- Experience with relational and NoSQL databases, including Redshift, RDS, DynamoDB, and MongoDB.
- Understanding of data transformation, data modelling, monitoring, observability, and performance optimisation.
- Knowledge of ML, GenAI, and data platform integration patterns.
- Excellent communication, stakeholder management, and problem-solving skills.
Desirable Skills
- Experience developing ML and Generative AI solutions using AWS SageMaker services.
- Familiarity with foundation models, AI platforms, and enterprise AI integration patterns.
- Experience with DataOps, MLOps, or DevOps practices.
- Knowledge of container technologies including Docker and Kubernetes.
- Experience with Infrastructure as Code tools such as Terraform or AWS CloudFormation.
- Knowledge of CI/CD pipelines and automated deployment frameworks.
- Experience working within enterprise-scale cloud migration or data transformation programs.
- Familiarity with Scala or other programming languages used within distributed processing environments.
- AWS certifications across Data Engineering, Machine Learning, Solutions Architecture, or Cloud disciplines.
Why Join Us?
- Work on large-scale cloud data, analytics, and AI initiatives supporting enterprise-critical services.
- Gain exposure to modern AWS data engineering, machine learning, and Generative AI technologies.
- Play a key role in designing next-generation real-time and batch data platforms.
- Collaborate with experienced data engineers, cloud specialists, architects, and AI practitioners.
- Influence enterprise data engineering standards, frameworks, and best practices.
