Company Description
Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, infrastructure, cybersecurity, DevSecOps, software engineering, and digital transformation programs.
We partner with government and enterprise organisations to build high-performing technology teams that deliver secure, resilient, and scalable solutions within complex and highly regulated environments.
The Role
We are seeking an experienced Prompt Engineer with strong expertise in Large Language Models (LLMs), prompt engineering, AI evaluation frameworks, and enterprise AI solution development.
This role will focus on designing, optimising, and governing prompt systems for production-grade AI applications. You will work closely with product teams, software engineers, data scientists, and AI specialists to improve model performance, drive AI quality standards, and deliver reliable, secure, and scalable AI-powered solutions.
Key Responsibilities
- Design, develop, and maintain prompt frameworks including system prompts, few-shot examples, role-based prompts, and reasoning workflows.
- Build and manage automated evaluation frameworks to measure model performance, accuracy, latency, and regression across releases.
- Conduct structured A/B testing across prompt variations, model versions, and configuration settings to optimise AI outcomes.
- Translate business requirements and complex edge cases into effective prompts, personas, constraints, and guardrails.
- Partner with engineering and product teams to determine when prompt engineering, fine-tuning, RAG architectures, or alternative AI approaches are required.
- Develop and maintain a centralised prompt repository including version control, documentation, and performance benchmarks.
- Lead red-teaming and adversarial testing exercises to identify jailbreak risks, hallucinations, and model vulnerabilities.
- Define evaluation criteria, annotation guidelines, and quality standards for AI-generated outputs.
- Drive continuous improvement initiatives through prompt optimisation, benchmarking, experimentation, and model evaluation.
- Manage context handling, prompt orchestration, and workflow design for complex AI use cases.
- Improve AI response quality through systematic assessment, tuning, testing, and refinement.
- Provide guidance and mentorship on prompt engineering best practices and AI evaluation methodologies.
- Present prompt strategies, performance insights, benchmark results, and recommendations to technical and business stakeholders.
- Support the delivery of enterprise-grade AI applications that are secure, scalable, and production ready.
What You'll Need
- Strong hands-on experience with prompt engineering and Large Language Models (LLMs) in production environments.
- Experience designing and optimising prompts using techniques such as zero-shot, few-shot, chain-of-thought, and role-based prompting.
- Strong programming skills in Python, C#, JavaScript, or another modern programming language.
- Experience building AI-powered applications, automation workflows, APIs, or evaluation pipelines.
- Knowledge of model evaluation, benchmarking, experimentation, and performance optimisation practices.
- Experience with prompt orchestration frameworks and modern AI development toolchains.
- Strong understanding of Retrieval-Augmented Generation (RAG), vector databases, and AI agent architectures.
- Experience working with REST APIs, JSON, system integrations, and enterprise software development practices.
- Familiarity with version control systems, CI/CD pipelines, and cloud platforms.
- Understanding of AI safety, responsible AI principles, testing methodologies, and governance frameworks.
- Ability to assess and mitigate risks such as hallucinations, prompt injection attacks, and jailbreak vulnerabilities.
- 4-7 years of experience across AI/ML products, NLP, software engineering, technical writing, or related disciplines.
- At least 2 years of direct prompt engineering experience supporting production LLM applications.
- Demonstrated experience owning prompt systems end-to-end, from design and implementation through monitoring and optimisation.
- Strong analytical thinking, problem-solving, and stakeholder engagement capabilities.
- Excellent communication skills with the ability to explain complex AI concepts to technical and non-technical audiences.
Why Join Us?
- Work on cutting-edge Generative AI and Large Language Model initiatives.
- Influence the design and governance of enterprise AI platforms and solutions.
- Collaborate with experienced AI engineers, architects, and product specialists.
- Drive innovation through prompt engineering, model optimisation, and AI experimentation.
- Help shape responsible, scalable, and secure AI capabilities for enterprise and government clients.
- Gain exposure to emerging technologies including RAG, AI agents, vector databases, and advanced LLM architectures.
- Join a consulting culture focused on innovation, continuous learning, and technical excellence.
