Senior/Lead Quality Engineer - Chatbot
- Role
- QA
- Experience
- Lead
- Employment
- Full-time
Open to Anywhere in Americas & Europe · UTC-5…UTC+1. Set where you work from to check your eligibility.
No BS summary
Senior/Lead Quality Engineer for chatbot and LLM-powered products. Needs strong hands-on QA/SDET/test automation experience, exposure to conversational AI/RAG/non-deterministic systems, and proficiency in at least one programming language. Remote only from the UK, Ireland, Sweden, Netherlands, Canada, Spain, the Americas, or Europe within UTC-5 to UTC+1.
What we are building
Mimica's mission is to empower enterprises, teams, and individuals to reclaim their most precious resource — time and work more efficiently, with greater purpose and impact.
Our AI-powered task mining observes employee actions across the desktop and categorizes them into detailed process maps. Mimica’s process intelligence highlights inefficiencies, prioritizes improvements based on ROI, recommends the optimal technology for automation (RPA, intelligent document processing, GenAI), and provides a blueprint for building new automations and transforming work.
Visit our Career Page to learn more.
Your mission
We are looking for a Quality Engineer to help us build a faster, and more reliable testing and release processes for one of our latest product engineering teams. You will work closely with Software and Machine Learning Engineers to establish the quality foundations for a product characterised by dynamic, non-deterministic outputs.
The team is a newly formed multidisciplinary team of 5 responsible for the conversational AI components across our WebApp. As the team’s founding Quality Engineer, you will play a central role in defining how we approach quality from the outset. You will introduce thoughtful, pragmatic practices that increase confidence in releases, reduce remediation and rollback work, and enable the team to move quickly without compromising user trust or product reliability.
You will have meaningful ownership from day one: exploring testing approaches, identifying quality gaps, and helping shape the systems and processes that support sustainable delivery. This is an opportunity for someone who thrives in ambiguity, combines hands-on technical testing with experimentation, and wants to define quality engineering at the intersection of software development and LLM-powered products.
Part of your day-to-day
- Develop and evolve quality practices, test strategy, and release processes for the Chatbot product.
- Help establish and run an output-evaluation framework for assessing Chatbot quality, reliability, and regressions.
- Test dynamic and non-deterministic LLM-based systems, designing pragmatic approaches for validating behaviour beyond conventional deterministic test cases.
- Identify, test, and improve guardrails for AI-specific risks, including prompt-injection scenarios and unsafe or unexpected model behaviour.
- Partner with engineers through code reviews, technical discussions, and collaborative debugging to improve quality earlier in the development lifecycle.
- Investigate current testing gaps, research relevant methods and tools, and make evidence-based recommendations to the team.
- Design proof-of-concepts for new testing approaches or tooling, then help productionise the solutions that create the most value.
- Support the evolution of release testing, including clearer quality signals and release-readiness criteria.
- Contribute performance-testing thinking for LLM-powered workflows where relevant.
- Share knowledge, facilitate technical discussions, and build the Quality Chapter across the wider engineering organisation.
Requirements / What we’re looking for
- Strong hands-on experience in software quality engineering, SDET, test automation, or a closely related engineering role.
- Experience testing chatbot, conversational AI, RAG, LLM-powered, or other systems with dynamic and non-deterministic behaviour; even early practical exposure is valuable.
- Proficiency in at least one programming language and enough technical depth to understand production code, contribute to code reviews, and collaborate effectively with software engineers.
- An experimental, investigative mindset: you can explore an uncertain problem space, assess alternatives, and turn findings into practical improvements.
- Clear written and verbal communication, including the ability to explain technical concepts, testing risks, design choices, and trade-offs.
- Strong collaboration skills and a pragmatic approach to working with Product, Engineering, and ML colleagues.
- Good judgement in prioritising quality work based on risk, user impact, and delivery goals.
Bonus / Nice to have
- Experience with AI security and safety testing.
- Experience testing conversational AI, chatbot platforms, or agentic workflows.
- Experience as the first Quality Engineer in a team.
- Experience mentoring engineers or influencing testing practices across a cross-functional team.
- Keeps pace with industry developments and actively fosters knowledge sharing by bringing new tech trends and methodology back to the team
Hiring Process
Stage 1 | Recruiter Screen (30 mins)
Stage 2 | HM Interview (45 mins)
Stage 3 | Take-home + Live Follow-up (60 mins) with members from the Chatbot team + Quality Chapter.
Stage 4 | Behavioral Interview (60 mins) with Co-founder/CTO + Lead Quality Engineer
Location
This is a fully remote position. You can be based anywhere in the UK, Americas, or Europe within a UTC-5 to UTC+1 timezone.
What we offer
💰 Generous compensation + stock options - aligned with our internal framework, market data, and individual skills.
🏢 Distributed work: Work from anywhere - fully remote, in our hubs, or a mix.
💻 Company-issued laptop, remote setup stipend, and co-working budget
📍 Flexible schedules and location
☀️ Ample paid time off, in addition to local public holidays
🍼 Enhanced parental leave
🧘♀️ Health & retirement benefits
📖 Annual learning & development budget
🌴 Annual workaways and regular virtual & in-person socials
🌍 Opportunity to contribute to groundbreaking projects that shape the future of work
Note: Some benefits may vary depending on location and role
⚠️ Mimica will only contact candidates from an @mimica.ai email address. We do not request banking or sensitive personal information during the recruiting process.
What you'll do
- Develop and evolve quality practices, test strategy, and release processes for the Chatbot product.
- Help establish and run an output-evaluation framework for assessing Chatbot quality, reliability, and regressions.
- Test dynamic and non-deterministic LLM-based systems, designing pragmatic approaches for validating behaviour beyond conventional deterministic test cases.
- Identify, test, and improve guardrails for AI-specific risks, including prompt-injection scenarios and unsafe or unexpected model behaviour.
- Partner with engineers through code reviews, technical discussions, and collaborative debugging to improve quality earlier in the development lifecycle.
- Investigate current testing gaps, research relevant methods and tools, and make evidence-based recommendations to the team.
- Design proof-of-concepts for new testing approaches or tooling, then help productionise the solutions that create the most value.
- Support the evolution of release testing, including clearer quality signals and release-readiness criteria.
- Contribute performance-testing thinking for LLM-powered workflows where relevant.
- Share knowledge, facilitate technical discussions, and build the Quality Chapter across the wider engineering organisation.
What they require
- Strong hands-on experience in software quality engineering, SDET, test automation, or a closely related engineering role.
- Experience testing chatbot, conversational AI, RAG, LLM-powered, or other systems with dynamic and non-deterministic behaviour; even early practical exposure is valuable.
- Proficiency in at least one programming language and enough technical depth to understand production code, contribute to code reviews, and collaborate effectively with software engineers.
- An experimental, investigative mindset: you can explore an uncertain problem space, assess alternatives, and turn findings into practical improvements.
- Clear written and verbal communication, including the ability to explain technical concepts, testing risks, design choices, and trade-offs.
- Strong collaboration skills and a pragmatic approach to working with Product, Engineering, and ML colleagues.
- Good judgement in prioritising quality work based on risk, user impact, and delivery goals.
- Preferred: Experience with AI security and safety testing.
- Preferred: Experience testing conversational AI, chatbot platforms, or agentic workflows.
- Preferred: Experience as the first Quality Engineer in a team.
- Preferred: Experience mentoring engineers or influencing testing practices across a cross-functional team.
- Preferred: Keeps pace with industry developments and actively fosters knowledge sharing by bringing new tech trends and methodology back to the team.
Benefits
- Generous compensation + stock options aligned with internal framework, market data, and individual skills.
- Distributed work: work from anywhere, fully remote, in hubs, or a mix.
- Company-issued laptop.
- Remote setup stipend.
- Co-working budget.
- Flexible schedules and location.
- Ample paid time off, in addition to local public holidays.
- Enhanced parental leave.
- Health and retirement benefits.
- Annual learning and development budget.
- Annual workaways and regular virtual and in-person socials.
- Opportunity to contribute to projects that shape the future of work.