Quality Assurance Engineer – AI Conversation Agent

About the role

We are looking for a dedicated Quality Assurance Engineer to own the end‑to‑end quality of our AI Conversation Agent, one of the most complex products in our portfolio. You will design realistic test scenarios, drive non‑functional testing, and build the automation infrastructure that enables rapid, defect‑free releases.

Key responsibilities

• Design and execute customer‑oriented test cases that simulate real conversations, including interruptions, overlapping speech, background noise, accents, and off‑script digressions.
• Translate customer‑reported issues into reproducible test cases and maintain regression suites.
• Own systematic non‑functional testing such as interruption handling, noise robustness, weak‑network conditions, and sustained‑load stability.
• Build and operate automated call‑testing and load‑testing platforms that simulate thousands of concurrent calls and monitor key performance metrics.
• Evaluate core experience metrics (ASR accuracy, TTS naturalness, end‑to‑end latency) and drive issues to closure with engineering.
• Establish end‑to‑end logging and tracing across the conversation pipeline to accelerate root‑cause analysis.
• Develop maintainable automated test frameworks (Python/Java) integrated into CI/CD pipelines for functional and non‑functional scenarios.
• Define, track, and report quality metrics to make release quality visible and data‑driven for stakeholders.

Required profile

• 4+ years of QA/testing experience, preferably with voice, AI, or real‑time communication products.
• Strong analytical mindset with a talent for uncovering edge‑case failures.
• Patience and rigor for detailed manual exploratory testing.
• Ability to work independently as the dedicated QA owner while collaborating closely with engineering and product teams.

Required skills

• Python
• Java
• Automated test framework development
• CI/CD integration
• Logging, tracing, and dashboarding tools
• Load‑testing platforms
• Experience with VoIP or WebRTC protocols
• Familiarity with ASR, TTS, and LLM‑based conversation systems