QA was largely done manually until a few months ago.
Today, customer support teams are turning to Auto QA to revolutionize decade-long practices.
However, simply automating certain tasks for your support team doesn’t mean you’re fully reaping the benefits of true Auto QA. The concept of Auto QA is still new and there’s not much out there on how to apply it within the customer service space.
The purpose of this article is to walk you through exactly what Auto QA implies, what its role is, and how you can get started with it.
Let’s start with the basics!
What is Auto QA in customer service?
Automated quality assurance (or Auto QA) is the use of tools, algorithms, or processes to identify issues affecting the quality of customer support services. Sentiment analysis, empathy analysis, automatic transcriptions, speech recognition, automatic ticket rating and tagging, and workflow automation are just a couple of the core use cases Auto QA can help support teams with.
It’s also important to understand what Auto QA is not:
- It’s not a complete replacement for human oversight and intuition
- It’s not a standalone solution
- It’s not just about automating repetitive tasks
Why is Auto QA important?
Sure, we’re already used to our classic manual processes for handling customer support quality assurance. But these manual actions won’t go away. At least not all of them.
Auto QA is meant to be used to process a high volume of interactions in a short time, whereas manual QA will always struggle to keep up with large datasets. And it’s more than just a matter of load. Automated quality assurance can also help you save time because it operates thousands of times faster than humans would.
The outcome?
You’ll be able to speed up the decision making process and take faster corrective actions to prevent errors.
As a bonus, you can get to this level of speed without needing a lot of human resources. Plus, the insights you gain stay consistent, unaffected by individual skills, experience, or mood.
And there’s another facet to consider: Doing things continuously. Automated quality assurance enables managers to run ongoing, comprehensive ticket analyses instead of limiting their evaluations to specific time frames. This proactive approach lets you identify agent performance issues early, preventing them from escalating into costly problems.
Manual quality assurance “boasts” one big flaw when it comes to the quality of the data you work with. It does not provide comprehensive data unless you have a large QA team, which is almost always impossible as it’s best to always hire from within the company. With automated quality assurance, you no longer have to wait for QA raters to assess tickets or QA admins to compile all this information. Simply decide on which metrics you want to rely on for your insights or use a feature like AI Summary to bring it all together in an instant.
The biggest asset Auto QA brings to your team is the actual time it gives back to your managers. By supplementing manual work and relieving the workload associated with repetitive tasks, customer support leaders and QA raters will be able to concentrate on complex scenarios or tasks that demand all of their attention.
That said, activities such as running agent feedback meetings or coaching sessions will never be fully replaced by an automated solution. Instead, you’ll be working with Auto QA to aid your manual efforts and do more of what you love.
And we’re here to show you where you can start.
How to get ahead in the Auto QA game
Using Auto QA in customer support will help you tackle a variety of use cases that would otherwise drain your time. But you need to know how to approach each use case first.
Let’s explore a complete QA process in the exact order you’d tackle it using Kaizo’s Auto QA.
The pre-rating stage
To get started with the Auto QA feature, you’ll need to first make sure that the External Samurai is enabled and check if you have the correct role and permissions assigned to you. This is where some human intervention is necessary to set up the correct Auto QA prompts. Rating will then be done based on the criteria you’ve selected at this stage.
This implies choosing which criteria to automate. Popular options include language-related criteria like Language or Grammar. However, you’ll want a more in-depth focus by picking procedure-related criteria that go beyond soft skills. This would allow you to identify scenarios such as cases where the refund policy was violated, improper escalation initiation, or simply a slacking reply.
➡️ Download our free ebook to look at common mistakes to avoid if you want to improve your scorecards.
Next, you’ll have to write down clear instructions for Samurai, Kaizo’s AI-powered virtual assistant, to know when it should pick a certain criterion.
This guarantees that the details you provided align with existing company policies (i.e. what your human raters are already doing). At this stage, you’re also expected to establish criteria outlining when and how language should be used, ensuring alignment with your brand image.
The process mimics instructing a new QA Expert during onboarding on how to best handle scorecards.
Don’t forget to test out these instructions before you run them on all your tickets. While Auto QA does allow you to rate everything, you want to make sure the final results are up to par with your expectations.
Here’s an example of successful criterion instructions:
Not sure where to start? You can use the Auto-generate instructions function to give you a starting draft and customize it to your needs. As a general rule, the more nuanced this is, the better the GPT model is going to respond.
For accurate testing, Samurai analyzes past tickets your human QA raters have already covered. Only this time, the AI system will use the new criteria you’ve just introduced. The more similar the results between the way the humans and the machine rated a ticket, the higher the match percentage.
Remember: You can improve match percentages by giving clear instructions to Samurai, taking into account all edge cases, and making sure the correct criterion is marked.
Once the match percentage is high enough, your QA team is ready to use the Auto QA feature for ticket evaluations.
The rating stage
Time to let Auto QA rate tickets in your place. QA raters simply need to head over to their QA Scorecard and click on the Auto QA button.
At this stage, the automated quality assurance process can run at the same time as the manual one. In other words, Samurai will rate the specific criteria you chose to be automatically rated within the tickets, the ones you want to rate manually will remain untouched.
In the end, all your QA raters have to do is review the notes Samurai generated and make changes where needed. Note they’re still responsible for ensuring the automated criteria are giving correct results.
This is a small intervention humans need to make to ensure result accuracy. In return, they’ll be able to get as many as 100% of tickets rated based on the criteria in the prompt. All this, while sticking to your established procedures and workflows.
Note: It’s best to use the auto-generated results to identify critical errors but look into them yourself. While auto-generated insights quickly scan through large ticket sets, identifying potential errors much faster than manual inspection, they might not always fully understand the nuances of a specific case or voice tone. That applies to any ticket intricacy that requires human judgment.
What else can you automate at this point?
Besides rating the quality of your tickets in terms of language, soft skills, and procedure, you can leverage several AI-based automation features to dig further into the quality of agent performance.
Agent Empathy Score
Kaizo’s AI-driven Agent Empathy Score enhances customer support quality by analyzing and tracking empathy levels agents display during interactions. With AI technology, Kaizo can evaluate all of your agent-user interactions more objectively and consistently. In turn, you can use these empathy scores to offer feedback and help agents identify areas for improvement when it comes to their empathy skills.
Customer Sentiment Analysis
The Customer Sentiment metric will give you an idea of the customer’s overall experience while also providing insights into where things might have gone wrong in the past. Samurai automatically calculates this metric to enhance the accuracy of your customer sentiment analysis, help you identify issues early, and pinpoint areas of strength or where extra training is needed for support agents.
AI Summary
Ever wanted a concise summary of every conversation from the get-go? AI Summary leverages AI to condense conversations within a ticket and generate them on demand. This will save QA experts time while providing a holistic view of the ticket that, alongside the Customer Sentiment and Agent Empathy Scores, will allow you to fully understand conversations in an instant. Additionally, summaries can be used to identify critical tickets you can then decide to further look into.
Phone Call Transcriptions
Phone call transcriptions use AI to automatically transcribe calls in over 50 languages. Having the transcriptions while listening to the call lets managers take quicker notes, spot compliance errors, and save time since they won’t have to endlessly replay the recording to fully grasp it.
📝 Save the time you’d waste on call listening! Kaizo’s Samurai AI extension automatically transcribes and summarizes tickets so QA experts and team leads can make informed decisions and streamline their evaluation process. Try Samurai!
Next steps: The feedback stage
Auto QA’s benefits go beyond the tickets per se. It will also give you a 360-degree view of agent performance. Once the ticket results are in, you can enable Samurai to provide agents with automated feedback on their performance.
On top of that, you can use Kaizo Missions to optimize goal setup with AI-powered recommendations and generate personalized and challenging missions for agents.
Kaizo’s AI can generate Missions based on your agents’ own performance. You can then use these to define specific and measurable metrics that can be used to gauge agents’ performance in areas related to empathy, customer satisfaction, and teamwork.
To make sure these goals align with your business objectives, Missions ensure that the generated goals go beyond repetitive tasks so agents can contribute to organizational targets, foster collaboration, and enhance customer satisfaction.
Go one step further and simplify your performance evaluation process by:
- Tracking personal or team achievements directly inside Zendesk or Salesforce
- Getting better access to relevant data within your work environment
- Building a data-driven culture by providing agents with real-time insights into their performance, encouraging accountability, and fostering self-motivation
Try Kaizo for free on Zendesk or Salesforce to turn your support workforce into your leading strategic advantage!
