Tamara runs engineering at a mid-sized lender. Her team ships every two weeks, and for the last year every release has gone the same way. Developers finish on Thursday. The in-house QA group of four tests through Friday and the weekend. Monday morning brings a list of bugs, an argument about which ones block the release, and a quiet sense that nobody is sure how much of the app has really been checked. Then one of the four testers resigns, and the release calendar starts to wobble.
At that point Tamara’s director asks what everyone in her position eventually gets asked: should we just hire a testing company? Tamara’s first move was to ask three people who had solved this differently. The answers showed why “just hire a testing company” is the wrong question, because “outsourced testing” and “managed testing” sound alike and are different things.
Three conversations
Her friend Jorge runs QA in-house at a payments firm. His testers plan, run and report everything. He controls it all, and he also carries the hiring, the training, the tools and the hole left whenever someone resigns. “It works as long as people stay,” he said.
A second contact, Mira, had brought in a vendor who supplies testers by the month. The vendor’s people work from Mira’s plan, under her team’s direction, and she pays for their time. That suits her, because her strategy is clear and she needs extra hands. But the strategy, the coverage and the results are still hers to answer for. Some call this staff augmentation.
The third, Colin, uses what’s called managed testing. His provider takes on the whole testing function. It plans the work, builds and runs the tests, automates the repetitive ones, reports the results and commits to service levels. Colin pays for an outcome, like regression finished within two days of code freeze, not for hours. The responsibility moves along with the money.
What a managed service covers
Colin’s list was typical. A test strategy and plan. Functional and regression testing. Test automation. Performance testing. Some security and accessibility checks on top. Then comes the dull but vital stuff. Test environments and data. A defect tracker. A weekly report: what was tested, what failed, what’s still open. Tamara’s wish list was shorter. Fast, predictable regression. And the repetitive checks taken off her people, so the remaining testers could dig into new features.
How the work is organized
Near Colin’s office sits a small team of the provider’s people, there for daily contact and anything urgent. The bigger group sits a few time zones away. They test while Colin sleeps, so by nine the results have landed. The contract makes promises you can actually measure. How quickly a regression cycle finishes. What share of defects are caught before release. How fast a critical bug gets triaged. How much of the suite is automated by a given date. “Without it in writing,” Colin said, “you’re buying hours and hoping.”
Where AI fits
AI tools suit testing rather well. They draft test cases from requirements, patch automated tests that break when a screen changes, and rank which tests to run first for a given change. Colin’s provider uses them, and his cycles got shorter. What they can’t do is replace people who understand the product. If a provider brings up AI, ask three things: what it does, what it saves, and who checks the output.
How Tamara chose
She wrote three requirements before she called anyone. Regression done within two working days of code freeze. A weekly report her director could read in five minutes. And the existing testers kept for exploratory work, not replaced. Then she asked each provider the same questions.
Which similar systems have you tested, and may I speak to a customer? Who’ll be on my account, and what happens if one of them quits? How do you set up and handle test environments and data, since ours are fiddly? What do the service levels measure, and what happens when you miss them? How do you handle my confidential data? Finally: how would we bring testing back in-house, or move to someone else?
She asked for a short paid pilot with her own application and a real regression cycle. The best provider’s answer to the exit question was the clearest sign of confidence she saw.
What she measured
Before the pilot, Tamara noted three numbers. Days regression took. Bugs that escaped to production over the last three releases. Hours her testers spent re-running the same checks. After the pilot she compared them. Without the baseline, she says, the whole thing would have been a matter of opinion.
What the pilot showed
The pilot ran for one release. The provider took the regression suite, automated the dozen checks Tamara’s testers repeated most often, and ran the rest overnight. By the second week the Monday argument about what had been checked had changed. There was a report, with a list of what was tested, what failed and what was still open, and Tamara’s director read it in five minutes. Her remaining testers, freed from the repeats, found two issues in the new loan screen that nobody had planned to look for. None of this was dramatic. It was just the first release in a year that felt predictable.
What tends to go wrong
Handing over the testing function doesn’t hand over knowledge of your product. Product owners still have to explain what matters. Knowledge can also drift to the provider, so ask for documentation of the test suite and regular handovers. And a contract based on cheap hourly rates can reward the wrong thing, so check what the service levels actually reward.
Our AI managed testing services are built around this kind of arrangement. To read more, see how AI QA services compare with an internal QA approach, what to look for in a software testing company, and how to choose an AI software testing services company.
Modernize with confidence. Build for what's next.
Talk to our team about embedding AI into the core of your business.