DevOps as a Service vs In House DevOps Teams: How SaaS Companies Should Decide
In January 2017, an engineer working to restore a database accidentally deleted data from GitLab.com’s main database server. The recovery team then discovered that the usual database backups were not available. GitLab later said nobody had been responsible for regularly testing that backup procedure. The incident cost the service many hours of downtime and roughly six hours of database changes that could not be recovered. Its published account explains what went wrong.
That’s the question I’d bring into any decision about DevOps as a service or an internal team. When production breaks, who can recover it? It doesn’t matter how reassuring the sales presentation is if nobody has tested the process.
Both an internal team and an outside provider can run reliable systems. Both can also miss important work when ownership isn’t clear. Judge the model by what people are responsible for and what they can prove they’ve tested.
Start With the Work Production Depends On
Someone must be responsible for getting changes into production safely. The team also needs to know whether services are working for customers and what to do if an update breaks them. Backups only become useful when somebody can restore the data.
Write down who is responsible for those jobs today. If it’s only the person who built the system, ask what happens when they’re away. That helps you compare an internal hire with managed support, whose contract should spell out which responsibilities it covers and during which hours.
You Have More Choices Than Hiring or Outsourcing Everything
Start with the work that needs covering. A one-off deployment problem may justify bringing in a specialist while the company’s developers keep production running. Frequent operational demands call for a different decision: a managed service with an explicit scope or an internal platform team that stays close to the product.
Many companies use a mix. A developer inside the business understands the application while an outside specialist helps with infrastructure changes. That can work, provided the two sides know who leads an incident and who is allowed to change production.
Never assume the other team is taking care of backups or overnight alerts. Make those responsibilities explicit before choosing the model.
One Excellent Engineer Still Needs Backup
Even an excellent infrastructure engineer can’t cover every night of the year and still have time to fix the issues causing the alerts. Smaller companies need a realistic rotation or outside support so reliability isn’t riding on one person’s availability.
Google’s guide to being on call explains why sustainable coverage and clear incident procedures matter. A company building its own team needs a plan for those responsibilities, especially when its best engineer is away.
Internal ownership is valuable when daily platform decisions depend on deep knowledge of the product. It still needs a plan for recovering from incidents without relying on one person’s memory.
What Should Stay Inside the Company?
Some production decisions depend heavily on the product. An external engineer may understand the infrastructure but not the business effect of disabling a particular feature. Keep someone inside the company who can make those decisions, even when an outside team handles the technical repair.
Managed DevOps support can fill gaps when a business needs occasional specialist coverage or help outside normal working hours. Frequent production work may justify an internal team that knows the product and can share operational responsibility. Compare who will actually respond to incidents and handle ongoing changes before deciding which model fits.
For a hybrid model, write down who leads the incident. Don’t leave both sides assuming the other one has permission to act.
Use Operational Pressure as a Sourcing Signal
If every production release depends on one engineer being available, the company has a problem regardless of who employs that person. The same is true when nobody can explain how to restore a database. Look at the work production depends on before deciding whether an outside team or new employees should take responsibility.
External support can make sense when the gap is urgent and the company lacks enough internal specialists to cover it. Internal hiring can make sense when the work is constant, product context is unusually deep, and leadership is prepared to fund a real team instead of one isolated “DevOps person.” If you still need to identify the technical pressure behind that decision, our guide to what breaks first as systems grow explores where to investigate.
Compare the Real Cost of Keeping Production Running
Compare the cost of actual incident coverage rather than lining up a salary with a retainer. Your best engineer may know the product deeply but can’t cover every holiday or overnight failure. Managed support can extend the response window, but the agreement should spell out who acknowledges an alert, who leads recovery and what requires internal approval.
For a fair comparison, price the work that falls outside a provider’s retainer, including emergency recovery. On the hiring side, count the hours engineers spend covering alerts and restoring services. Those hours come out of the time available for planned development.
I’d be cautious of any quote that makes reliability look cheap by ignoring the person who has to wake up when something goes wrong.
Who Can Act During a Real Incident?
Paying for overnight monitoring isn’t the same as giving someone authority to restore service. The person responding needs to know what can be changed safely. They also need a way to reach the person who understands the application’s effect on customers.
GitLab’s postmortem exposed a painful gap. Nobody had clearly taken responsibility for testing the backup recovery process, and the team discovered the problem when it needed those backups most. Before trusting a provider with production, I’d want to see a recent restore test and know who could step in if the usual engineer were unavailable.
Outside Engineers Need Access You Can Revoke
During a Kubernetes upgrade, outside engineers may need temporary administrator access. The delivery pipeline may also hold credentials that can deploy software to production, so those permissions need to match the work and end when it does. Individual accounts and a record of changes help the business see who did what.
NIST’s Zero Trust Architecture describes granting only the access needed for the task. The business should be able to revoke a provider’s access without losing its own infrastructure or the instructions needed to operate it.
Measure Whether Production Is Getting Better
A provider can generate a beautiful dashboard while the business still suffers the same outages. Ask whether releases are becoming safer and whether the team can recover more reliably after something fails.
The DORA delivery metrics look at how software changes move into production and how often they require recovery or extra work. They’re useful evidence when teams interpret them in context. For the service itself, measure something customers can feel. Does checkout work? Can the business restore data when needed?
Google’s error budget approach provides one way to agree on how much unreliability a product can tolerate before the team slows down releases to improve stability. A small business doesn’t need a complicated version. It needs targets people understand and can use to decide what work comes next.
What I’d Ask Before Hiring a DevOps Provider
Ten minutes on a support proposal sounds reassuring, but does that cover nights and weekends or only the time it takes to answer a message? Someone might acknowledge an alert and still lack permission to restore service. I’d want the provider to explain how the incident moves from that first response to recovery and which decisions need the business’s approval.
Ask about a past incident the provider can explain without revealing private details. You should be able to follow what went wrong, whether the team managed to restore the data and what changed afterward. That tells you far more than a polished promise about best practices.
Finally, ask what happens if the relationship ends. The business should keep control of its infrastructure and enough documentation for another qualified team to operate it.
What the First Engagement Should Prove
After an infrastructure and CI/CD audit, the team should know what runs in production, which access or recovery gaps need attention and what to fix first. The first engagement should also prove that a change can be deployed and rolled back safely.
I’d want a real backup restore test before a long presentation about future automation. If the team can’t restore a system, you have learned something urgent about the operating model. A sensible plan follows those findings rather than installing new tools just to fill a schedule.
Make the Decision on Paper Before Making It in Production
During an outage, nobody should have to hunt for the person authorized to approve a rollback. The response plan needs to say when the business joins in and how much downtime or lost data it can tolerate. That standard applies to employees and providers alike, and it should be revisited after the team has worked with the actual system.
Could You Change Providers Without Losing the System?
Owning the cloud account helps, but a handoff can still fail if every deployment depends on credentials held by the outgoing provider. Another qualified engineer should be able to access production, release a change and follow the current recovery instructions without relying on that provider.
When ownership changes, let the new team prove it can perform a recovery before removing the old team from the schedule. Then revoke permissions that are no longer needed. A good provider should make that handoff possible even if you never use it.
The Engineering Work Behind the Contract
CMX runs Kubernetes on physical servers, so replacing a failed machine takes more planning than starting another cloud instance. We need spare capacity for workloads to move onto while hardware is being replaced. Our article on bare metal Kubernetes with Cilium explains the network and recovery responsibilities behind that choice. A smaller company may be better served by a managed platform, depending on what its team can support.
If your team needs help deciding which operational work to bring in, our DevOps services explain what CMX can take on. The conversation starts with how production runs today and what the team can recover when it fails.
Ask Them to Restore Something
The GitLab outage is memorable because the usual fallback wasn’t available when the team needed it. That is the test I’d take into a DevOps hiring decision. Show me a recent recovery exercise. Explain what the team learned and what changed afterward.
Either staffing model can work. What matters is having somebody qualified to respond who has practiced the job before production breaks. A promise to be available is easier to make than proof that the system can recover.





