Managed High Availability
From 4 Hours to 10 Seconds
Four architectures, four kinds of failure absorbed, four budgets. Here is what we deploy and, more importantly, what we operate for our clients — with real failover times, not brochure ones.
High availability is not a product, it is an assembly
You do not buy "high availability" the way you buy a server. The building blocks — keepalived, Pacemaker, a hypervisor's HA manager, a routing daemon — are all open source and free of charge. What you pay for is choosing the right assembly, fitting it together without creating new points of failure, and keeping it honest over time. Installing keepalived takes twenty minutes; that is not what you are paying for.
What it absorbs, and what it does not
High availability answers one question: how do we stay up when a component dies? It does not answer how do we come back after a disaster? — that is recovery, and it rests on backup. The two are paid for separately because they cover opposite risks: a hardware failure on one side, human error or ransomware on the other. Against the second, high availability actively works against you, faithfully replicating the corrupted data.
Recovery and backup are covered in depth on NimbusBackup and in our article on disaster recovery with Proxmox. This page is about continuity only.
How it differs from a resolution time SLA
A resolution time SLA commits to a deadline for human intervention: someone is paged, connects, diagnoses, repairs. High availability removes the need for that intervention for a whole class of failures — the service fails over on its own, and the team then works calmly to rebuild redundancy. The two are complementary: you keep the SLA for everything automation cannot handle, which is most real incidents.
The question that comes first: what does an hour of downtime cost you?
It is the only question that allows an honest trade-off. An architecture that divides failover time by fifty costs more than one that divides it by five — and for many companies, going from four hours to five minutes is plenty, while going down to ten seconds cannot be justified. Until the cost of an hour of downtime is quantified, you are comparing architectures in a vacuum.
The calculation is quick and it often changes the conversation: our downtime cost calculator gives you an order of magnitude in two minutes, from your revenue and headcount.
The 4 architectures we deploy
They are not sorted by technology but by the failure they absorb — and above all by what detects it. That distinction drives everything else.
| Level | What it absorbs | What detects it | Failover |
|---|---|---|---|
| 1 · VM restart | Loss of a hypervisor | The cluster: a node goes silent | 4 h → 5 min |
| 2 · Service failover | Loss of an instance — hardware, OS or service | The peer: the service stops answering | 5 min → 10 s |
| 3 · Two layers | Both, and each other's blind spots | Both layers, at their own granularity | A few seconds |
| 4 · Internal anycast | Loss of a site or a VLAN | Routing: the route disappears | Routing convergence |
1. Restart the VM elsewhere
A hypervisor dies and its virtual machines restart on their own on the surviving nodes. We install the three prerequisites that make it possible: at least three nodes so the cluster can decide without ambiguity, shared or replicated storage so the VM can boot elsewhere, and a fencing mechanism that guarantees a doubtful node will not restart on its own side.
Without that setup you have to restore: expect one to four hours, a good part of which is spent waiting for someone to notice. With it, expect two to five minutes and nobody needs to notice. Mind what it does not do: the VM restarts, it does not fail over — in-flight sessions are lost, and a dead application inside a running VM will never be detected at this level.
2. Fail the service over
Two nodes watch each other and share a floating IP address. Whichever one is answering holds it; if it stops answering, the other takes it over in one to three seconds. The client only ever sees one address, and never knows there are two machines behind it.
Contrary to a common belief, this level also absorbs hardware failure: two physical servers are enough, with no hypervisor at all — it is in fact the historical setup. What sets it apart from level 1 is not the kind of failure but the granularity: here it is the service that is watched. Dead disk, frozen kernel, crashed process: the reaction is the same.
3. Stack the two
A service cluster installed inside virtual machines, themselves running on hypervisors in high availability. Each layer covers the other's blind spot: the service fails over within seconds, and then redundancy rebuilds itself automatically as the lost machine restarts elsewhere.
It is our reference architecture when the outage itself is unacceptable — and also the easiest to get wrong. The two layers do not talk to each other: you have to declare a rule forbidding both VMs from landing on the same hypervisor, align the detection timeouts so they do not contradict each other, and decide which of the two has the final say when both want to fence a node at once.
4. Leave the site
An infrastructure service — internal DNS resolver, log collector, RADIUS server, outbound proxy — present in every site and every VLAN under a single address. Each instance announces that address to the network; the one that dies simply stops announcing, and routing sends traffic elsewhere.
No internet, no transit and no AS number required: interior routing protocols are enough. The trap is elsewhere, and it is a quiet one: failover is automatic, capacity is not. If each site is sized for its own load, the survivor absorbs that load on top of its own and collapses without a sound.
Failover time and reach, mechanism by mechanism
You do not really choose a mechanism: you choose a reach, and the reach dictates the mechanism, and therefore the failover time. That is what explains the order of the four levels above.
| Mechanism | Failover time | Reach |
|---|---|---|
| keepalived (VRRP) | From a few hundred milliseconds to a few seconds, depending on the configured timers | Same broadcast domain: within a datacenter, or layer 2 stretched over VXLAN |
| Heartbeat, Corosync, CARP | The same — it is the same family of mechanism, a heartbeat and a silence threshold | |
| Hypervisor HA | Same detection (same layer underneath), plus fencing confirmation and booting the virtual machine → 2 to 5 min end to end — provided the VM boots cleanly. A filesystem check on a large volume, or an encrypted disk waiting for a passphrase, blows that estimate away | |
| IGP (OSPF, IS-IS) | Immediate when a dedicated port goes down — the physical link is the signal. A few seconds otherwise, the time the timers take | Your own routed network: from a few kilometres to national scale, depending on the links you have — dark fibre, MPLS, leased circuits. The protocol is not the limit, latency is |
| BGP | 30 s with aggressive tuning, 180 s with default timers | Worldwide |
The constraint that actually decides: latency, and it is paid on every write
Widening the reach does not only cost failover time. With synchronous replication between two sites — a block-level mirror such as DRBD, for instance — a write is only acknowledged to the application once the other site has confirmed it. 15 ms of latency between the two sites means 15 ms added to every single write. Not on average: to each one.
The arithmetic is brutal and it does not negotiate. A transaction chaining ten sequential writes pays 150 ms of pure network before the disk has done any work at all. A job doing thousands goes from a handful of seconds to several minutes. This is not gradual degradation you absorb by adding hardware: it is a floor imposed by the speed of light and the path of the fibre.
But not every synchronous replication pays at the same rate, and that is what decides the architecture. A block-level mirror pays on every write, because it knows nothing about what happens above it. Application-level synchronous replication such as a Galera cluster understands transactions: it groups the changes and only synchronises at commit time. That same ten-write transaction costs one round trip there, not ten. At identical latency, one becomes unusable where the other stays comfortable — which is why replication moves up to the engine level as soon as sites get further apart.
So it is latency, not budget, that sets the limit of what is achievable. It is measured before designing, not after — and when it rules synchronous replication out, the answer is not to push harder but to move to asynchronous replication and accept the data loss that comes with it.
The tools we use
All open source, all battle-tested, none under licence. We deploy them, we operate them, and for some of them we contribute to their code.
| Tool | What it does for our clients | Level |
|---|---|---|
| Proxmox VE HA[1] | Restarts a virtual machine on another node. Covered in depth on our Proxmox site. | 1 |
| Proxmox Backup Server[2] | Makes the VM restorable elsewhere. Covered on NimbusBackup. | 1 / recovery |
| keepalived[3] | Moves an IP address between two nodes, following the VRRP[4] standard. The same daemon can also do layer-4 load balancing, through IPVS/LVS[5]. | 2 |
| Pacemaker & Corosync[6] | Orders resources, arbitrates quorum and fences failing nodes, when moving an IP is no longer enough. | 2 |
| Galera[7] & replication-manager[8] | Synchronous replication and orchestrated failover for MariaDB. Covered on our MariaDB site. | 2 (data) |
| FRRouting[9] / BIRD[10] | Announces a service from several sites of the internal network. We also run it on our own network, and we contribute to its code. | 4 |
| Monitoring & on-call | Detects what automation does not — and checks that redundancy is still real. | All |
How we choose
Single-site SMB
A few minutes of interruption is tolerable, half a day is not. Level 1 is almost always enough, and it has a decisive advantage: the setup that lets a VM restart elsewhere is exactly the one that constitutes your recovery plan. One investment, two benefits.
E-commerce, SaaS
Every minute of downtime is counted in lost orders, and a failover that drops in-flight sessions is expensive in abandoned carts. Level 3 earns its keep: failover within seconds, sessions preserved where the application allows it, and redundancy rebuilt automatically.
Multi-site, regulated
Several sites, several VLANs, and infrastructure services everything else depends on. Level 4 becomes relevant: a resolver or a collector that follows the network rather than the machine, and survives the loss of a site without reconfiguring anything on the client side.
And when it is not failure, but load?
Everything above answers a single question: what happens when a component fails? That is high availability. There is a second question, answered with the same tools but for the opposite reason: what happens when everything works, but there is too much of it?
The two overlap heavily. A shared address that fails over is one step away from being a shared address that distributes; keepalived, which carries the VIP, also drives the Linux kernel's own load balancer. It is in fact the trap named above at level 4: failover is automatic, capacity is not. A survivor inheriting everyone else's load falls in turn, and no high-availability mechanism prevents it.
We treat that second axis separately, under the same rule: start as low as the need allows, and only move up a level for what that level uniquely provides — from DNS with several addresses to global BGP announcement, by way of layer 4 and layer 7.
The 5 levels of load balancing →What we operate afterwards — the part nobody sells
A cluster is not something you install and forget. It has to be maintained, tested, and re-checked after every maintenance window. The most common situation we find at new clients is not a badly designed cluster: it is a correctly designed cluster whose redundancy quietly disappeared months earlier.
A migration moved both virtual machines onto the same hypervisor. An update disabled fencing. A certificate expired on the replication link. None of that raises an alert if nobody is watching redundancy itself — as opposed to the service, which keeps answering perfectly right up to the day it stops.
That is why we charge for operation and not only for deployment: regular failover tests, verification that the anti-affinity rule is still applied, quorum checks, and 24/7 on-call for everything automation cannot handle.
What is included, what is extra
Design and deployment
Charged on quote, based on €150 excl. VAT per hour. The scope depends entirely on what you already have: how many nodes are available, which provider you use and what it allows in terms of private networking and movable addresses, and which application needs to stay available. We audit before we quote.
Operation
From €150 excl. VAT per month per server (Pro plan, 24/7, 4-hour response time SLA), with a reduced rate from the second node of the same cluster. Monitoring, updates, on-call and failover tests included. See the managed server plans.
The hardware is not ours
You buy your servers directly from the provider of your choice — OVHcloud, Hetzner, Scaleway or any other — and you remain their owner. We charge for design, deployment and operation, never an all-inclusive package that would hide the real cost of your infrastructure. Sizing depends on your load, and we help you define it before you buy.
Frequently asked questions about high availability
Does high availability replace backups?
No, and the two solve opposite problems. High availability protects you from a failure: one component dies, another takes over. Backup protects you from a mistake: an accidental deletion, ransomware, silent corruption. In that second case high availability actively works against you, faithfully replicating the corrupted data to every node. A serious infrastructure has both.
What is the minimum number of servers for high availability?
It depends on what the cluster carries. For a stateless service — proxy, DNS resolver, NTP, relay — two nodes are perfectly fine: the worst case of a network cut is an address held twice for a few seconds, annoying but not destructive. As soon as data is involved you need a third point of view, to stop two instances writing in parallel and diverging silently — a lightweight witness, a fencing mechanism, or a full third node. On the hypervisor side, three nodes remain the rule.
Does high availability protect against human error or ransomware?
No. A deletion command, a purge script or ransomware encryption are all valid operations from the infrastructure's point of view: they are replicated, propagated and made highly available just as efficiently as anything else. The only answer to those scenarios is backup, with suitable retention and isolation — which is NimbusBackup's job. High availability covers failure, not error.
Can you build high availability with a single dedicated server?
No. A single server remains a single point of failure, whatever its internal redundancy: dual power supplies, RAID and ECC memory reduce the probability of a failure, they do not remove the scenario where the machine stops. What you can do with a single server is shorten recovery time by keeping a backup that is restorable elsewhere. That is useful, and often the right first step — but it is not high availability.
Do we need high availability if we already have a 4-hour resolution time SLA?
The two are complementary and do not cover the same thing. A resolution time SLA commits to a deadline for human intervention: someone is paged, connects, diagnoses and repairs. High availability removes the need for that intervention for a whole class of failures: the service fails over on its own, within seconds, and the team then works calmly to rebuild redundancy. You keep the SLA for everything automation cannot handle — which is most real incidents.
Can you deploy high availability on infrastructure we already own?
Yes, and it is the most common case. We start from what exists: how many servers are available, which hosting provider you use and what it allows in terms of private networking and movable IP addresses, and which application needs to stay available. Those three factors determine the achievable architecture, and sometimes we conclude that a node must be added or an option enabled at the provider first. You buy your infrastructure directly; we charge for design, deployment and operation.