Loading…
7
Our redundant fiber plan was a lie, one cable cut proved it
We signed a 5-year contract with a big carrier for dual diverse paths into our data center in Phoenix. I never bothered to trace the actual physical routes. Then a backhoe took out a main conduit last Tuesday, and both links went dark at the same time. Turns out they ran parallel in the same trench for the last 3 miles. My whole disaster recovery plan assumed zero downtime, but we were down for 6 hours. The carrier's SLA refunded us 4 grand, but our own clients saw the outage. What tipped me off was a field tech's photo of the cut, both cables side by side. Now I ask every ISP for a route map and a site survey before signing anything. Has anyone else found their 'redundant' setup was really just one point of failure?
1 comments
Log in to join the discussion
Log In1 Comment
barbaraschmidt2mo ago
Ugh, that's the classic bait and switch. And once you see that photo, you can't unsee it. But here's the next level of that trap: even if the routes are "diverse" on paper, they might still share a manhole, a pole, or a bridge somewhere. I've had a buddy who traced his two paths and found they crossed at a single ancient splice vault. One flooded vault took out both. So yeah, asking for the map is step one, but you also gotta actually walk it or have a tech verify the physical separation at every junction, not just the last mile. It's a pain, but it beats explaining to your boss why the "unbreakable" link broke.
3