As a wild fire burned in the Santa Cruz mountains, a key AT&T fiber line was cut nearby, reportedly by a road maintenance crew doing previously scheduled work just before 5:00 a.m. on Tuesday of last week.
In 2009, a break in a different AT&T cable effectively knocked Santa Cruz, Watsonville and most of the rest of the county off of the Internet for most of a day. Since then, AT&T, Comcast and independent broadband companies have upgraded and diversified cable routes running north and south. A few thousand customers were affected by last week’s break, but it went unnoticed by most people in Santa Cruz County.
But not all. The County of Santa Cruz’s IT infrastructure was connected directly to the severed AT&T cable, and there was no failover capacity in place. So the county’s website went down, just as residents in and near the evacuation area would have been waking up and going on line, looking for information about the Bear Fire.
Fortunately, Cruzio, a local Internet service provider of 30 years standing, had a solution ready to go. As a result of an earlier swap with the county, Cruzio installed a 100 Mbps auxiliary circuit in the county’s main building on Ocean Street in Santa Cruz. It was connected to Cruzio’s high capacity links that rely on newly installed, redundant fiber routes, one going north to Sunnyvale and the other, subsidised by the California Advanced Services Fund, heading south through Watsonville to the Internet backbone along the U.S. 101 corridor.
With Cruzio’s assistance, county staff routed their traffic through this connection, and got back on line “a little after noon”, according to a county spokesman.
But it wasn’t enough. The combination of external web traffic and internal county business quickly overloaded the connection – the AT&T service it replaced was specced at 250 Mbps – and county staff asked that the connection be bumped higher. Cruzio replied by opening up a gigabit port. “No charge for any of this of course” said James Hackett, director of business operations and development at Cruzio.
Had there been a repeat of the outage caused by the 2009 AT&T fiber cut, on top of a growing fire threatening lives and homes in the Santa Cruz Mountains, the result could have been a major, and dangerous, disruption. Instead, the work that’s been done over the past eight years to build independently-owned fiber optic lines in the region, led primarily by U.C. Santa Cruz, kept the focus where it needed to be: on fighting the fire.
This post is taken from an article I wrote last week for Santa Cruz TechBeat, and has been updated with information provided by the County of Santa Cruz..