Telephone and cable companies stonewalled California emergency officials during massive power outages

by Steve Blum • , , , ,

Cell site outages 28oct2019

Mobile carriers generally cooperated with California emergency officials during the week long siege of public safety power shutoffs in October, while cable and telephone companies hid behind confidentiality claims. Paul Troxel, who heads the 911 program at the California office of emergency services, testified at a California Public Utilities Commission hearing on Wednesday and told commissioners that neither the state’s emergency operations center or local officials knew where access to 911 service and disaster information, such as evacuation orders, were unavailable…

Outage data was not reported by all providers. Some providers were very responsive and provided outage data as requested by Cal OES, while others were slow to respond due to confidentiality concerns related to outage data. Frankly, information from the wireline and VoIP providers was not provided until the end of the event. Because of the lack of complete reporting, Cal OES had to work with the Federal Communications Commission to activate the disaster information reporting system.

The FCC’s data isn’t intended to support real time operations. It’s typically 12 to 24 hours old and only aggregated data is provided – at the county level for mobile carriers and the state level for telephone and cable companies. But it did provide a useful check on the accuracy of the data provided voluntarily by telecoms companies through their industry association, which is their standard method.

It didn’t work so well. According to Troxel, at one point the industry’s organisation, the California Utilities Emergency Association (CUEA), reported that 57,000 wireline customers were out of service, while the FCC’s disaster information reporting system said the figure was 224,000. CUEA’s mobile outage reports weren’t much better – in one county the FCC said 133 cell sites were down while CUEA claimed only nine were. The problems with the industry association’s data were severe enough that OES requested reports directly from telecoms companies, with varying degrees of success.

During the hearing, Verizon, T-Mobile and, perhaps, AT&T executives promised to provide timely, detailed outage information in future emergencies, and make it public. A senior AT&T operations executive said the company would do so, after a company lawyer deflected the question by extolling the wonderfulness of the current system – that’s an arm wrestling match that needs to be resolved. The other representatives, from Frontier Communications, Comcast, Charter Communications, Cox and Sprint, wouldn’t make any promises at all.

Meaningful answers and we’ll get back to you, as CPUC drills down on phone, broadband outages in emergencies

by Steve Blum • , , , ,

Cpuc phc telecoms outages 20nov2019

Telecoms company representatives – telco, cable and mobile – were grilled for three hours yesterday by CPUC commissioners about their ability to maintain communications capabilities during power outages and other emergencies. And their willingness to provide actionable, real time network status information to officials and the public.

The central issue is whether the California Public Utilities Commission should establish regulations for things like backup power, network resiliency and outage reporting, for voice, text and, perhaps, broadband service. Commissioner Cliff Rechtschaffen cut to heart of it, asking the eight representatives “would you support this as a regulatory requirement?”.

Three of the mobile companies – AT&T, Sprint and T-Mobile – were represented by senior operational and engineering executives. Although they didn’t express any great enthusiasm for new regulations, they engaged with questions posed by commissioners and generally gave knowledgable answers about their networks, back up capacity and emergency management procedures.

Verizon sent a lobbyist. He reiterated an early statement by Verizon that it would be happy to provide lots more information about future outages in something like real time, and make it public because our network is so damn good.

AT&T’s and T-Mobile’s execs signed onto that pledge. Mobile networks were a particular focus – 80% of 911 calls are made using mobile phones, according to a CPUC staffer. The objective is to identify and publicise communications gaps, where people can’t call 911 or access evacuation maps on the web. Sprint’s rep was more reticent, but it might not matter if T-Mobile is successful in acquiring the company.

It wasn’t clear whether AT&T’s wireline network was included in the promise. At one point, an AT&T lawyer jumped up and seemed to say no. Instead, he lauded AT&T platoons of lobbyists and public relations people, and their “longstanding” efforts to keep state and local officials informed.

Right.

Frontier Communications also sent a corporate lobbyist to the hearing. Not much came of it. She didn’t promise to share detailed or real time outage information, let alone make it public. She did say that only 85% of Frontier’s customers are served by central offices that have back up generators that can keep facilities powered for at least 72 hours. “Facilities further out” in “remote areas” rely on shorter-lived batteries and portable generators.

Translation: the urban systems we got from Verizon are okay, rural communities, not so much.

The three cable companies – Charter Communications, Comcast and Cox – sent regional managers, who typically have a lot of operational responsibility at the local level, but take their marching orders on corporate policy from headquarters. That seemed to be the case yesterday. All three were cordial and, within their field of expertise, knowledgeable enough, but not forthcoming when pressed for information sharing commitments. I’ll get back to you was a frequently heard response. Back up power on cable networks didn’t seem to be as robust as telcos. Comcast’s rep said that all their network devices in the field have back up power, but only 4 to 24 hours worth.

Telecoms companies to explain broadband, phone failures during California power cuts

by Steve Blum • , , , ,

Green acres utility pole

With another Pacific Gas and Electric company power shut off looming later today, the California Public Utilities Commission is calling in telecommunications companies and demanding that they be prepared to explain their “responsiveness during the latest wildfires and public safety power shut offs to keep telecommunications services on”.

A hearing is scheduled for this morning in San Francisco, with “top officials” from California’s major telecoms companies directed to “publicly appear and publicly address their response during the latest wildfire events [and] public safety power shutoffs”. The list includes California’s two big telephone companies – AT&T and Frontier Communications – as well as Comcast, Charter Communications and Cox on the cable side, and all four mobile carriers – AT&T, Sprint, T-Mobile and Verizon. Electric utility executives are also invited, albeit a bit more politely.

During the last wave of power cuts, intended to prevent wildfires, hundreds of thousands of Californians lost telephone and broadband service. Data collected by the Federal Communications Commission indicated that 455,000 telco and cable subscribers in 32 California counties lost connectivity, and 3.3% of cell sites were down. There might have been more – the FCC didn’t say how many telecoms companies were reporting outages – reporting was voluntary – or whether the total included all customers who couldn’t connect because they lacked back up power in their homes and businesses.

Who shows up will be as significant as what they say. Expect AT&T and Frontier to send people – they’re still regulated to an extent by the CPUC – but whether their representatives are actual decision makers or lobbyists with inflated titles remains to be seen.

Cable companies tend to resist any encroachment on their turf by the CPUC, so their response, if any, will be interesting to watch. Normally, mobile carriers don’t have much to say to state officials, but since AT&T has to be there anyway, and Sprint and T-Mobile are trying to make nice with the CPUC so they can gain approval for their merger, we might get some meaningful information from them.

California’s marquee industries are two halves of the same brain

by Steve Blum • , , ,

Egghead

Disney and Apple launched online video services this month, with both companies falling short of perfection. It’s interesting to compare the two platforms, dubbed Disney+ and Apple+. One is the brain child of an entertainment giant struggling with technology, the other was created by a tech giant struggling with content.

When Disney+ went live last week, demand outstripped capacity and users were locked out. Apple+, on the other hand, had no such problems. Its programming could be seen by anyone interested enough to log in. Unfortunately, the content offered has not excited anyone. It was reckoned workman-like, at a moment when Apple needed blockbuster pizzazz to break out of the over-the-top pack.

Disney’s server problem was solved in hours, if not minutes. By now, I doubt many people remember it. Fixing technical issues is a left brain, linear process. Apple, on the other hand, has to contend with a chaotic, right brain challenge. You don’t create world class content by assigning more engineers and spinning up more servers. So now there’s talk of former HBO chief Richard Plepler doing a deal with Apple – he has a proven track record. That’s no guarantee in the entertainment business, but it’s the way to bet.

Silicon Valley and Hollywood have a lot more in common than people realise. In both ends of California it’s about finding executives who can manage very talented, highly mobile people who can create marvels out of thin air. A track record of success, even if liberally sprinkled with failures, will attract investors in Los Angeles and San Francisco alike. Both cities are magnets for risk-tolerant capital, outrageous concepts and creative talent. The difference is that in Silicon Valley fortune seekers of modest gifts end up in cubicles making a hundred grand or two a year, while in Hollywood they’re waiting on tables.

For now, anyway.

Video entertainment “should not be considered essential” says AT&T. Amen say Comcast, Charter

by Steve Blum • , , , ,

Darth leia 625

For a company that paid $85 billion to become a video entertainment giant, AT&T has an odd idea of what’s essential and what’s not. In objections to a California Public Utilities Commission staff proposal, AT&T argued that “video entertainment” should play no role in determining what level of broadband service is “essential” and whether it’s affordable or not. It specifically targeted Netflix and ESPN+ as examples of non-essential services that are not “appropriate essential functions” and should not be included in calculations of what level of broadband speeds and data caps are necessary for Californians to conduct their every day lives.

In reply comments, the lobbying front organisation that Comcast, Charter Communications and other cable companies use to push their interests in Sacramento and at the CPUC endorsed AT&T’s position, paraphrasing it as “entertainment service such as Netflix is not essential”. It’s easy for the California Cable and Telecommunications Association to trash talk Netflix; the Walt Disney Company – ESPN’s majority owner – not so much.

AT&T and its amen corner got it wrong, for at least a couple of reasons. First, the CPUC staff white paper in question identified fixed broadband service at 20 Mbps download and 3 Mbps upload speeds as the minimum necessary for a Californian household to meet its “basic needs” such as education, telehealth and safety, and enjoy “full participation in society” by doing such things as “completing job applications and accessing government assistance programs”. Netflix and ESPN aren’t on the list. Neither is HBO Max, which AT&T is hoping will pull it out of the video subscriber death spiral it’s in, or Spectrum TV Essentials or Xfinity Instant TV. Calling out entertainment services is a red herring.

It’s also arrogant.

I’ve sat in many meetings in Sacramento and listened to telco and cable lobbyists speak with contempt about people who are misguided enough to think they ought to be able to watch video via the Internet, if they don’t provide sufficient profit to be worth it to those companies to deliver modern broadband service. Full participation in society requires more than just getting email or reading a web page. It includes access to the full range of online information and social and political interaction that’s available – and essential – to those of us who are fortunate enough to have it.

Collected documents from the CPUC’s investigation into essential service and affordability metrics for utilities are here.

Fitbit deal tests Google’s willingness, ability to follow California privacy law

by Steve Blum • , ,

Fitbit

Google’s $2.1 billion purchase of Fitbit will, if nothing else, be an excellent test case for California’s new consumer data privacy law, which takes effect in January. The California Consumer Privacy Act (CCPA) requires companies above a certain size let their customers know what kind of personal data is being collected and what it’s being used for, and gives individuals a level of control over the collection and use of their data.

The activity, location and health data collected by Fitbit devices is highly personal. It’s also highly valuable to Google’s business model. Which is about collecting, cross referencing and publishing data. Fitbit collects a flood of data from its users, and Google will be sorely tempted to mash it up with geo-referencing, email, search history and every other kind of data it has.

Most users probably won’t care, and will probably see a benefit from the kind of cross referencing Google might do – correlating heart rates to real time air quality data, for example.

But some users won’t like that at all. If Google is transparent about what it’s doing, and figures out a user-friendly process and interface to implement the procedures that CCPA mandates, users should have the knowledge and tools to control who else, if anyone, profits from their data.

That’s a big if, though. The functionality of fitness and activity trackers depends on the ability to transfer the data collected from the device to a platform that can store the data and perform value added analysis. If it’s done well, adding external data will increase the value of the analysis, but it also means commingling data sources, often in a complex way. Users have to understand that interplay in order to give (or withhold) informed consent. Figuring out how to do that with health and fitness data is about as hard as the problem gets.

“Framework” for telecoms competition in rural telco territories considered by CPUC

by Steve Blum • , , , ,

Tesoro viejo 2

A rousing and thoroughly disingenuous defence of telecommunications competition doesn’t appear to be enough for Comcast to get permission right now to cherry pick affluent households in Ponderosa Telephone Company’s territory. A pair of California Public Utilities Commission administrative law judges (ALJs) said in a ruling last Friday that even though allowing competitive telecoms companies into the protected service areas of California’s small, rural telcos should be considered on a case by case basis, those decisions should be made within a common framework.

The two ALJs – Mary McKenzie and Hazlyn Fortune – are managing what the CPUC calls a rulemaking proceeding that’s looking at the way California subsidises, and consequently protects, small telephone companies that serve remote and sparse rural communities that aren’t lucrative enough to attract big telecoms service providers. Or at least used to be. As California’s suburbs spread further out from cities, new developments are springing up on farm and ranch land that’s served by rural telcos.

Citing Comcast’s case as an example, they decided that the next step in that process is to establish a general set of rules that will guide future decisions about who should provide telephone service and, in some cases, broadband service in those new communities…

The Commission will first consider adopting general criteria in this Rulemaking as a framework for allowing competition, which will then be evaluated on a case-by-case basis considering local conditions for each individual small [rural telco] service territory where an application is filed by a potential competitive local exchange carrier (CLEC) seeking a certificate of public convenience and necessity (CPCN).

Comcast’s request to be allowed to provide telephone service in the upscale Tesoro Viejo development north of Fresno is being handled by another ALJ, Zhen Zhang, in a separate case. In theory, Zhang doesn’t have to wait for McKenzie and Fortune to finish their work, which could take months. In practice, since ALJ’s produce draft decisions for consideration by CPUC commissioners, it would probably be a waste of time to, as Ponderosa described it, put “the cart before the horse”.

Frontier digs a deeper digital divide in rural California with taxpayers’ shovel

by Steve Blum • , , , ,

Frontier verizon pole santa barbara county 10oct2015

A handful of rural communities in Lassen, Modoc and Kern counties will get their first taste of wireline broadband service from Frontier Communications if the California Public Utilities Commission approves infrastructure construction grants next month.

Unfortunately, it’s just a taste.

Frontier’s (and AT&T’s) strategy, as identified by a CPUC study earlier this year, of “disinvesting in infrastructure overall”, which is “most pronounced in the more rural and low-income service areas”, continues to be business as usual. Both of Frontier’s projects up for California Advanced Services Fund grants propose to deliver low speed service over ageing copper telephone lines. The $11 million would be spend on a desperately needed 137 mile fiber route and essential central office equipment upgrades, but Frontier’s interest in improving rural infrastructure, even when taxpayers are picking up the tab, ends there. As the CPUC’s draft resolution approving the Kern County grant describes the project, “Frontier will upgrade the existing communications facilities to increase broadband capacity but will not replace the copper cable infrastructure”. Likewise, the northeastern California project adds middle fiber and electronic equipment, but leaves “legacy copper infrastructure” in place.

It’s not an accident or anomaly. It’s deliberate.

Frontier continues to bleed customers and revenue, and selective fiber upgrades are the solution, according to CEO Dan McCarthy, who spoke about the company’s third quarter 2019 financial results

We achieved a sequential improvement in fiber net losses with only 1,000 in the third quarter. However, consumer copper losses of 52,000 were worse than the second quarter. In copper, although we experienced a sequential increase in gross additions, this was offset by a sequential increase in churn and we continue to manage this business for a decline. Fiber broadband gross additions increased sequentially in the third quarter and we also had a slight sequential improvement in fiber broadband churn. With the completion of the upgrades of the fiber network to be 10 gigabit capable, we have increased our emphasis on selling at higher speed tiers.

Frontier’s strategy is economically rational, and is probably its best shot at pulling shareholder value out of penny stock territory. What makes it rational, though, is the California legislature’s irrational (but well compensated) decision to subsidise 1990s era broadband service over 1890s era copper wires, and not hold incumbent telcos to the same standards in rural communities as they voluntarily and rationally adopt in densely populated, high income cities and suburbs.

CPUC commissioner asserts “a significant role” over broadband affordability and essential service

by Steve Blum • , , , ,

Rechtschaffen 2 20may2019

In a ruling issued on Friday, CPUC commissioner Clifford Rechtschaffen ended any doubt over whether an inquiry into the affordability of utility services includes the cost and quality of broadband access: it does. The decision puts wind in the sails of an analysis of broadband pricing and service speeds prepared by California Public Utilities Commission staff, and meets strident objections from AT&T, Comcast, Charter Communications and other monopoly model incumbents head on…

This amended scoping memo confirms that communications services, such as broadband internet access, are included within the scope of this proceeding. This amended scoping memo finds that [California Public Utilities] Code Sections 709, 280, 281, 275.6, and the Moore Act all demonstrate that the Legislature contemplated a significant role for the Commission in closing the digital divide in California and bringing advanced communications services, including broadband internet access, to all Californians. This proceeding may assist in that goal.

The California Cable and Telecommunications Association, which is a Sacramento lobbying front organisation for Comcast, Charter and other cable companies, argued that the CPUC shouldn’t look into the affordability of broadband service because it is “an interstate service governed by federal law, and defined as an “information service” and not a “telecommunications service”. In reply comments, AT&T agreed, saying “broadband is not a public utility service”. A joint filing by small rural telephone companies said much the same thing.

Unfortunately for them, the primary legal basis for their objections – the Federal Communications Commission’s blanket preemption of state broadband regulations was overturned by a federal appeals court. So long as the FCC says that broadband is an information service, it can’t wield its telecommunications authority as a magic weed whacker to chop down state regulations.

In his ruling, Rechtschaffen also set next June as the deadline for CPUC action on affordability and service standards for broadband and other utilities, including electricity, water, gas and voice services.

Collected documents from the CPUC’s investigation into essential service and affordability metrics for utilities are here.

T-Mobile gives CPUC some insight into post-Sprint merger plans for California, but won’t make it public

by Steve Blum • , , , ,

The hundred-plus pages of testimony submitted by three T-Mobile executives to the California Public Utilities Commission sheds a little more light on what the company intends to do in California when – if – it acquires Sprint and spins off customers, employees and assets to DISH. But most of the specific plans for California submitted to the CPUC last week were filed confidentially.

Chief operating officer Michael Sievert toned down the company’s weasel words about T-Mobile and Sprint workers in California, saying that the number of employees three years after the merger closes will be “equal to, or greater than” the current T-Mobile and Sprint total, even taking into account employees who might be transferred to DISH. However, he didn’t reconcile his pledge with another T-Mobile commitment to open a call center in the San Joaquin Valley and staff it up with 1,000 workers. That leaves open the possibility that hundreds of employees, from all over California, will be given a choice between unemployment or moving to Fresno County to begin a new career as a customer service rep.

Chief technical officer Neville Ray’s testimony boils down to we weren’t planning to use Sprint’s 800 MHz spectrum for 5G or much of anything else, so nothing changed. He also discusses leasing 600 MHz spectrum from DISH – T-Mobile has an option to do so but, according to Ray, hasn’t decided whether or to what extent to exercise it. If T-Mobile did lease that spectrum, though, it would mean cash flowing back to DISH and, presumably, less pressure to do something with it before the Federal Communications Commission takes it back.

In other words, T-Mobile’s option might result in a significant financial benefit to DISH, that could offset some of the financial penalties it might incur if it doesn’t meet its obligations to build out a competitive 5G network. Given DISH’s history of spectrum dealing, there’s reason to question the settlement’s fundamental premise that a new, facilities-based mobile broadband network will result. The lower the net cost of walking away, the less incentive DISH has to meet its nominal commitments.

Ray also submitted California-specific information about T-Mobile’s 5G deployment and service plans, mirroring the national level information given to the FCC. But unlike the national data, T-Mobile wants to keep its promises to California secret. Those could become public as the CPUC review moves ahead, but there’s no particular reason at this point to think they will.

The filing made by executive vice president Thomas Keys made minor updates to his previous testimony, but wasn’t particularly enlightening.

Opponents of the merger have two weeks to digest T-Mobile’s testimony and respond. The CPUC public advocates office’s quest for more information from DISH is running in parallel. Opponents could similarly ask T-Mobile to provide additional data and/or ask for more time to review it all, but at this point the case is still on a trajectory for a final decision in February 2020.

Links to the stack of arguments and exhibits everyone has filed are here.

My clients include California cities who do business with T-Mobile. I like to think that has no bearing on my commentary, but I like to think I’m good looking too. Take it for what it’s worth.