If the fight against the coronavirus is a world war, we humans against the bug, consider this blog post
Although I have been working in IT for more than half a century, I didn't run into my first case of a “community spread” computer virus until the late 1980's. Since then, detecting, removing, and repairing the damage done by computer viruses has been a constant battle that we have had to wage, and win, just to keep the Internet, and our IT infrastructure, afloat. Now I fear that the Internet may be brought down by a virus of the biological type, considered “novel” in this day and age, at a time when we will need it the most, and almost no one seems to be paying attention outside of a few tech circles, or sounding the alarm, or making plans to deal with what could quickly become an emergency situation that could cost the world many lives in the context of this growing coronavirus crisis.
Certainly not President Trump, or the federal coronavirus crisis response he is leading. When pumping up the stock market with his Friday White House Press Conference, he showed us just how clueless he is in this area, and the role he has for it in his show. He shook hands, and in one case, rubbed elbows, with corporate CEOs he pledged would help the nation weather the coronavirus storm. He said they were “the biggest business people, the greatest retailers anywhere in the world.” Not one of them was from the IT world. Certainly not Jeff Bezos, who is probably the single biggest business person, and greatest retailer anywhere in the world, not to mention the biggest name in cloud computing and Internet services with his AWS [Amazon Web Services]. AWS has 5 times more deployed cloud infrastructure than its next 14 competitors in aggregate. It's the open secret that allows Amazon to offer good prices, fast delivery, and pay a $15/hr. minimum wage.
While Google may not have been invited to this party, that didn't stop Trump from making false claims about what Google was doing to help us through this crisis. Trump claimed Google had 1700 engineers “helping to develop a website. It is going to be very quickly done, unlike websites of the past, to determine whether a test is warranted and to facilitate testing at a nearby convenient location." This was news to Google!
Trump thinks he can whistle up anything just by "bully pulpitting" it into existence. He lives in a fantasy world. Just as he refuses to understand that vaccines take a minimum of a year to 18 months to develop and produce, he doesn't understand that good, comprehensive websites take time to create, even for a powerhouse like Google. Such a website would need to be very accurate and well designed, and that is the easy part. It would need to be able to scale massively, from the very first day if rolled out nationally all at once like Trump said, and handle database records for tens of millions. This on top of all the traffic Google is regularly handling, especially now. I doubt if even Google has that much excess capacity, and they would be doing the world no favor if it crashed coming out of the gate, or brought down all their other services, or the Internet. This can't be just another one of your brochure websites, Donald. The smart thing is to roll it out in baby steps so there is time to scale it up without disruption, but nobody in Trumpland seems to understand that.
Mike Pence joined Trump in promoting this vaporware website as real, Dr. Deborah L. Birx even brought a chart to show how it would work. They really think that all it takes is good props.
Had he said truthfully that the website another member of the Google family owned by parent Alphabet, Verily, is putting together “is in the early stages of development, and planning to roll testing out in the Bay Area, with the hope of expanding more broadly over time,” he could have challenged Google to get involved, and put hundreds of their engineers to work to make it a nationwide website that could help people determine if a test was needed as quickly as possible.
Such a website could be extremely useful. For one thing, if only a small sampling of the people that filled out a form describing their COVAD-19 like symptoms, were tested, we could quickly extrapolate to get a good idea of what percentage of such people actually had COVAD-19, as compared to the percentage suffering the much less dangerous flu, or the common cold. This alone makes the effort worth it, so while we can't ask Google to rise to Trump's challenge, we can hope that they rise to Trump's lie, and overlook the insult.
But even if the website is everything any of us could want, it won't be very useful if people can't reach it, and that is a real danger in coming weeks.
A Storm may be brewing
A poster to slashdot.org [News for nerds] made this comment on Friday, based on data from Applopia:
Global downloads of business apps including Tencent Conference, WeChat Work, Zoom, Microsoft Teams and Slack have risen nearly five fold since the start of the year, data showed, as the coronavirus outbreak changes how corporations work.Another pointed to the vulnerability of the cloud computing that we have become so heavily dependent on in recent years:
All these cloud apps are suddenly seeing a surge, causing many of them to have issues - cloud is fine as long as everyone is sharing and surge capacity is limited to one or two major customers. Now suddenly every one of their customers needs max capacity which traditionally these customers would've had individually in a largely offline datacenter and simply spin up nodes for events like these.Devin Coldewey writes on Techcrunch:
Zoom's calling capacity during peak hours is severely limited, WebEx has similar problems, Teams is laggy and Office 365 janky as hell. Look at DownDetector for major services - ~10am-~4pm spikes a lot more reported issues over the last few days. We've just sent thousands of employees home and thousands of students need to join online classes starting next week and already having problems with small test sessions.
The cloud was built for temporary surge capacity for singular failures, instead business leaders decided they never needed to plan for emergencies again, just spin it up in the cloud. I'm going to predict that within the next 2-4 weeks we'll see massive failures and even data losses as Amazon and Microsoft infrastructure gets brought to its knees.
The coronavirus threat and official policies of “social distancing” are leading millions to stay home, doing meetings via video chat and probably watching Netflix and YouTube the rest of the time. That means a big uptick in bytes going through the tubes, both simultaneously and cumulatively.The public, and non-technical business community, has been taught to think of the Internet Cloud as a magical place where they can store endless amounts of data, create elaborate websites, and run businesses, with little concern for how the cloud does it. Those of us tasked with keeping cloud computing up and running, know that it runs on real hardware in datacenters with finite capacities. All those photos we upload to the cloud are stored on a hard disk somewhere in the world, and so for those that think we can instantly expand the cloud to meet whatever demand coronavirus throws at us, Coldewey says:
Nice of you to think Fiber, switches and routers grow on trees along with the labor to install it. What happens if everyone uses it simultaneously and then the Internet goes to a crawl?Already, Facebook, Netflix, and Twitter faced significant distress this Saturday:
Sunday Update: The number of outages Down Detector recorded from the websites it monitors was 48.
Outages overview - March 2020 (CT)
| Company | Date | Started |
|---|---|---|
| Bank of America |
03/15/2020
|
1:32 a.m.
|
| Hulu |
03/15/2020
|
12:45 a.m.
|
| Netflix |
03/15/2020
|
12:29 a.m.
|
| Chase |
03/15/2020
|
12:08 a.m.
|
| Disney+ |
03/14/2020
|
11:56 p.m.
|
| Suddenlink |
03/14/2020
|
11:30 p.m.
|
| Streamlabs |
03/14/2020
|
10:58 p.m.
|
| Roblox |
03/14/2020
|
10:54 p.m.
|
| Twitch |
03/14/2020
|
10:42 p.m.
|
| Call of Duty |
03/14/2020
|
10:20 p.m.
|
| Dazn |
03/14/2020
|
8:44 p.m.
|
| Walmart.com |
03/14/2020
|
7:35 p.m.
|
| Black Desert Online |
03/14/2020
|
6:44 p.m.
|
| 2k |
03/14/2020
|
1:48 p.m.
|
03/14/2020
|
1:44 p.m.
| |
| Disney+ |
03/14/2020
|
1:24 p.m.
|
| Roblox |
03/14/2020
|
12:50 p.m.
|
| Call of Duty |
03/14/2020
|
11:44 a.m.
|
| Netflix |
03/14/2020
|
11:41 a.m.
|
| Evernote |
03/14/2020
|
11:04 a.m.
|
| Destiny |
03/14/2020
|
10:52 a.m.
|
| Rainbow Six |
03/14/2020
|
9:42 a.m.
|
03/14/2020
|
8:18 a.m.
| |
| MLB The Show |
03/14/2020
|
6:09 a.m.
|
| Midcontinent Media |
03/14/2020
|
4:33 a.m.
|
| Tinder |
03/14/2020
|
3:38 a.m.
|
| Southern California Edison |
03/14/2020
|
2:02 a.m.
|
| Pokémon Go |
03/14/2020
|
1:26 a.m.
|
| Call of Duty |
03/14/2020
|
12:52 a.m.
|
| Escape from Tarkov |
03/14/2020
|
12:28 a.m.
|
| Netflix |
03/13/2020
|
11:45 p.m.
|
| TD Ameritrade |
03/13/2020
|
9:46 p.m.
|
| Madden |
03/13/2020
|
9:37 p.m.
|
| Secom |
03/13/2020
|
8:46 p.m.
|
| Armstrong |
03/13/2020
|
8:32 p.m.
|
| Overwatch |
03/13/2020
|
8:30 p.m.
|
| Playstation Network |
03/13/2020
|
8:26 p.m.
|
| Smite |
03/13/2020
|
8:06 p.m.
|
| PG&E |
03/13/2020
|
7:06 p.m.
|
| United Airlines |
03/13/2020
|
6:46 p.m.
|
| Blizzard Battle.net |
03/13/2020
|
6:08 p.m.
|
| Path of Exile |
03/13/2020
|
5:14 p.m.
|
| Roblox |
03/13/2020
|
4:58 p.m.
|
| Dota 2 |
03/13/2020
|
4:56 p.m.
|
| TDS Telecom |
03/13/2020
|
4:18 p.m.
|
| Quora |
03/13/2020
|
3:06 p.m.
|
| Steam |
03/13/2020
|
2:02 p.m.
|
| Xbox Live |
03/13/2020
|
1:59 p.m.
|
| Expedia |
03/13/2020
|
1:36 p.m.
|
| NBA 2k |
03/13/2020
|
1:09 p.m.
|
Two years ago the Oracle Internet Intelligence Group created a free map of global Internet Health. As of this writing, the only area of the United States to suffering significant Internet disruption is Puerto Rico, which is rated the fifth most disrupted place on Earth in the past 48hrs, just behind Somalia, Egypt, Burundi, and Djibouti. This speaks volumes about the third world status the United States gives to its last colony. This map will be important to monitor going forward because in the coming weeks of “social distancing,” as we rely on the Internet to work from home, entertain ourselves, maintain social contacts, and communicate critical health information, US Internet outages might not be just for Puerto Rico anymore. In this coming crisis, the health of the Internet will seriously affect the health of both the people and the financial markets.
There may be a time in the near future when we may have to triage bandwidth, which will be considerably less painful than triaging medical ventilators. If we get to that point, I vote that the Internet traffic nobody wants to receive, SPAM, be the first to go.
Clay Claiborne
Linux Systems Administrator
