Featured Post

The white-Left Part 1: The two meanings of white

Saturday, March 14, 2020

What will we do if coronavirus takes down the Internet?

Last week, in response to the coronavirus, hundreds of corporations joined Google, Microsoft, Amazon, Intel, Twitter, HP and Apple in rolling out comprehensive work-from-home programs for their employees. Most are scheduled to start next week. Classrooms at every level are being shut down, with online learning expected to fill the breach.  Even churches are turning to Internet services. With sports games, concerts, theaters, and festivals all shut down, millions of us are staying home more, and relying on the Internet for both information and entertainment, billions of us, worldwide. The Internet will be stressed as it never has been before, and there isn't as much “excess” capacity as many may think, or we may need, in the coming months, as we use it in a multitude of ways to fight the coronavirus. What happens if the Internet comes to a screeching halt under this load? This is an international dilemma. Is anybody planning for this? I've seen nothing from governments or the industry. How many are aware of the danger? I've seen no talk of this in the mainstream media, and I am one to stream MSNBC, CNN, and Fox News all at the same time through YouTubeTV. Even before this, YouTube and Netflix accounted for half of all Internet traffic.

If the fight against the coronavirus is a world war, we humans against the bug, consider this blog post




Although I have been working in IT for more than half a century, I didn't run into my first case of a “community spread” computer virus until the late 1980's. Since then, detecting, removing, and repairing the damage done by computer viruses has been a constant battle that we have had to wage, and win, just to keep the Internet, and our IT infrastructure, afloat. Now I fear that the Internet may be brought down by a virus of the biological type, considered “novel” in this day and age, at a time when we will need it the most, and almost no one seems to be paying attention outside of a few tech circles, or sounding the alarm, or making plans to deal with what could quickly become an emergency situation that could cost the world many lives in the context of this growing coronavirus crisis.  

Certainly not President Trump, or the federal coronavirus crisis response he is leading. When pumping up the stock market with his Friday White House Press Conference, he showed us just how clueless he is in this area, and the role he has for it in his show. He shook hands, and in one case, rubbed elbows, with corporate CEOs he pledged would help the nation weather the coronavirus storm. He said they were “the biggest business people, the greatest retailers anywhere in the world.” Not one of them was from the IT world. Certainly not Jeff Bezos, who is probably the single biggest business person, and greatest retailer anywhere in the world, not to mention the biggest name in cloud computing and Internet services with his AWS [Amazon Web Services]. AWS has 5 times more deployed cloud infrastructure than its next 14 competitors in aggregate. It's the open secret that allows Amazon to offer good prices, fast delivery, and pay a $15/hr. minimum wage.

While Google may not have been invited to this party, that didn't stop Trump from making false claims about what Google was doing to help us through this crisis. Trump claimed Google had 1700 engineers “helping to develop a website. It is going to be very quickly done, unlike websites of the past, to determine whether a test is warranted and to facilitate testing at a nearby convenient location." This was news to Google!

Trump thinks he can whistle up anything just by "bully pulpitting" it into existence. He lives in a fantasy world. Just as he refuses to understand that vaccines take a minimum of a year to 18 months to develop and produce, he doesn't understand that good, comprehensive websites take time to create, even for a powerhouse like Google. Such a website would need to be very accurate and well designed, and that is the easy part. It would need to be able to scale massively, from the very first day if rolled out nationally all at once like Trump said, and handle database records for tens of millions. This on top of all the traffic Google is regularly handling, especially now. I doubt if even Google has that much excess capacity, and they would be doing the world no favor if it crashed coming out of the gate, or brought down all their other services, or the Internet. This can't be just another one of your brochure websites, Donald. The smart thing is to roll it out in baby steps so there is time to scale it up without disruption, but nobody in Trumpland seems to understand that.

Mike Pence joined Trump in promoting this vaporware website as real, Dr. Deborah L. Birx even brought a chart to show how it would work. They really think that all it takes is good props.


Had he said truthfully that the website another member of the Google family owned by parent Alphabet, Verily, is putting together “is in the early stages of development, and planning to roll testing out in the Bay Area, with the hope of expanding more broadly over time,” he could have challenged Google to get involved, and put hundreds of their engineers to work to make it a nationwide website that could help people determine if a test was needed as quickly as possible.

Such a website could be extremely useful. For one thing, if only a small sampling of the people that filled out a form describing their COVAD-19 like symptoms, were tested, we could quickly extrapolate to get a good idea of what percentage of such people actually had COVAD-19, as compared to the percentage suffering the much less dangerous flu, or the common cold. This alone makes the effort worth it, so while we can't ask Google to rise to Trump's challenge, we can hope that they rise to Trump's lie, and overlook the insult.

But even if the website is everything any of us could want, it won't be very useful if people can't reach it, and that is a real danger in coming weeks.

A Storm may be brewing

A poster to slashdot.org [News for nerds] made this comment on Friday, based on data from Applopia:
Global downloads of business apps including Tencent Conference, WeChat Work, Zoom, Microsoft Teams and Slack have risen nearly five fold since the start of the year, data showed, as the coronavirus outbreak changes how corporations work.
Another pointed to the vulnerability of the cloud computing that we have become so heavily dependent on in recent years:
All these cloud apps are suddenly seeing a surge, causing many of them to have issues - cloud is fine as long as everyone is sharing and surge capacity is limited to one or two major customers. Now suddenly every one of their customers needs max capacity which traditionally these customers would've had individually in a largely offline datacenter and simply spin up nodes for events like these.

Zoom's calling capacity during peak hours is severely limited, WebEx has similar problems, Teams is laggy and Office 365 janky as hell. Look at DownDetector for major services - ~10am-~4pm spikes a lot more reported issues over the last few days. We've just sent thousands of employees home and thousands of students need to join online classes starting next week and already having problems with small test sessions.

The cloud was built for temporary surge capacity for singular failures, instead business leaders decided they never needed to plan for emergencies again, just spin it up in the cloud. I'm going to predict that within the next 2-4 weeks we'll see massive failures and even data losses as Amazon and Microsoft infrastructure gets brought to its knees.
Devin Coldewey writes on Techcrunch:
The coronavirus threat and official policies of “social distancing” are leading millions to stay home, doing meetings via video chat and probably watching Netflix and YouTube the rest of the time. That means a big uptick in bytes going through the tubes, both simultaneously and cumulatively.
The public, and non-technical business community, has been taught to think of the Internet Cloud as a magical place where they can store endless amounts of data, create elaborate websites, and run businesses, with little concern for how the cloud does it. Those of us tasked with keeping cloud computing up and running, know that it runs on real hardware in datacenters with finite capacities. All those photos we upload to the cloud are stored on a hard disk somewhere in the world, and so for those that think we can instantly expand the cloud to meet whatever demand coronavirus throws at us, Coldewey says:
Nice of you to think Fiber, switches and routers grow on trees along with the labor to install it. What happens if everyone uses it simultaneously and then the Internet goes to a crawl?
Already, Facebook, Netflix, and Twitter faced significant distress this Saturday:



They were just 3 of 26 big websites outages on Saturday. On Friday, the number was 23 according to Down Detector:
Sunday Update: The number of outages Down Detector recorded from the websites it monitors was 48.

Outages overview - March 2020 (CT)
CompanyDateStarted
Bank of America
03/15/2020
1:32 a.m.
Hulu
03/15/2020
12:45 a.m.
Netflix
03/15/2020
12:29 a.m.
Chase
03/15/2020
12:08 a.m.
Disney+
03/14/2020
11:56 p.m.
Suddenlink
03/14/2020
11:30 p.m.
Streamlabs
03/14/2020
10:58 p.m.
Roblox
03/14/2020
10:54 p.m.
Twitch
03/14/2020
10:42 p.m.
Call of Duty
03/14/2020
10:20 p.m.
Dazn
03/14/2020
8:44 p.m.
Walmart.com
03/14/2020
7:35 p.m.
Black Desert Online
03/14/2020
6:44 p.m.
2k
03/14/2020
1:48 p.m.
Facebook
03/14/2020
1:44 p.m.
Disney+
03/14/2020
1:24 p.m.
Roblox
03/14/2020
12:50 p.m.
Call of Duty
03/14/2020
11:44 a.m.
Netflix
03/14/2020
11:41 a.m.
Evernote
03/14/2020
11:04 a.m.
Destiny
03/14/2020
10:52 a.m.
Rainbow Six
03/14/2020
9:42 a.m.
Twitter
03/14/2020
8:18 a.m.
MLB The Show
03/14/2020
6:09 a.m.
Midcontinent Media
03/14/2020
4:33 a.m.
Tinder
03/14/2020
3:38 a.m.
Southern California Edison
03/14/2020
2:02 a.m.
Pokémon Go
03/14/2020
1:26 a.m.
Call of Duty
03/14/2020
12:52 a.m.
Escape from Tarkov
03/14/2020
12:28 a.m.
Netflix
03/13/2020
11:45 p.m.
TD Ameritrade
03/13/2020
9:46 p.m.
Madden
03/13/2020
9:37 p.m.
Secom
03/13/2020
8:46 p.m.
Armstrong
03/13/2020
8:32 p.m.
Overwatch
03/13/2020
8:30 p.m.
Playstation Network
03/13/2020
8:26 p.m.
Smite
03/13/2020
8:06 p.m.
PG&E
03/13/2020
7:06 p.m.
United Airlines
03/13/2020
6:46 p.m.
Blizzard Battle.net
03/13/2020
6:08 p.m.
Path of Exile
03/13/2020
5:14 p.m.
Roblox
03/13/2020
4:58 p.m.
Dota 2
03/13/2020
4:56 p.m.
TDS Telecom
03/13/2020
4:18 p.m.
Quora
03/13/2020
3:06 p.m.
Steam
03/13/2020
2:02 p.m.
Xbox Live
03/13/2020
1:59 p.m.
Expedia
03/13/2020
1:36 p.m.
NBA 2k
03/13/2020
1:09 p.m.
Monday is the first day millions, including almost 100,000 from Google alone, will be working from home. We will see how things fare then. These numbers bear watching more than the Dow Jones Industrial Average, because while falling stock prices should have limited short-term impact on our ability to weather this storm, and if it is a short one, the markets should quickly recover. A failed Internet, on the other hand, will take down many lives, as well as the DOW.

Two years ago the Oracle Internet Intelligence Group created a free map of global Internet Health. As of this writing, the only area of the United States to suffering significant Internet disruption is Puerto Rico, which is rated the fifth most disrupted place on Earth in the past 48hrs, just behind Somalia, Egypt, Burundi, and Djibouti. This speaks volumes about the third world status the United States gives to its last colony. This map will be important to monitor going forward because in the coming weeks of “social distancing,” as we rely on the Internet to work from home, entertain ourselves, maintain social contacts, and communicate critical health information, US Internet outages might not be just for Puerto Rico anymore. In this coming crisis, the health of the Internet will seriously affect the health of both the people and the financial markets.


There may be a time in the near future when we may have to triage bandwidth, which will be considerably less painful than triaging medical ventilators. If we get to that point, I vote that the Internet traffic nobody wants to receive, SPAM, be the first to go.

Clay Claiborne
Linux Systems Administrator

Wednesday, March 11, 2020

Has President Trump come down with something?

How sick is President Trump now?

We know from news reports that Trump has come into contact with people who have been in contact with people who have been effected by the coronavirus. We know that he claims not to have been tested. Now he has cancelled two of his most beloved political rallies. Has he done that out of a new found concern for the health of his supporters, or is something else going on?

Watch and listen carefully to these two videos of Donald Trump speaking in the Oval Office, and make up your own mind. The first is tonight's Oval Office address in which he announced the ban on travel from Europe, but not the UK!!! The second video is Trump's Oval Office defense of Roger Stone less than a month ago.



Ask yourself "Is this the same man?" or is this the same man trying to hide a serious illness? Notice especially the squinting eyes and labored breathing at the end of sentences, compare that to his demeanor on 12 February, 2020. Is he medicated, or sick, or both?


Here he thinks drug addicts, the victims of drug trafficking, should get like nine years in prison for having an addiction. That is also sick, but normal for Trump.

Then, there is also this:


UPDATE #1: CNN is reporting that Trump has cancelled trips to three states and the Saint Patrick's Day reception at the White House. I suspect we won't be seeing Trump much after this speech.

Clay Claiborne