Featured Post

The white-Left Part 1: The two meanings of white

Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Sunday, June 3, 2018

Google decision to end military AI drone work reboots old memories

A major US corporation, arguably the richest and most powerful on the planet, has just adopted an explicitly anti-war decision to not renew a contract that would involve using artificial intelligence [AI] to kill people. It did this, in large part, because of opposition from its employees. Although the workers, like Google, stood to benefit economically, they rejected it on moral grounds. THIS IS HUGE! The application of AI to the art of war, which is the business of killing people, is one of the most dangerous technological frontiers facing us. It is the scientific nightmares of "Terminator" and "The Matrix" engineered into the living world. This nightmare future can only be created if the techie proletariat, the only ones capable of creating it, allow their skills to be manipulated by the masters of war to this end.

Now Google, propelled by its workers, has reversed an earlier decision that was pushing them, and us, closer to that dark future. It is understandable that this news has receive little attention in the corporate-owned media, but those on the Left should be demanding the same, or better, from all the other tech giants. Friday CNET reported:
Google reportedly not renewing contract for Project Maven drone program

Backlash against the company's involvement in the project led to the decision, according to Gizmodo.

By Abrar Al-Heeti
Google reportedly won't seek another contract for Project Maven, a Pentagon pilot AI program that could be used for drone strikes.

Google Cloud CEO Diane Greene made the announcement at a Friday employee meeting, people familiar with the matter told Gizmodo. Strong backlash against the company's involvement in the project reportedly led to the company's decision not to continue to pursue the project. Greene said Google decided to work on Project Maven "at a time when Google was more aggressively pursuing military work," Gizmodo reported.

According to BuzzFeed News, Greene said "We've always said this was an 18-month contract that we did, so it ends in March of 2019." She added, "There will be no follow-on to Maven." More...
Gizmodo covered this in story in greater detail, owing to their access to internal Google emails:
Google Plans Not to Renew Its Contract for Project Maven, a Controversial Pentagon Drone AI Imaging Program

By Kate Conger
1 June 2018
Google will not seek another contract for its controversial work providing artificial intelligence to the U.S. Department of Defense for analyzing drone footage after its current contract expires.

Google Cloud CEO Diane Greene announced the decision at a meeting with employees Friday morning, three sources told Gizmodo. The current contract expires in 2019 and there will not be a follow-up contract, Greene said. The meeting, dubbed Weather Report, is a weekly update on Google Cloud’s business.

Google would not choose to pursue Maven today because the backlash has been terrible for the company, Greene said, adding that the decision was made at a time when Google was more aggressively pursuing military work. The company plans to unveil new ethical principles about its use of AI next week. A Google spokesperson did not immediately respond to questions about Greene’s comments.

Google’s decision to provide artificial intelligence to the Defense Department for the analysis of drone footage has prompted backlash from Google employees and academics. Thousands of employees have signed a petition asking Google to cancel its contract for the project, nicknamed Project Maven, and dozens of employees have resigned in protest.

Google, meanwhile, defended its work on Project Maven, with senior executives noting that the contract is of relatively little value and that its contribution amounts merely to providing the Defense Department with open-source software.

But internal emails reviewed by Gizmodo show that executives viewed Project Maven as a golden opportunity that would open doors for business with the military and intelligence agencies. The emails also show that Google and its partners worked extensively to develop machine learning algorithms for the Pentagon, with the goal of creating a sophisticated system that could surveil entire cities. More...
Linux Hardware Partners T-Shirt ~ 1997 while Linux
was still in the "ignore you/laugh at you" stage.
A kind of class warfare has gone on at Google over many questions. It appears that for now the less greedy, more ethical side has won something on this one. That Google would take the lead on this, among the tech giants, would surprise few in the IT world. Google has not yet drifted that far from its Free Software Movement roots. Google was the product of the Linux community, and even today runs on Linux. Google became the principal financial sponsor of another Linux-based Open Source Software [OSS] project, Android, that short-stopped Apple's plans to monopolize the emerging smartphone market.

Google came out of a research project done by Larry Page and Sergey Brin when they were graduate students at Stanford. The project was nicknamed "Backrub," written in Java and Python, themselves open source projects, and run on Linux servers at http://google.stanford.edu. These were the early days for Linux. It was less than five years old.

with Linus Torvalds - 1996
About the same time Page and Brin were sending their first project web crawlers out to explore and catalog the Internet in March 1996, I was founding the first Linux Users group in Los Angeles. I was president of Linux Users, Los Angeles [LULA] for eight years.

While the focus was on the developing an open alternative to Microsoft world domination, the social and political implications of the technology we were promoting was always a part of the discussion. In 1999 we help launched one of the first websites designed to reconnect refugee families in response to what was happening in Kosovo. We were able to do this by networking with Linux users worldwide connected through Linux International, and since I am digging through old papers and emails that have no record on the Internet, this is the core of the argument I made to get things rolling, 15 April 1999:
Kosovo: The pen is mightier than the sword.

And it was printing that gave the pen that power. Well the PC is the modern pen and the Internet is the publisher. I’m not saying that occasionally the sword isn’t necessary, as I believe it is in the present instance. I merely want to point to the enormous power of information technology in the sphere (spear) of war. The Serbia government understands this. Just look at their website: http://www.gov.yu/kosovo_facts/index.html
“The Federal Republic of Yugoslavia recognizes and guarantees the rights of the members of the national minorities to the preservation, development and expression of their ethnic, cultural, language and other specific features, and the use of their national symbols, in accordance with the International law”
Yea, right. Goebbels never had it so good. In the face of their acts on the ground, these lies in cyberspace cannot go unopposed. Fortunately for us, it is not just governments who can set up websites. I think Dan summed up the two things that we are well equipped to do:
“1. carrying out a campaign of public education to educate the average Serb about the human costs of ethnic cleansing, and
2. documenting on video as much of what Serbia is doing as possible, and showing it to the Serb people.”
Certainly we can put together a website (or provide technical assistance to others) that exposes what is really going on. Facts can be presented as documents, reports, photos, sound and video. I agree with John on how this could be done. And certainly we can enter into a dialog with the enemy. If we can email them and can post to discussion groups, we can get our point of view heard. Sure our posters may be quickly torn down by the opposition, but its easy enough to re-post them.

I also know that it is almost impossible to charge hearts and minds of people given over to ethnic hatred. I’ve run into Southerners who are still fighting the Civil War, and these Serbs are still fighting a war that took place over 600 years ago. But I personally believe that the only ultimate salvation of humanity lies overcoming precisely such hatred. Therefore, I think it is worth the effort.
Although we were located in Los Angeles, we didn't feel such goals were beyond our reach because the Linux user groups community gave us contacts worldwide, including 85 in 19 cities in Yugoslavia, so soon we were sending out this appeal:
We represent a special interest group who’s goal is to assist in reuniting Kosovo refugees. Specifically, we are establishing an internet web server and database to facilitate refugees with locating families and loved ones. Your assistance\advice would be greatly appreciated towards accomplishing this goal. 
Our initial appeal led to some very interesting email exchanges. We thought it absolutely revolutionary that we could now communicate on a person-to-person level with ordinary people on all sides of a conflict half a world away. We were no longer prisoners of media reporting on a distant conflict. We could make our own news, and did. Now that the Internet has matured into facebook and Twitter, we expect this sort of thing, but in 1999 to be able to circulate independent reports and images of what was happening in Kosovo was mind blowing! Although, I have to admit that all Linux users didn't see things the way we did. I especially remember one fellow from Belgrade that suggested that the best way we could help out was to convince the Pentagon to use more Microsoft products.

Even then, that was a hard sell, the Pentagon was finding disfavor with Redmond's shaky products. When computer crashes can cost lives, half-baked solutions become hard to shallow.

Anyway, after an initial attempt to do our own thing, we discovered an NGO successfully doing the same thing, and folded our efforts into its. It was the first time such a thing was being tried. It was the first time, in eons of war, that such a thing was even possible! I argued in an email:
I think that one master database is the best solution for Kosovo refugees. That avoids duplication and allows for all efforts to be concentrated. It means one place to gather information, and one place to search.

It means one link on the web page for refugee information. One URL to publicize among relief workers, news media, refugee organizations, etc.

Thanks to the web, one site can full fill this worldwide need. Also, thanks to the web, all who want to can participate in this effort.

We are less interested in building another independent database than we are in supporting a unified effort led by those that are closer to the crisis.

What I think we can offer is the necessary systems infrastructure for such an effort, namely bandwidth and hardware. Thorough our ties to the Internet and corporate communities in Southern California, and especially given the current atmosphere, I feel confident we can scrounge an almost unlimited supply of both.  We can also provide people in this area with a wide range of programming and sysadmin skills to help to put together and maintain the site.

What domain name it is under, and who manages it, these are separate questions.  As you know, this can be done from anywhere in the world.  
We didn't know it at the time, but we were pioneering political uses of the Internet that wouldn't be fully realized for more than a decade with its dynamic role in the Arab Spring.

As Linux developed into an industrial strength system, the question Google is currently grappling with, participation with the war machine, became ever more pressing as the metal of this publicly owned ["copyleft"], labor donated, open source computer operating system was shown to best all others, and the Pentagon came a calling.

When the Windows NT system running the US Navy's new "Smart Ship," Aegis missile cruiser USS Yorktown crashed so badly that it had to be towed in 1998, we had great fun with it in Linux circles. We said that Microsoft had given "blue screen of death," a whole new meaning.

Soon the military was looking to Linux for the kind of stability and robustness they needed [ Imagine that: Stability and utility from a worldwide collective of volunteers versus a corporate behemoth! There is an important lesson here.] Soon, even tiny Linux companies, like my own Cosmos Engineering Company, were receiving lucrative contracts from arms makers like General Dynamics.

For some, including me, the beginning of the Iraq War in 2003 became a turning point. On 4 January 2004, I published this to the group's mailing list:
By some estimates the United States has killed more than 6.5 million people of color since the end of WWII. That is more human beings than were murdered in the Holocaust. That total includes maybe a million people killed by us during the Korean War, as many as 2 million in Vietnam, not to mention Laos and Cambodia, a million killed by sanctions in Iraq. Secretary of State Madeleine Albright thought that the death of 500,000 Iraqi children was "worth it" because Iraq had WMD, except now it appears they didn't. (Our bad).

Then there are the numerous smaller massacres like the thousand of Panamanians slaughtered and bulldozed into mass graves in the name of catching a drug dealer.

Already we have killed tens of thousands of Iraqis in the name of bringing democracy to those oppressed people. Exactly what our idea of democracy for the Iraqis is is illustrated by "A statement issued by the U.S.-led authority and broadcast by the Iraqi media network Wednesday, December 31, [that] said no individual or group is allowed to organize marches or demonstrations or even gather in streets, public places or buildings at any time without a prior from the occupation command." And the fact that all across Iraq we have been shooting unarmed demonstrators.

Up until now Linux has had very little to do with this carnage, but this is rapidly changing.

'The US Army has abandoned Windows and chosen Linux for a key component of its "Land Warrior" programme, according to a report in National Defense Magazine. According to program manager Lt Col Dave Gallop this is part of a broader move towards Linux by the US Army: "Evidence shows that Linux is more stable. We are moving in general to where the Army is going, to Linux-based OS."'

Linux has been designated the core OS of the Stryker Brigades being deployed in Iraq. The Stryker Brigades are based on a new rubber tired armored vehicle, the Stryker. These new units represent the cutting edge of a military that is rapidly retooling itself from being a defense against a creditable threat like the USSR, to becoming a instrument of repression of third world people's struggle for self-determination. The Stryker brigades are also highly computerized. Plans call for complete computerization down to each individual soldier wearing a computer, and all those computers will be running Linux. General Dynamics has the contract. Time was they hired my company, Cosmos Engineering to do Linux development for them, but they are way pass that now. Embedded Linux is also making important inroads in the military. It is starting to show up in smart bombs and cruise missiles.

Since it's deployment in Iraq, the Stryker Brigades have been among the most brutal of the occupation forces and this brutality is not unrelated to the computer game cocoon in which many of our forces operate. For example while we have been celebrating the Christmas holidays, soldiers from the Fort Lewis-based Stryker brigades cordoned off the city of Samara, conducted house to house searches and shot anyone who resisted. "The soldiers move cautiously into this clutch of concrete-block homes, rousting the men, women and wide-eyed children. Uninvited, the Americans head inside, poking through cabinets, closets and shelves, and peering beneath the beds...this is the boot of the American occupation"

"It provokes people's hatred," said Mohammed Ibrahim, imam of a Sunni mosque near Forward Operating Base Vanguard. "When your life is threatened with raids, and you are expecting that everyday, life becomes so cheap, so worthless that you would rather die than live."

How many innocent people did we kill over the holidays? We are spared from knowing because, as an Army spokeswoman said "We don't take Iraqi casualties" and we have forbidden the Iraqi authorities from counting the dead.

If the pentagon has its way, 2004 will be the first year that Linux is used for killing on a wide scale. I for one oppose this. What meager contributions I have made to Linux over that past 8 years, including founding this organization, have been done because I see it as an instrument to advance humanity. I will not see it turned into a instrument of death and destruction without a fight.
....
In the past my company has done work for the Army, Navy and Air Force, not to mention General Dynamics, Lockheed, Rockwell, and Boeing. I have sinned in the past, and I know how tempting fat military contracts can be, especially now when development dollars can be hard to come by. But the question facing my company, and the rest of the Linux community, now is are we developing the brains of a monster?
When another LULA member complained that anti-war posts were a violation of list etiquette,  I responded, 5 November 2003:
As our President and many other officials have made clear recently we intend to stay in Iraq no matter what. At the same time we are facing growing popular opposition to our occupation. What we are planning, and in fact carrying out, amounts to little more than mass murder because we are clearly willing to use whatever level of violence is needed to conquer the Iraqi people. Already we are selectively putting certain communities in 'lock down,' and killing or imprisoning anyone who dares oppose us. We are even shooting at journalist who try to cover it, although frankly most are so tame there is no need.

The poor Americans that have been forced to turn to the military for a job will be forced to do the killing and dying. Fortunately for us, most LULA members do not fall into that category. All that is required of us is nothing. Just carry on business as usual and let it happen.

It is my position that war is not just another subject. It is my position that when your country embarks upon a war of aggression against a much smaller, weaker country, and starts killing other people in large numbers, you lose of your right not to be bothered by it. Because we are such a powerful country, we can be in the happy position destroying another country, killing thousands of its people, and expect that it should not even be brought up in a Linux meeting. What 'white man's burden' arrogance! It has been said that all is fair in love and war. While I don't know that I would go that far, I will say that I am not terribly bothered if I infringe upon netiquette in opposing the war.

I just finish watching ABC's Nightline, which was sponsored tonight by IBM, with a Linux commercial no less. Of course they talked about the American causalities. In a more perfect world they would have also talked about the much more numerous Iraqi casualties caused by us. Then I wouldn't have to.
And in another email:
My words aren't 'inflammatory.' Cluster bombs and depleted uranium shells are. From today's L.A.Times (11/4/03) "The attack Friday night destroyed two houses in the village of Warez [Afghanistan] in the eastern province of Nuristan, killing four children, a woman and a young man..."

Let's try another tack. Transport LULA and this disagreement in time and place. Say to Nazi Germany. Would I be wrong to speak in opposition to the extermination of the Jews among Germans that found such talk 'inflammatory' and 'off topic'? Or the United States in the mid-19th century. Would I be wrong to speak out against slavery even if it was considered a divisive subject? I think not! But since at that time there would have very likely been LULA members who favored slavery and the killing of Jews, I can only assume you would have found such talk a grave violation of etiquette. Of course hindsight is easy. I seriously doubt if today any LULA member supports slavery or the killing of Jews. That is why it is so easy to take a stand on historical questions, but it is also irrelevant to do so. Today it is people in Iraq and Afghanistan that are dying, and it is we who are doing the killing. And since there are very powerful economic interests that favor this, (as there was at the time for slavery and the holocaust), the reality of these wrongs is somewhat muddied.

The Bottom Line is This:

War is simultaneously the most coercive and most destructive of all human endeavors. With the current levels technology it will lead to the complete destruction of humanity, quite possibly in our own lifetimes. It is quite obvious that war is forced upon those that are the subject of aggression. The people of Baghdad were given no say but to suffer the effects of 8,000 or so cruise missiles slammed into their town for example. But even in the aggressor nation it is generally a matter of coercion for a majority. Even with our "volunteer" army, many feel they have no real choice, and as for the financing of the enterprise, the money is appropriated from those that oppose the aggression as well as those that it support it.

It is because of this coercive nature of war that those that oppose it cannot, I repeat CANNOT, confine their activities to those who would 'opt in' to such opposition. To allow those who promote the war, and those who think they have no part in it, to go about there business in peace, is to allow the war to continue even if the majority oppose it. This simply is not an acceptable solution to the problem of humanity's success.
When I resigned as president of LULA in April 2004 to focus more of my energy on anti-war work, including producing my first feature length documentary, News Clips from the Iraq War, which was streamed on the Internet years before YouTube [which runs on Linux], by the Linux Public Broadcast Network [LPBN], I wrote:
I once had high hopes for Linux. I felt sure it could make a real contribution to the success of humanity, now more and more I have my doubts. I have a real and growing fear that if the Mr. Smith’s of Linux [a Matrix reference. Matrix was all the rage with the Slashdot crowd.] have their way, in the future they will look back and say “Wasn’t it nice that so many smart people worked to hard for free to forge their own chains."
Newsforge ran an article with the title LUG Pres Resigns Over Military Linux Use, and when Joe Barr posted a link to Slashdot with this comment:
"NewsForge is carrying the news that the founder and president of Linux Users Los Angeles (LULA) has resigned because of his opposition to the war in Iraq and the U.S. Armed Forces' use of Linux."
It ignited a discussion [i.e. flame-war] that extended to 1361 comments.

In the fourteen years since that discussion, the Pentagon has largely been able to count on the greed and cooperation of the tech industry. This new decision by Google represents an important move in the right correct direction. It makes me hope that a little of its open source ancestry lives on at Google.

My other blogs on Google:
Google correct the record: Little girl was beheaded by Assad not ISIS [GRAPHIC!] 4/30/17
Why are they protesting Google?  6/19/13
Coming Soon: Obama versus Google 5/19/11
Google Supports Revolts | Anonymous does too! 2/27/11
The Google Search for Wael Ghonim 2/9/11
Google Goes Rebel, Supports Egyptian Protest 2/6/11
The Internet Takeover: Why Google is Next 12/05/10
EFF on the Google\Verizon Net Neutrality Proposal 8/30/10
Court rules -> Google Must Be Evil & Maximize Profits 8/30/10
Google-Verizon: What is the Free Press Agenda? 8/13/10
Will Android make Google Money? 8/10/10
Google Verizon Announce Terms of Deal 8/9/10
Why I like Google: Reason #38 6/6/08

Syria is the Paris Commune of the 21st Century!

Click here for our posts on the 2016 US Election
Click here for a list of our other blogs on Syria
Click here for a list of our other blogs on Libya

Friday, June 21, 2013

Trust but Verify: How to Download YouTube Videos

YouTube videos are playing an important role in documenting the social news of our time.

This is the way news breaks today, whether it is on the battlefield in Aleppo or a protest in São Paulo, someone uploads a video to YouTube and then Tweets it out. Often it is completely unedited recordings from a cell phone or camera and not minutes old.

I have used hundreds of these YouTube videos in my blog posts at WL Central, the Daily Kos and here, on the revolutions in Libya, Syria and elsewhere in MENA and over time I have noticed a troubling phenomenon: Some of them have disappeared.

Most often they are replaced by a black screen with the notice "This video is no longer available because the YouTube account associated with this video has been terminated."

Whenever I see that, I think "terminated," that's a bad word to use in connection with activists and journalist in a revolutionary situation. It brings me visions of someone sitting in front of a computer and deleting their YouTube, Facebook, Gmail and Twitter accounts at gunpoint just before they, themselves are "terminated."

Hopefully, the reality is not that grim, but the fact remains YouTube videos have occasionally been known to disappear for whatever reason and unless there is a downloaded copy elsewhere, a piece of recorded history may be lost forever.

We must also guard against the future possibility that a change in management at Google or a change in ownership of YouTube could lead to a change in policy and the wholesale deletions of material that, shall we say, may prove an embarrassment to the powers that be. That is why I say "Trust but verify." So far, I think YouTube has been a pretty good steward of the people's video history but we need a backup.

I am writing this post in the hopes that many activists will start creating their own collections of YouTube videos from the many blogs, tweets and other sources covering the people's revolutionary struggles around the world. If it's important to you, download it and help insure that it will never disappear.

With the right tools it is easily. I use Linux and the Linux command line or terminal interface, so I will show you how I do it. No doubt, there are other ways to do this on a Windows box or a Mac, and if you know them, please share what you know in comments.

Linux and especially the command line interface [CLI] may seem a bit intimidating at first, but as you will see, its really not that hard and can be very powerful. And look at it this way, if you use Google, you already use Linux. It runs their servers. Hell, If you use the Internet or own an Android, you already use Linux.

Moreover, if you think yourself progressive or revolutionary, or if you want the best possible foundation for keeping an intrusive government out of your computer files, you really should get to know Linux. On the other hand, if you want to make it easy for the NSA to just browse your hard drive at will, just keep using Windows.

And as a bonus, since all the software we will use to download YouTube is Free Software, this solution won't cost you anything.

If you don't already have Linux I suggest you download the latest DVD image of Ubuntu Linux with this link. There are more favors of Linux than there are of socialism. Ubuntu is the world's favorite, mind too, so we will work with this. Ubuntu is an ancient African word meaning 'humanity to others'. It also means 'I am what I am because of who we all are'.

You will them need to burn it to a DVD. The same page that the link above leads to will also have help in how to burn a DVD from Windows, OS X, and Ubuntu in the lower right-hand column. There are also instructions for creating a bootable USB stick for those that don't have a DVD drive.

These instructions were tested on Ubuntu Desktop version 12.10.

Once you reboot from the DVD or USB stick, you will be given the option to run Linux without installing it, most likely this is what you want at first. Click on the [Try Ubuntu] button.


Once the Linux Graphic User Interface [GUI] is up, you will need to open a terminal, either with the keyboard combination Alt-Ctrl-T or by clicking Dash Home [Ubuntu logo, upper left corner], typing term and then clicking on the terminal icon.

Either way, a terminal will open and you can now type commands into the command line interface.

The key to our ability to download YouTube videos from the Linux CLI is a little program called youtube-dl, if you follow that link you will see that they also have a Windows version. We are interested in the Linux version.

That doesn't come with the standard Ubuntu install but it is easy enough to add. First we have to enable the "Universe" Ubuntu software repository and rather that repeat what has already been well said, I will refer you to this link: How do I enable the “Universe” repository?

Once you have done that, you can return to the terminal (click the mouse cursor anywhere in the terminal window.) and type:

sudo apt-get update

this will update your local software list:

Then type:

sudo apt-get install youtube-dl

Once you answer by pressing Y, it will download and install youtube-dl and a number of other packages it requires.

You can read more about the sudo and apt-get commands by clicking the names.


If you install Ubuntu Linux on your hard disk, you will have to perform the above steps only once, because the LIVE Version runs entirely in memory, all changes will be lost as soon as you reboot. Also because the LIVE system runs on a ram disk, you will need change to a suitable directory on a mounted drive so that the downloaded YouTube videos will be saved to the hard drive.

Most likely at suitable drive was mounted automatically and mounted with write permission. Otherwise you may have to do a little extrema work and get familiar with the mount command.

Look at the bottom of the column on the left and you should see the icons of your mounted drives. You may browse these to find the drive/partition and directory you want to save the YouTube videos to. Remember the PATH to that directory given at the top of the file manager. For example if it says media windows pictures
you are going to want to type
cd /media/windows/picture
in the terminal window to change to that directory. You can read about the cd command here
Here is a helpful hint: Open this page in the Linux Desktop and then you can simply cut-n-paste these command line instructions to the terminal command line. That makes this stuff real easy. You can open Firefox with the logo button on the left side.
youtube-dl is an amazing little script. You feed it the URL of a YouTube video and it retrieves the page, captures certain meta-data about the video and downloads the file to your computer. The command you might type into the terminal could look something like this:
youtube-dl -t http://www.youtube.com/v/YncVKgpTd6E

The -t option tells the program to include the video's title in the filename and is useful in identifying files. In this case: Google_s_domination_of_the_world_and_loss_of_mojo_谷歌征服世界-YncVKgpTd6E.flv

The .flv extension indicates that it is a flash video file. YouTube videos are generally either flash [.flv] or mpeg4 [.mp4] and to view them under Windows you may need to install a new video player or new video codecs.

Mplayer is a good video player for them under Linux and ffmpeg will allow you to convert them to other formats such as those used on DVDs or suitable for editing.

You can capture the URL of an embedded YouTube video by hovering over it, clicking the right mouse button and choosing "Copy video URL" at the top of the list.

This brings us nearer to the real power of using youtube-dl from the command line because the key to that power is not the ease with which it can download a single video but the ease with which it can download many videos.

Because the terminal understands the bash scripting language natively a simple command like
for f in `cat YT-blog.lst`; do youtube-dl -t $f; done
will take a list of YouTube URLs, each one on a separate line, and one by one, download them all to your hard drive. In this case the list is in a file named YT-blog.lst in the working directory and looks something like this
http://www.youtube.com/v/YncVKgpTd6E
http://www.youtube.com/v/yOLc3B2V4AM
http://www.youtube.com/v/YQqjM8WIpM0
http://www.youtube.com/v/YtTUsqra-MU
http://www.youtube.com/v/Yx3WjOMivoU
http://www.youtube.com/v/z6VBcpsm9VI
http://www.youtube.com/v/ZEMxLWZjdoI
http://www.youtube.com/v/zJu6VQHH714
http://www.youtube.com/v/zOIyOyrxsqU
http://www.youtube.com/v/zQRHUqkZ2pI
http://www.youtube.com/v/zy7i3Xb2N40
http://www.youtube.com/v/Zybm3iaHcmc
http://www.youtube.com/v/zzt7LLT1XX8
You can read about the cat command here, the for loop is an internal bash command.

Such a list can be created in the Ubuntu environment by opening a text editor [ click on Dash Home (Ubuntu logo in upper left corner, them type 'text' and click on the text editor icon when it comes up. ] and pasting the


copied URLs to it, one per line and then saving it to the filename you will use later.


This file could also be created beforehand in the Windows environment with Notepad and Windows Exploder and there are even Unix like environments for Windows that might allow you to use these commands under Windows, but I won't go there.

A more advanced example, using other tools in the Linux environment, such as grep, sed, sort, and uniq and by piping '|' a text stream between them I can create a filter that can read the entire xml download of my blogspot blog [filename: blog-06-07-2013.xml ] and create a file named YT-blog.lst

The command that did this look like this:
grep "http://www.youtube.com/" blog-06-07-2013.xml|sed "s|http://www.youtube.com/|\nhttp://www.youtube.com/|g"|sed 's|"|\n|g'|sed 's|?|\n|g'|sed 's|&|\n|g'|grep "http://www.youtube.com/"|sort|uniq>YT-blog.lst
So by going to my Blogger dashboard->Settings->Other

choosing Export Blog and exporting my blog to the above file and them running the above command on it to create the video URL list and then running this command on that list:
for f in `cat YT-blog.lst`; do youtube-dl -t $f; done
I was able to download the over 900 YouTube videos that I used in over 600 Linux Beach blog posts that still remain on YouTube. It took 2 days on my slow ass DSL line and takes up 17GB on my hard drive but I will no longer have to look at a video has been terminated notice and feel powerless to do anything about it.

I only hope this blog post will encourage other people to do the same and between all of us, we can preserve this video history of the people's struggles to such a depth that it can never be erased

Click here for a list of my other blogs on Syria
Click here for a list of my other diaries on Libya

Wednesday, June 19, 2013

Why are they protesting Google?

I've known and used Google even before it was a corporation, so when I heard they were going to protest Google in Venice, I went just to see what it was all about.

Google was started by a couple of guys at Stanford who had the lofty goal of making all of humanity's knowledge available to all of humanity. In the 16 years since they started, they have come incredibly close to fulfilling that goal.

At the protest, I was turned off early by what appear to be harassment of two black, female Google security guards that obviously had no clue that this protest would be happening by the leaders of this group of ~50 demonstrators. After Mark Lipman finished his argument with the security guards, he went on to complain that Google provides "no jobs and no opportunities for the poorest in our community, who need it the most."

It seems that Google is to be blamed for "gentrifying our community." Apparently they pay their employees too much. If they moved a sweetshop to Venice would that satisfy this group?

True enough, Google requires mainly highly-skilled employees. The future requires mainly highly skilled employees. Have these protesters investigated the many ways Google has made it relatively easy to gain those skills, even in some of the lowest income communities in the world?

And BTW, Google has a strong history of hiring from their community, the Open Source community. When I was president of Linux User, Los Angeles [LULA] they would regularly recruit through me because they believe in hiring first those who had contributed to this community.

In 2004, when Google was in Santa Monica, near where the anti-war movement was protesting Bush at SM airport, the Google office practically emptied out to join the protest. Afterwards, a Google employee that had come from LULA invited me in for free sodas and a tour.

Most people use the Internet everyday. Few have a deep understanding of how it was built and by who.

Speaking of Google employees, everybody knows of (1) Larry Page, (2) Eric E. Schmidt and (3) Sergey Brin. How many know of David C. Drummond, a black man who joined Google 11 years ago and is number 5 in their management team. I mention this so you don't think they just hire black people for "lowest income" jobs.

After I returned from the Google protest, I watched an Al Jazeera report on Google's latest project to fulfill their mission: Solar powered WiFi access points lofted high with hot air balloons. Google sees it as a way to extend Internet access to the least connected and poorest people in the world.

One of the demands of the protest was that Google "give back" to the Venice community. The day before the protest, some Google engineers were giving a programming class for youth at the Abbot-Kinney library. I didn't hear anybody at the rally mention that. Did they even know about it?

Even before Google moved to Venice, I was told, on the q-t, by the person delivering food to homeless people in Venice, that Google was funding the regular food drops.

Last year, Google came in at #1 in CNN/Money/Fortune list of "100 Best Companies to Work For," so obviously they should not be welcomed to Venice. Do these protesters know that all Google employees can spend 20% of their time, that's one day in five, on personal projects? How many corporations do that?

That's how Google engineers, with official Google backing, were able to build the first Speech-to-Tweet system when Mubarak tried to shut down the Internet in Egypt during the revolution. Google, working with Twitter, made sure that anybody with access to a land-line could still tell the world what was happening in Egypt. They did the same thing when Qaddafi tried to shut down the Internet in Libya. I know of no US corporation that did as much for the democratic movement in MENA.

In Nov-Dec 2010, when the first indications of what would be called the Arab Spring were developing, Google paid schools in Cairo to help improve Google Translates Arabic. That would prove extremely important in the coming months. I personally know of at least one use that may have saved lives, and saved the day, in Tahrir Square. Likewise, they rushed to improve their Farsi when people were in the streets in Iran, 2009.

"We feel that launching Persian is particularly important now, given ongoing events in Iran" said Google principal scientist Franz Och in a statement on Goolge's Official Blog.

And remember Wael Ghonim, the Google employee that played a leading role in the Egyptian uprising? After Wael Ghonim was released from the custody of Mubarak's thugs, he said he would like to return to work at Google if he was not fired. Outgoing Google CEO Eric Schmidt tweeted back "We're incredibly proud of you, @Ghonim, & of course will welcome you back when you're ready."

Google started out as an Open Source project. Like many successful Open Source & Free Software projects, it had to become a corporation to accomplish its mission. That's just the way the world works now.

But even though Google has become a giant corporation, I think I can accurately paraphrase Churchill by saying "Never before has a corporation done so much for so many for so little cost to the user."

I first discovered Google at the 1998 Linux Expo in North Carolina. They were just a Linux search engine working out of a garage then. They were, and are, a member in good standing of the Linux, Open Source or Free Software community.

True to the Free Software creed, they operate on a very different intellectual property model than all the other corporation in the world that are not part of that community.

Did those protesters even know that Google doesn't own Android? True Google has contributed heavily to Android development, but the software remains under a GPL or "copy-left" licensing. [Apache Software License, 2.0] Anybody can build an Android device without paying Google a single dime. Just try that with Iphone software. There is a big difference.

They say charity begins at home, but home for Google is not Venice, in spite the big building here that they have taken over. Google is a citizen of the world and home is the Free Software movement that spawned it. So if you want to look for Google's charitable side, you should look at all that they have done for Free Software. First, they continue to make most of the software they develop freely available under a GPL like license, then there are grant programs like the "Summer of Code"
Google Summer of Code is a global program that offers post-secondary student developers ages 18 and older stipends to write code for various open source software projects. We have worked with open source, free software, and technology-related groups to identify and fund projects over a three month period. Since its inception in 2005, the program has brought together over 6,000 successful student participants and over 3,000 mentors from over 100 countries worldwide, all for the love of code.
If you use the Internet, a Droid phone, or many other Internet appliance, you benefit from code paid for by Google and made available for free. Microsoft only does that by way of exception, and then only because they have to compete with the Open Source Community.

Much of what we know about Internet censorship in China is because Google very publicly fought it. All the others, Microsoft, AOL, Yahoo went along with the Chinese program without so much as a mummer of public protest.

It was much the same when Bush started demanding access to people's search records in 2006. Again, all the other search engines went along with the program. It was only when Google made clear they were going to fight it that those plans were abandoned.

Google has also taken a strong position in fighting bogus claims of copyright infringement and in support of the "fair use" exception to the copyright laws. For that reason my Vietnam:American Holocaust remains up on YouTube [434,929 views] even as it was removed from Facebook because they don't recognize fair use. When Google received a copyright infringement claim about one piece of music I used, they didn't take down this very valuable anti-war film, instead they put a link to where viewers could buy the song.

The Open Source movement is a revolutionary movement, I call it communist software. Oddly enough it was through my connections to this movement, rather than the Left, that first alerted me to the rising "Arab Spring" before anyone else in the LA Left.

Obviously, I have a long association with this movement, and with the most significant corporation spawned by that movement, Google. That's why I think Google is the best corporation in the world and that's why I come to its defense.

Since this protest has adopted as its slogan "Google: We're Watching You!," I have to assume they have seen everything I have related above. What I can't figure out is why, of all the corrupt corporations in the world, they choose to target Google.

Personally, my protest time will continue to be focused on Syria, where an uncaring world is letting 200 people a day be slaughtered by their government, with the exception of occasional flurries into other areas, like this one.