Monday, 31 August 2009

GIH servers suffer sustained DoS attack

After spending a couple of days off-line, I found out on Sunday 30th August that our servers in London had been hit by a denial of service (DoS) attack through mail-bombing from different sources.

This happens every now and then. The last time, it took place when I was at the ICANN conference in Cairo, and another European participant's systems had been hit at the same time. Thankfully the GIH servers in London survived then, and thankfully, they survived (barely) this time.
The strength of the attack this time round was higher than before since every time there is an attack, I find out what the weak point is in our network and upgrade it. I'm glad that the servers actually manage to recover each time, but it is still very disruptive indeed.
So this time round, the main mail hub got attacked with 360 emails per second from zombie computers all around the world, for a very sustained amount of time, starting at 8:00pm on saturday night. As a result, the server ran out of memory+swap (500Mb + 500Mb) and froze by going into some kind of panic recovery mode, thus sending the attack to our backup route, and this then sent the emails to our back-end machine via another path, in a more controlled manner (aka - via UUCP over IPv6, the new with the old working very well together). As a result, the back-end stored 28 000 emails, all but 200 being spam sent to wrong addresses.
I managed to reboot the main mail-server remotely in a short window of time that it allowed me to log in. It took 30 minutes to shut down, so clogged were its processes, running at a load of 58. I spent sunday afternoon trying to find out how to process the backlog of emails, bearing in mind, more was coming in. It felt like dealing with a flood.
Finally, I commissioned a third computer running Linux, which I had kept running for the past 4 months as standby and which has now taken the load off the front and back end machines and is shifting through the now 24 000 remaining emails. All in all, the GIH computer systems in London have filtered 98 000+ spams in 24h. That is a lot more than at any time before. If this is a taste of things to come in the near future, we're going to have to beef those servers more with a lot more processing power.
When I remember that the first computer system for GIH.COM was a 20Mhz 80386 running linux & 4 Mb memory, downloading emails via UUCP and sending them via SLIP/PPP through a half hourly telephone call using a 9 600 baud modem, and it was running smoothly, even when downloading a few USENET newsgroups that I enjoyed reading...

...I just wonder where this spam is leading us to.

...sigh...


Sunday, 9 August 2009

Green Networking - Google's Data Centres

Google's released an interesting set of documents and videos out detailing their set-up in their brand new Green Data Centres.

You can check it out on:

http://www.google.com/corporate/green/datacenters/summit.html

Whilst other industries have been under intense pressure to reduce their Carbon emissions, the ICT Industry has, so far, not done that much. Of course, the energy rating of computers and telecom equipment has been in the spotlight and most hardware manufacturers have displayed Energy Star ratings, but our ever increasing thirst for information has asked for faster and faster data centres. And that means more servers, more speed, more energy consumption and more heat to get rid of.

As other industries have complied with a reduction of carbon emissions, so will the ICT industry. Google's current initiative is pioneering. Others will no doubt follow. The physical topology of the Internet network might be changed radically, as data centers located in the middle of towns make way for out of town more secure mega-facilities built close to cheap and clean energy, and expandable at will.

NGN - clean slate or not ?

Attending meetings about Internet Governance, I keep on hearing how so many things will influence the architecture of the Internet. A lot of the heated discussions tackle the short term instead of the long term. Furthermore, some governments are under the impression that a "Next Generation Network", aka NGN, will be a Clean Slate approach to networking.

For the record, let me say here that these are pipe dreams.

Apart from increasing control of the next generation network by governments and multi-national corporations, I cannot see a single reason why a next generation network should be created out of a Clean Slate approach. The Internet might not be perfect, but its imperfections are being worked on. At the very least they are known!

Currently, there are thousands of engineers around the world who are working on ways to remedy the Internet's Achille's heels. The experience which has been acquired thus far by running an international network spanning the whole world, is more valuable than any theoretical study that will ever be conducted. The amount of "known knowns" and "known unknowns" is vastly superior to the amount of "known knowns" and "known unknowns" in a Clean Slate approach. In fact, I'll venture out to say that the danger, as we all know, is the amount of "unknown unknowns", which the Clean Slate approach is full of.

So what do we really know about the Clean Slate approach? Nothing. In fact, at the time of writing, I suspect that the people proposing the Clean Slate approach don't know either.

How will it perform? How reliable will it be? Will it be hacked? Will it be secure? Will it encourage innovation? Will it encourage democracy? Will it be embraced internationally?

Perhaps should we find answers to these questions before launching into huge development costs for a programme which might yield little more than few gigabytes of presentations and writings.

TOOL: Google Wave

Taking part in working groups both at ICANN, ISOC, but also IETF, I spend a fair amount of my time in conference calls.

Each organisation has its own preferred tool for online collaboration. Whilst some of them involve contracts with a commercial provider, others are "free" - as in they might be test systems or freeware. If all else fails, we sometimes use Skype. But finding a proper tool that can let you share documents and presentation material, speak to each other in a meaningful way, interface with the real world, as well as allow for simultaneous text input is easier said than done. The bottom line is that we've *always* has a problem with communications.

The VMEET working group at IETF has been formed specifically to find a solution to this problem, whether it is evaluating what's out there, or drawing up specifications for a new set of tools. Its findings risk being very helpful for other organisations, since an increasing amount of collaborative work is required if the Internet model of governance is to be sustained. For more information on this exciting challenge, go to: https://www.ietf.org/mailman/listinfo/vmeet

That said, Google seems to have come up with its own version of online collaboration. Not quite ready yet, but there's a preview on:

http://wave.google.com

It looks like it has potential, especially with extensions, since it follows the Open Source concept.

50 great examples of data visualization

The following link is a particularly well researched/documented blog entry about data visualization.

http://www.webdesignerdepot.com/2009/06/50-great-examples-of-data-visualization/

It has links to many very interesting visualization engines and concepts. If, like me, this sort of this fascinates you, I recommend setting aside a few hours before you embark on looking at this page. :-)

Visualisation is, of course, a major part of our cognitive processes and we, as humans, will probably require increasingly complex visualization tools to enable us to make a more complex world easier for our limited minds to understand. But looking through many of these examples, it also struck me that a great deal of analysis often took place before visualization was even possible, and I wonder whether some of the examples are not merely enabling our mind to understand, but also open the door for machines to understand each other.

How? By searching for ways to format data in a form which can be read and displayed by a machine (after all, graphical display tools are run by computers), we are stumbling on the possibility of that formatted data to be used in other ways than just being displayed on a graph.

Another thing which struck me is the worth of data visualization in reminding us of the past. I had lunch earlier this week with a French philosopher who advised me that unfortunately, one common human trait is the ability to forget the past too easily. As a result, mistakes are repeated and only a fraction of knowledge is transmitted in the long term. Trend patterns are completely obliterated.
Take a piece of software like "Flare", for example, used by some of the data visualization examples above, but not directly referred to by the article I point to above. One example which I was particularly impressed about was their "Job Voyager":

http://flare.prefuse.org/apps/job_voyager

Can you see which jobs you should avoid because they are, literally, dead ends? :-) Yes, reminding us of the past can point us to the future.

I hope you enjoy the visualizations.

Friday, 25 July 2008

One Web Day

Today, Friday 25th July 2008, I have accepted to be a "One Web Day Ambassador" for the day, promoting its cause by publishing a story at:
http://onewebday.org/stories/?p=40

It tells the story of how the Internet has changed my life over the past 20 years and I encourage you to read it because it might stimulate some reflection in your mind about a network of networks that we take for granted but is under threat.

OneWebDay is an Earth Day for the Internet that takes place each Sept. 22.
Its main Web page is: http://www.onewebday.org

This year, 2008, is the third OneWebDay. Around the world, we're focusing attention on the importance of the internet to political participation - that's this year's theme. We're also encouraging people to talk about (and do something about) internet issues they're worried about - censorship, the digital divide, inadequate connectivity generally.
The idea behind OneWebDay is to create a platform for a global constituency that cares about the future of the internet. We make progress when we make things visible, and with OneWebDay we're showing this global constituency that it exists.

To start brainstorming about possible actions on 9/22 itself, see:
http://www.onewebday.org/base/index.php/OneWebDay_in_a_box

I highly encourage you to get involved: http://www.onewebday.org

Olivier

OneWebDay

One Web Day: Living the Internet

Living the Internet is my contribution for One Web Day Stories, as found on
http://onewebday.org/stories/?p=40



Foreword


I first heard about the OneWebday project a few years ago and always felt too busy to contribute in any meaningful way, be it by becoming an ambassador, or by writing an actual story for the event.


Since I’ve lived the Internet (as I’d like to say) since 1988, it occurred to me that perhaps this year was the right time to write. That’s nearly 20 years of Internetting. Naturally in 20 years, the Internet has changed my life not once but many many times and in my contribution, I’ll focus on 4 main stories which might be of interest to everyone.


Each story contains a lesson. I hope you’ll find them an interesting read but most of all, I dearly hope that we’ll all remember those lessons.


1. Discovery



The story of my first week on the Internet.


VAX 11/785

VAX 11/785


I first discovered the Internet in late October 1988. I was at university (King’s College London) and was logged in one of these (now) antique computers called a VAX 11/785 running an operating system called VMS.

The first distribution list I subscribed to was the Virus-L Discussion list, a discussion about computer viruses. I had an interest in the subject ever since I had heard the story of the (C)BRAIN Virus. Although the concept of a Computer Virus had already been imagined over 40 years earlier by John von Neumann, I wondered whether such a critter could be applicable to this network of networks that I had heard so much about.


Amazingly, 3 days after I joined (2nd November 1988), the Internet crashed as a result of a worm written by Robert Tappan Morris, a 23 year old Cornell Graduate Student. In short, Morris had miscalculated the speed of the Internet. Re-infection of computers that had already been infected took place at a much higher rate than he originally imagined. Multiple reinfection meant a sudden peak in traffic and slowdown to a halt. Being located in the UK and accessing the Internet through a local academic network called JANET (Joint Academic Network), our computer systems were not really affected because TCP-IP was not running in native mode on JANET.


However, I had access to the Internet through a gateway at University College London. We got immediately cut off for a while. Then a flood of emails came in from the Virus-L Discussion List, with on the spot reactions from system managers all around the United States. It was like watching a movie, “victims” e-dying, a task force forming itself, a counter-attack being spontaneously set-up and a final defeat of the rogue code. I was also reading messages on USENET - which still used UUCP dial-up, so it was a resilient path to have information distributed.


My first week on the Internet showed me how amazing a communications medium this was, how much of a giant it was going to be, but also how fragile this giant was. It was a lot to learn in a full week. Today there are fewer resilient channels of communication than there used to be. USENET has all but faded away. TCP-IP rules everywhere. This might be its strength and its weakness. Beware of Achilles heel…



Lesson 1: the Internet is a fragile resource. Take care of it.


2. Chinese Dreams


A story about Freedom.


On 9th June 1989, I received a forwarded message in my mailbox. It was an email that had originally been sent by a Sun Microsystems employee in Beijing on May 23rd 1989, relating the situation on the ground in Tiananmen Square. The message was both reassuring and (now we know) naive, signaling that everything was fine in Beijing and the dream of democracy was finally coming true. We all know what happened on 5 June 1989.



I am including an excerpt of the original message here.


> From: GROVE::ZDEE042 “Princess Leia” 9-JUN-1989 11:57


> To: ZDEE036,ZDEE038,ZDEE041,ZDEE699,ZDEE762,ZDEE763,ZDEE764,ZDAP808,UDEE705,ZDAC128,ZDAC131,ZDAC161,ZDAC166,ZDCA717

> Subj: illusions of students in China. anybody knows what happened after!!

>

> From: zmacv61@uk.ac.ic.doc 7-JUN-1989 21:57

> To: ZDEE042

> Subj: letters from china

>


>

> Date: Wed, 7 Jun 89 21:47:04 BST

> From: zmacv61@uk.ac.ic.doc

> To: zdee042@uk.ac.kcl.cc.elm

> Subject: letters from china

> Message-ID: <8906072148.aa10542@flamingo.doc.ic.ac.uk>


>

> the header to this makes just as interesting reading as the letter! it has

> got around, quite a bit, i can tell you!

> ============================================================================


Further Headers deleted - for reasons of confidentiality.

The path taken to reach me from the other side of the world was roughly:

Beijing -> Hong Kong -> Japan -> Palo Alto -> Rest of Silicon Valley -> MIT -> Princeton -> Edinburgh -> Oxford and then London using UUCP, as well as TCP-IP and several other email protocols to go from China to my desk.



We pick it up at Hong Kong. I have replaced all addresses/names by [XXX].


> >

> > The network is a wonderful thing. This was sent “this morning” at 4 am (Beijing

> > time). Much more interesting than anything I’ve read in the Chronicle.

> >


> >

> > - XXX

> >

> > - —– Begin Included Message —–

> >

> > > From XXX@XXX Tue May 23 09:57:25 1989


> > To: XXX@XXX, XXX@sun.com

> > Subject: A Beijing status report

> > Status: RO

> >

> > [ XXX ] and [ XXX ]


> >

> > I thought I would share this personal account of what’s happening on

> > the streets of Beijing. The writer is our own [XXX] . The “XXX”

> > referred to is [ XXX ].

> >


> > I apologize to members on both lists for receiving duplicates of this.

> >

> > - [ XXX ]

> >

> > - ———————————————————————

> >


> > > From sunhk!sunbj!XXX Tue May 23 04:39:34 1989

> > From: sunhk!sunbj!XXX (XXX - Sun Beijing XXX)

> > To: sunhk!sun!sun!XXX

> > Subject: Re: hello??

> > Cc: sunhk!XXX


> >

> > Yes, I am all right. Thank you, my friend.

> >

> > The situation here seems getting better and better. All army members

> > are blocked outside Beijing city. The people’s life in the city looks


> > as normal as usual. You may not able to see any difference than

> > ordinary life on the streets or in the shops now. Although the

> > students direct the traffic instead of the police, the accidents are

> > less than before. The buses started to work yesterday. Many people

> > went to their work unit this morning.


> >

> > There are still thousands of students in Tan’anman square. They said

> > “we will not end until our aims are reached”. The student area is

> > circled and controled by the students. There are alot of people

> > demonstrate to support them outside the area and on the Chang An street


> > which is in front of Tan’anmen.

> >

> > Beside Tan’anmen, the crowded areas are the places where the armies

> > are. The PLA rounds the city but the people round them. Hundreds and

> > thousands of people and students block at all the gateways. They


> > circle the soldier cars, the gas cars and the armoured cars. They tell

> > the soldiers the truth, they give them news paper, water and food.

> > Some soldiers droped their tears. They said that they did not know

> > what is happenning in Beijing and what to do here. A group of BeiDa’s


> > students and teachers went to “convey greeting to people’s son and

> > brother army” yesterday.

> >

> > So right now, the life in Beijing is very peaceful, there are no any


> > reason for the army to entry the city. The soldiers themselves don’t

> > want to get in to face to the students and the people there. But just

> > in case, a lot of people go to the streets in the evening and wait

> > there all night - they are ready to block the army’s cars using their


> > bodies, in the meantime, they are talking about the jokes of Li Peng,

> > shouting him abuses in the street.

> >

> > The martial law while was signed by Li Pang totally failed, nobody even

> > pay any attention to it. The demonstrations are still going on. The


> > government hasn’t done, even said anything to this after the martial

> > law was declared. The government already lose the control. I think

> > China is in a turning point and they have to fill the requests of the

> > people. I believe that the students and the people will win the


> > struggle.

> >

> > It is very very quiet this morning, it is said that there will be a big

> > demonstration this afternoon.

> >

> > I went to Tan’anmen very often these days. I have spent almost a night


> > with the hunger strikers there last week. I wish we had a “Sun

> > Microsystems supporting group”. Don’t worry please, I am no problem

> > here. We got a command from HK yesterday, it asked all foreign staffs

> > go to HK. [XXX] said it is not necessary. I think so too. The status


> > here is not so bad, “it is the best status during these 40 years”, [XXX]

> > said. He is going to stay here. In fact, he is one of people who

> > blocked the army’s cars in the nights. [XXX], do you wanna go with me

> > to see what type of guns the soldiers have if you are here?


> >

> > I can understand that how worry you were when you heard about the

> > martial law in Beijing. I hope I can tell you how strong the people

> > are and how great the students are. I am proud of them.

> >

> >


> > Xie Xie Ni, Wo De Peng You,

> >

> > - -[ XXX ]

> >

> >


Although this message was published in a discussion list at the time, I have not found it archived anywhere on the Web. I hope that at a future One Web Day, I will be able to publically show the full message with all of its headers and all identities, without fear of putting its originator in trouble.



Back then, I learnt how the Internet could be a medium for free speech, how it could bring freedom of speech to the world, how it gave a voice to the people in the street.


China did not have Internet in 1989 (it was first implemented in 1993), but UUCP email dial-up was already in place and email was somehow uncensored.


Lesson 2: the Internet is a warrant for your freedom.

Make sure it stays that way.


3. Creation of the .PS domain


A story about Internet Politics.


On 18 August 1996, having been the maintainer of the “Frequently Asked Questions (FAQ)” document on International E-mail Accessibility for several years, I tracked Internet Connectivity worldwide, a bit like those people tracking twisters in the US Midwest. My list was referenced using the ISO3166 International Country Codes, some of which were used as Country Code Top Level Domain.



I received a message from someone close to the Palestinian Authority, asking for my input to create the .PS domain for the newly created Palestinian Territories. It had just been a short span of time after the signing of agreements between Israel & the Palestinian Authority. Palestinians wanted to establish a symbolic presence in cyberspace.


I pointed them to ISO (International Organisation for Standardization, a United Nations funded Organisation), then liaised with the Internet Assigned Numbers Authority (IANA - namely Jon Postel) to create the domain etc.


The problem was that ISO had noted .PS as “reserved” and it therefore had not yet been officially created as a confirmed country code. But IANA needed issuance of a Country Codes that had been confirmed and published.


I pointed out that .AC had also been on the “reserved” but not published list and it had already been given a ccTLD. Call me a smart alec…


My pro-bono help landed me in the middle of a “flame” war between militant jewish & palestinian groups and I received hate email. Correspondents obviously thought I had more to do with the whole procedure than I really had. Events which I saw in the TV news had now entered my mailbox. Oh boy…



The whole process of creating a new Country Code Top Level Domain took time and I felt the heat after making this suggestion, was accused of taking sides, was even asked to “revert my decision” (uh? what decision?). I wondered if this was the true life of a politician…


Tatreez Embroidery from the Palestine ShopUltimately, .PS was created on 22 March 2000 (and I had absolutely no authority in the matter whatsoever).


I learnt that the Internet had become political. What I had considered a communication resource was now an instrument of geopolitical power. Little did I ever imagine how even more political it would become in the future!


Lesson 3: the Internet is a powerful political force. It is feared but also used by governments and militants. It is used by various stakeholders with strong political agendas.

For better? For worse? Only time will tell.


4. 911


A story about communication.



WTC image from wikipedia

WTC image from wikipedia


On 11 Sept 2001, I lived in Manhattan. 7 minutes after the first aircraft struck the twin towers on that horrific day, my father called me from France. I lived midtown and was therefore thankfully safe and out of danger.

This is the last call that I received for nearly a week.


As the news of the emergency spread worldwide, telephone coverage was overloaded. The Mobile Phone network is the first medium that went dead, requisitioned by emergency services. “No network”, it said. Then it was the turn of landlines. You would pick up the handset and had a busy signal. A while later, I could dial all Manhattan numbers only. As the Island was closed to the outside world, we lived times of total isolation, whilst we had so much emotional outpour to tell.


But the local dial-up number for my ISP still worked. Although very slow (due to overloading), the Internet was my only means of telling all my friends worldwide about what was going on, and for the first three days after the disaster, I wrote a daily summary of what I had done/seen/lived. I wrote it while breathing the stench of burning plastic and grilled flesh that blew our way when the winds blew in our direction.


I learnt how resilient the Internet could be in times of disaster, be it 911, an earthquake, a tsunami. I learnt how important it was to have a connection to the outside world because believe it or not, being isolated in Manhattan, we felt alone. You have to understand how empty Manhattan was below 34th Street to understand what I mean. I had read about this kind of feeling when, a few years earlier, I received daily email updates from a friend of mine living in Anguilla in the aftermath of Hurricane Hugo.


This time, I was on the other end of the line.


To me, the Internet was a lifeline to my family and friends around the world. To emergency services, it was a means of communicating with a lot of people when the infrastructure was overloaded.



Lesson 4: with times of turmoil appearing on the horizon, we as a people need to be more and more in touch with each other and with the rest of the world. In order to survive, our civilization needs the Internet.

No Internet, no future. That’s all.


My experiences are no different than those of thousands and thousands of other Internet users. But I’ve come to realise that the Internet (and by that, I obviously also mean the Web) was shaped by the millions of experiences lived by its users.


This is what I would like to celebrate on One Web Day.


Olivier MJ Crepin-Leblond, PhD.

http://www.gih.com/ocl.html