Monday 31 August 2009

GIH servers suffer sustained DoS attack

After spending a couple of days off-line, I found out on Sunday 30th August that our servers in London had been hit by a denial of service (DoS) attack through mail-bombing from different sources.

This happens every now and then. The last time, it took place when I was at the ICANN conference in Cairo, and another European participant's systems had been hit at the same time. Thankfully the GIH servers in London survived then, and thankfully, they survived (barely) this time.
The strength of the attack this time round was higher than before since every time there is an attack, I find out what the weak point is in our network and upgrade it. I'm glad that the servers actually manage to recover each time, but it is still very disruptive indeed.
So this time round, the main mail hub got attacked with 360 emails per second from zombie computers all around the world, for a very sustained amount of time, starting at 8:00pm on saturday night. As a result, the server ran out of memory+swap (500Mb + 500Mb) and froze by going into some kind of panic recovery mode, thus sending the attack to our backup route, and this then sent the emails to our back-end machine via another path, in a more controlled manner (aka - via UUCP over IPv6, the new with the old working very well together). As a result, the back-end stored 28 000 emails, all but 200 being spam sent to wrong addresses.
I managed to reboot the main mail-server remotely in a short window of time that it allowed me to log in. It took 30 minutes to shut down, so clogged were its processes, running at a load of 58. I spent sunday afternoon trying to find out how to process the backlog of emails, bearing in mind, more was coming in. It felt like dealing with a flood.
Finally, I commissioned a third computer running Linux, which I had kept running for the past 4 months as standby and which has now taken the load off the front and back end machines and is shifting through the now 24 000 remaining emails. All in all, the GIH computer systems in London have filtered 98 000+ spams in 24h. That is a lot more than at any time before. If this is a taste of things to come in the near future, we're going to have to beef those servers more with a lot more processing power.
When I remember that the first computer system for GIH.COM was a 20Mhz 80386 running linux & 4 Mb memory, downloading emails via UUCP and sending them via SLIP/PPP through a half hourly telephone call using a 9 600 baud modem, and it was running smoothly, even when downloading a few USENET newsgroups that I enjoyed reading...

...I just wonder where this spam is leading us to.

...sigh...


Sunday 9 August 2009

Green Networking - Google's Data Centres

Google's released an interesting set of documents and videos out detailing their set-up in their brand new Green Data Centres.

You can check it out on:

http://www.google.com/corporate/green/datacenters/summit.html

Whilst other industries have been under intense pressure to reduce their Carbon emissions, the ICT Industry has, so far, not done that much. Of course, the energy rating of computers and telecom equipment has been in the spotlight and most hardware manufacturers have displayed Energy Star ratings, but our ever increasing thirst for information has asked for faster and faster data centres. And that means more servers, more speed, more energy consumption and more heat to get rid of.

As other industries have complied with a reduction of carbon emissions, so will the ICT industry. Google's current initiative is pioneering. Others will no doubt follow. The physical topology of the Internet network might be changed radically, as data centers located in the middle of towns make way for out of town more secure mega-facilities built close to cheap and clean energy, and expandable at will.

NGN - clean slate or not ?

Attending meetings about Internet Governance, I keep on hearing how so many things will influence the architecture of the Internet. A lot of the heated discussions tackle the short term instead of the long term. Furthermore, some governments are under the impression that a "Next Generation Network", aka NGN, will be a Clean Slate approach to networking.

For the record, let me say here that these are pipe dreams.

Apart from increasing control of the next generation network by governments and multi-national corporations, I cannot see a single reason why a next generation network should be created out of a Clean Slate approach. The Internet might not be perfect, but its imperfections are being worked on. At the very least they are known!

Currently, there are thousands of engineers around the world who are working on ways to remedy the Internet's Achille's heels. The experience which has been acquired thus far by running an international network spanning the whole world, is more valuable than any theoretical study that will ever be conducted. The amount of "known knowns" and "known unknowns" is vastly superior to the amount of "known knowns" and "known unknowns" in a Clean Slate approach. In fact, I'll venture out to say that the danger, as we all know, is the amount of "unknown unknowns", which the Clean Slate approach is full of.

So what do we really know about the Clean Slate approach? Nothing. In fact, at the time of writing, I suspect that the people proposing the Clean Slate approach don't know either.

How will it perform? How reliable will it be? Will it be hacked? Will it be secure? Will it encourage innovation? Will it encourage democracy? Will it be embraced internationally?

Perhaps should we find answers to these questions before launching into huge development costs for a programme which might yield little more than few gigabytes of presentations and writings.

TOOL: Google Wave

Taking part in working groups both at ICANN, ISOC, but also IETF, I spend a fair amount of my time in conference calls.

Each organisation has its own preferred tool for online collaboration. Whilst some of them involve contracts with a commercial provider, others are "free" - as in they might be test systems or freeware. If all else fails, we sometimes use Skype. But finding a proper tool that can let you share documents and presentation material, speak to each other in a meaningful way, interface with the real world, as well as allow for simultaneous text input is easier said than done. The bottom line is that we've *always* has a problem with communications.

The VMEET working group at IETF has been formed specifically to find a solution to this problem, whether it is evaluating what's out there, or drawing up specifications for a new set of tools. Its findings risk being very helpful for other organisations, since an increasing amount of collaborative work is required if the Internet model of governance is to be sustained. For more information on this exciting challenge, go to: https://www.ietf.org/mailman/listinfo/vmeet

That said, Google seems to have come up with its own version of online collaboration. Not quite ready yet, but there's a preview on:

http://wave.google.com

It looks like it has potential, especially with extensions, since it follows the Open Source concept.

50 great examples of data visualization

The following link is a particularly well researched/documented blog entry about data visualization.

http://www.webdesignerdepot.com/2009/06/50-great-examples-of-data-visualization/

It has links to many very interesting visualization engines and concepts. If, like me, this sort of this fascinates you, I recommend setting aside a few hours before you embark on looking at this page. :-)

Visualisation is, of course, a major part of our cognitive processes and we, as humans, will probably require increasingly complex visualization tools to enable us to make a more complex world easier for our limited minds to understand. But looking through many of these examples, it also struck me that a great deal of analysis often took place before visualization was even possible, and I wonder whether some of the examples are not merely enabling our mind to understand, but also open the door for machines to understand each other.

How? By searching for ways to format data in a form which can be read and displayed by a machine (after all, graphical display tools are run by computers), we are stumbling on the possibility of that formatted data to be used in other ways than just being displayed on a graph.

Another thing which struck me is the worth of data visualization in reminding us of the past. I had lunch earlier this week with a French philosopher who advised me that unfortunately, one common human trait is the ability to forget the past too easily. As a result, mistakes are repeated and only a fraction of knowledge is transmitted in the long term. Trend patterns are completely obliterated.
Take a piece of software like "Flare", for example, used by some of the data visualization examples above, but not directly referred to by the article I point to above. One example which I was particularly impressed about was their "Job Voyager":

http://flare.prefuse.org/apps/job_voyager

Can you see which jobs you should avoid because they are, literally, dead ends? :-) Yes, reminding us of the past can point us to the future.

I hope you enjoy the visualizations.