GIH servers suffer sustained DoS attack

After spending a couple of days off-line, I found out on Sunday 30th August that our servers in London had been hit by a denial of service (DoS) attack through mail-bombing from different sources.

This happens every now and then. The last time, it took place when I was at the ICANN conference in Cairo, and another European participant's systems had been hit at the same time. Thankfully the GIH servers in London survived then, and thankfully, they survived (barely) this time.
The strength of the attack this time round was higher than before since every time there is an attack, I find out what the weak point is in our network and upgrade it. I'm glad that the servers actually manage to recover each time, but it is still very disruptive indeed.

So this time round, the main mail hub got attacked with 360 emails per second from zombie computers all around the world, for a very sustained amount of time, starting at 8:00pm on saturday night. As a result, the server ran out of memory+swap (500Mb + 500Mb) and froze by going into some kind of panic recovery mode, thus sending the attack to our backup route, and this then sent the emails to our back-end machine via another path, in a more controlled manner (aka - via UUCP over IPv6, the new with the old working very well together). As a result, the back-end stored 28 000 emails, all but 200 being spam sent to wrong addresses.

I managed to reboot the main mail-server remotely in a short window of time that it allowed me to log in. It took 30 minutes to shut down, so clogged were its processes, running at a load of 58. I spent sunday afternoon trying to find out how to process the backlog of emails, bearing in mind, more was coming in. It felt like dealing with a flood.

Finally, I commissioned a third computer running Linux, which I had kept running for the past 4 months as standby and which has now taken the load off the front and back end machines and is shifting through the now 24 000 remaining emails. All in all, the GIH computer systems in London have filtered 98 000+ spams in 24h. That is a lot more than at any time before. If this is a taste of things to come in the near future, we're going to have to beef those servers more with a lot more processing power.
When I remember that the first computer system for GIH.COM was a 20Mhz 80386 running linux & 4 Mb memory, downloading emails via UUCP and sending them via SLIP/PPP through a half hourly telephone call using a 9 600 baud modem, and it was running smoothly, even when downloading a few USENET newsgroups that I enjoyed reading...

...I just wonder where this spam is leading us to.

...sigh...

Green Networking - Google's Data Centres

Google's released an interesting set of documents and videos out detailing their set-up in their brand new Green Data Centres.

You can check it out on:

http://www.google.com/corporate/green/datacenters/summit.html

Whilst other industries have been under intense pressure to reduce their Carbon emissions, the ICT Industry has, so far, not done that much. Of course, the energy rating of computers and telecom equipment has been in the spotlight and most hardware manufacturers have displayed Energy Star ratings, but our ever increasing thirst for information has asked for faster and faster data centres. And that means more servers, more speed, more energy consumption and more heat to get rid of.

As other industries have complied with a reduction of carbon emissions, so will the ICT industry. Google's current initiative is pioneering. Others will no doubt follow. The physical topology of the Internet network might be changed radically, as data centers located in the middle of towns make way for out of town more secure mega-facilities built close to cheap and clean energy, and expandable at will.

NGN - clean slate or not ?

Attending meetings about Internet Governance, I keep on hearing how so many things will influence the architecture of the Internet. A lot of the heated discussions tackle the short term instead of the long term. Furthermore, some governments are under the impression that a "Next Generation Network", aka NGN, will be a Clean Slate approach to networking.

For the record, let me say here that these are pipe dreams.

Apart from increasing control of the next generation network by governments and multi-national corporations, I cannot see a single reason why a next generation network should be created out of a Clean Slate approach. The Internet might not be perfect, but its imperfections are being worked on. At the very least they are known!

Currently, there are thousands of engineers around the world who are working on ways to remedy the Internet's Achille's heels. The experience which has been acquired thus far by running an international network spanning the whole world, is more valuable than any theoretical study that will ever be conducted. The amount of "known knowns" and "known unknowns" is vastly superior to the amount of "known knowns" and "known unknowns" in a Clean Slate approach. In fact, I'll venture out to say that the danger, as we all know, is the amount of "unknown unknowns", which the Clean Slate approach is full of.

So what do we really know about the Clean Slate approach? Nothing. In fact, at the time of writing, I suspect that the people proposing the Clean Slate approach don't know either.

How will it perform? How reliable will it be? Will it be hacked? Will it be secure? Will it encourage innovation? Will it encourage democracy? Will it be embraced internationally?

Perhaps should we find answers to these questions before launching into huge development costs for a programme which might yield little more than few gigabytes of presentations and writings.

TOOL: Google Wave

Taking part in working groups both at ICANN, ISOC, but also IETF, I spend a fair amount of my time in conference calls.

Each organisation has its own preferred tool for online collaboration. Whilst some of them involve contracts with a commercial provider, others are "free" - as in they might be test systems or freeware. If all else fails, we sometimes use Skype. But finding a proper tool that can let you share documents and presentation material, speak to each other in a meaningful way, interface with the real world, as well as allow for simultaneous text input is easier said than done. The bottom line is that we've *always* has a problem with communications.

The VMEET working group at IETF has been formed specifically to find a solution to this problem, whether it is evaluating what's out there, or drawing up specifications for a new set of tools. Its findings risk being very helpful for other organisations, since an increasing amount of collaborative work is required if the Internet model of governance is to be sustained. For more information on this exciting challenge, go to: https://www.ietf.org/mailman/listinfo/vmeet

That said, Google seems to have come up with its own version of online collaboration. Not quite ready yet, but there's a preview on:

http://wave.google.com

It looks like it has potential, especially with extensions, since it follows the Open Source concept.