Keep Saving Those Elephants

This past week we received our copy of the 2011 Annual Report from Save The Elephants. As you may recall we made a donation last year during one of Godaddy's seemingly myriad public gaffes involving an elephant snuff vid made by then-CEO Bob Parsons.

Since then, the internet marches on, some other outrage, some other media fracas, possibly involving Godaddy again but for the most part, we all tend to react, maybe do something in the moment, and then we move on.

Well I'm glad STE sends their annual report out to donors, because I've just finished reading it and I find myself in awe of the work STE is doing in pursuit of their mission. They have truly embraced an all encompassing approach, from collaring and monitoring elephant families, to building schools and helping local communities struggle out of poverty so that they have viable economic alternatives to poaching. Read more »

[RESOLVED] Web control panel unreachable

At about 10:35am the web interface for the new platform became unreachable due to the problem with the Xen server holding the VM instance. Remote hands are looking into it while the rest of the systems group are working on why the warm-spares are currently pretty frigid.

DNS services, URL and email forwarding, easyMail are NOT affected.

Now loading an image into a new Xen container now and should be back forthwith.

We're sorry for the inconvenience.

Update: The interface is back online. The culprit was a blown switch which is currently being replaced. We have loaded a new instance of the UI and are investigating this incident.

Starting Wednesday, All New Domains Will Be Added via the New Platform

Just a quick heads up that starting Wednesday, if you are still using the legacy platform to manage your domains, any new domains you add to your account will be added via the new platform.

This will be a pretty frictionless process. If you are logged into the old platform, your password will work on the new oneĀ  and you simply continue the signup process on the new platform.

Other added benefits:

If you are after URL Forwarding or Full Email Forwarding then you no longer need DNS Plus (or DNS Pro) to get it, on the new platform these features have been moved down into the regular DNS Hosting package.

"The Voice of The Internet Community Has Been Heard"

A couple of encouraging developments out of the US reported as today via TheDomains blog make us cautiously optimistic that the Stop Online Privacy Act (a.k.a "SOPA") bill (to which we posted our objections in December) may be on ice for now. We were not alone in opposing SOPA and related initiatves, numerous domain registrars and DNS hosts also came out against it (except for one notable exception). Read more »

Updates on the easyroute53 service

In the wake of the DD0S and our own comments about it, there's a been a lot of interest in the additional functionality of the easyroute53 interface that allows you to control an existing route53 DNS service automatically through your easyDNS control panel. It's a neat tool, and we're glad folks like it. :)

We've updated the tutorial on our wiki at http://helpwiki.easydns.com/index.php/Using_our_easyRoute53_service as well as adding some more information on the interface itself.

A couple of points have come up, so we wanted to make it clear here on the blog as well :

1) our easyroute53 tool is just that : a tool. We are not partnered with Amazon in any way, nor does being a client with easyDNS automatically grant you an account with them. You'll need to set up an account with them through their system before configuring and using our tool. They provide the secret key, user info and so forth for their system.

2) simply enabling the automatic export/update function does not perform an initial export from our system to route53. You must perform a manual synchronization first to push an initial zone out to the Amazon servers. After that, changes you make to your zone through our system will automatically push out.

We've made that clearer in documentation now, and are sorry for any confusion that might have arisen.

Regards

easyDNS Support

 

 

 

 

Post-Mortem of the Jan-07 DDoS Attack

On Saturday, Jan 07 commencing at approximately 3:30pm EST (almost the exact moment I hit "publish" on a post about Save The Elephants, which we'll repost later) we were hit with a multi-faceted DDoS Attack across three anycast constellations: dns1, dns2 and dns3.

The attack was a combination SYN, ICMP and DNS Flood, in excess of 1 Gig/sec across our anycast IPs with packets per second ranging from 500K/sec to 1M/sec across each nameserver.

At the outset of the attack it looks like all three affected DNS constellations were rendered non-responsive for a period of 30 to 60 minutes.

We were able to identify the target of the attack and had them delegate away from our nameservers (this domain has now cycled through 8 other DNS providers in under 48 hours, bringing this DDoS with it to every one of them).

Read more »

[UPDATE]: Web Forwarding Slowness

Greetings all,

Apologies for the delayed update regarding the webforwarding slowness.
We have put some measures into place to account for the webforwarding slowness we have been experiencing. We will continue to monitor the situation over the next couple of days to see what else we can tweak to ensure the service is both speedy and available.

We'll post more updates as we have them.

Thanks very much for your patience and continued support of easyDNS. As always, don't hesitate to contact support should you have any questions.

 

[ original post below ]

Greetings,

 

We are experiencing some intermittent slowness issues with our webforwarding services. Our operations team are working to resolve the issue and we will post more info as it becomes available.

As always your patience is appreciated. Feel free to contact support should you have any issues.

Mini-FAQ about the Jan 07 DDoS Attack

Poor Les, is on Sunday support today and knows he's facing a lot of email and calls. On his way in, asked me for some guidance on three specific questions he is expecting to face a lot today.

Those questions are:

  1. How did something like this happen, and why did it affect me/my domains/my client's domains so hard? I thought you had fixed all this so it couldn't happen.
  2. This happens way too much with you guys (several references to the second time this year already);
  3. Can you reassure/guarantee that this won't happen again to me. What can I do make sure that it doesn't affect me the way it just did. Is there something I can do with my settings to better ride out something like this? What's my best practice?

And our responses are as follows:

Read more »

DoS Attack persists.

 

Also See:

Mini-FAQ about the Jan 07 DDoS Attack

 

[UPDATE: 12:54AM Jan 08] The attack traffic is still coming in fairly heavily. We are working on a couple of avenues of adjusting our defenses.

 

[UPDATE: 2:33AM EST Jan 08] DNS1 is back online. dns2 has been mostly online througout most of this. We are now working on dns3. ]

[UPDATE: 3:17AM Jan 08] We have rerouted dns3.easydns.CA and dns3.easydns.ORG to dns4.easydns.info for now. We will be bringing the main DNS3 anycasts back up Sunday during the day.

 


 

We think the worst is over for today's DOS attack which hit us on dns1.easydns.com, dns2.easydns.net and dns3.easydns.org (and dns3.easydns.ca) anycast constellations.

The attack was a multi-faceted multi-gig combination of SYN, ICMP and DNS Flood.

DNS1 and DNS3 totally imploded. DNS1 is coming back in pieces, DNS3 is still down hard.

DNS2 went down when the attack first hit, but Prolexic was able to bring enough of it back up after 30 minutes or so to restore partial service.

We are working on bringing the rest of DNS1 up, and a workaround to route DNS3 traffic elsewhere until the attack traffic abates.

On that note, the target of the attack has been identified and has removed its nameserver delegation from us. Until about an hour ago there were still nameservers reporting our nameservers as the delegation for the target domain. Now that those are gone, we expect the attack traffic to drop.

I also by accident pulled our previous post on this subject back into draft mode, making it invisible on the blog, because I meant to revoke my (now, seemingly idiotic "Save the Elephants" post), which I hit publish on almost the exact moment the attack started. Because it's been that kind of a day.

This isn't the post-mortem. I will post that later. Just wanted to update everybody with where we're at.

There will be serious, structural changes here as a result of today. The worst DOS attack impact we've suffered since 2005.

DOS Attack In Progress

We are currently experiencing an Denial of Service Attack against DNS1, DNS2 and DNS3 anycast strands.

We are working on mitigation and will post updates as they become available.

UPDATE 7:55 PM EST:

Our mitigation techniques have brought the dns2.easydns.net anycast cluster back online, which should resolve slow response or timeout issues for end users. We are still experiencing attack traffic, but are working hard to minimize impact. We've also managed to identify the target of the attack, and are working on resolving the issue in that direction as well.

Further updates will be posted as they come available.

 

Also See:

Sign up for Domain Insights From The easyDNS Guy

Get "The easyDNS Guy's" 3-part Domain and DNS Boot-camp delivered free to your inbox, plus topical musings on the state of the domain landscape.
* = required field
Interests