As many of you know, this site was hacked last week, along with several other ColdFusion powered sites including Mike Dinowitz’s House Of Fusion. The attackers found a vulnerability in Galleon which allowed for a file to be uploaded and executed. The actual uploaded file appears to be similar to (or the same as) the ones used in an FCK upload attack a while back, the attackers just found a new way to get it on to the server.
The hole was quickly plugged as soon as I was made aware of the attack. But, by then the damage was done. I started to do a manual repair (based on very detailed steps posted by Mike Dinowitz), but just did not have the time to commit to it. So, instead, I opted to do a full system restore to the day prior to the attack and then plug the holes before bringing the server online. The problem is that between being buried in MAX planning, flying back and forth between MI and CA, and then being offline for 2 days for Rosh Hashanah, I just ran out of time, and reluctantly opted to leave the server offline for a few days. Fortunately, that did not happen, and I was back online far quicker than that. I still need to get Google to unlist me as hosting malware, but that’s a work in progress.
Now that I am back, I do need to make a few comments …
First, lots of you e-mailed me to tell me that you were seeing malware warnings, and I really appreciate the heads-up from each and every one of you. The e-mails brought the issue to my attention quicker than I’d have discovered it myself.
Secondly, thanks to Mike Dinowitz for investing the time into figuring out the details of this attack, and then sharing so that others could benefit from his efforts.
Thirdly, thanks to Ray Camden for proactively reaching out with the hole details and for quickly and aggressively patching things up. I know Ray feels terrible about all of this, and I want to be very clear that I don’t blame him at all. (I do, however, reserve the right to use this in the future as leverage when I need him to work on a project!).
And finally, I need to thank HostMySite. I’ve been hosting my sites with these guys since August 2004, and have always been satisfied with the service I receive. I’ve had no emergencies, and no major issues, everything just worked. But this time I needed their help, and they came through. When I called from an airport asking them to take all of my HTTP servers offline they did so within minutes. And when I asked them to do a full system restore for me, they initiated the process immediately. Unfortunately, they ran into a technical problem and had to restart the restore, and by the time it was done I was already offline for 2 days. But, HostMySite stepped up, brought the sites online, implemented all of my needed changes, reset a list of security settings that I had requested, tested things thoroughly … and all without my involvement. This was more than they were required to do, and beyond what I had asked. Bottom line, the first time I really needed their support, they stepped up and took care of things. Thanks Michael, Randy, Duncan, and Lawrence, your assistance is greatly appreciated!
And now, it’s back to our regularly scheduled programming.

8 thoughts

  1. Ben, I did a write up of this particular exploit on my blog (http://bit.ly/51h4Z) and I noticed Ray mentioned it today. Having battled this issue on 5 servers now for folks in need we have noticed that there are actually several different exploits – including this one – that are being used as initial attack vectors. It seems like the hacker will "keep at it" until he or she finds a way to get an arbitrary file on your system. The load test scenario is one, but we have also found FTP, WebDav, FCK Editor (widely publicised by now) and a vulnerable ASP script. I only mention it because it is important for folks to know that just plugging the galleon hole will not go far enough to protect their server. They need to take a look at all aspects of file management on the server not just the application code.
    -Mark

  2. James, yes, that is what I was referring to when I said I am waiting for Google to delist my site as being compromised. The site is fine now, but Google takes time to update those lists. 🙁
    — Ben

  3. I had to switch to Google Chrome in order to access your website as it was either too frustrating or not even loading in Firefox.
    We recently had a website listed because of a redirection script. The script was for tracking purposes and took a single URL parameter containing the URL of the website to redirect to. The attacker exposed the integrity of our domain name by using it to redirect requests to bad websites, embedded javascript, etc. So if you have any redirection scripts that rely on the forwarding link to be passed URL, search Google to make sure that no one else is taking advantage of it and potentially blacklisting you. (I’m not sure if this is considered a true compromise or exploit, but it was enough for Google to blacklist us.)

  4. I don’t know anything about servers security, but think on a server as a private house where you use to receive thousands of guests a day. That house has
    some kind of alarm connected to police station. If, for any reason, somebody tries to perform a not expected action inside the house then you know
    what’s next. The solution would be registering every single system (not user) action in order to know werther that action is allowed or not.
    As I said at the very beginning of this post, I don’t know anything about server security.
    Regards.
    Emilio

Leave a Reply