Soylent Red

I can succinctly describe Apache's mod_negotiate in the words of Homer Simpson: "It can't be turned off… Though it does break easily."

I've spent longer than I care to contemplate wrestling with a mod_rewrite directive that was acting up on the about and /projects/ sections of this site. I'll record the resolution here for posterity.

The directive was a simple one:

RewriteRule ^projects/(.+) /projects/?project=$1

This should take a URL such as /projects/make-link and map it onto /projects/?project=make-link. As mod_rewrite goes, this is a simple one. It gave me a 404 response every time.

After much poking and prodding I discovered that it would do the correct redirect for the url /projects/foo. (This took a lot of fiddling to discover, because the script that serves that part of the site will return a 404 response for projects that don't exist, making that indistinguishable from a failed redirect. I had to edit that script to discover that the redirect was working for /projects/foo but not for /projects/make-link.) This turned out to be the work of mod_speling (sic). It will "fix" a single typo in any url you request (a missing character, an extra character or a transposition), so it was rewriting "make-link" to "makelink", which happens to be the name of a directory in that section.

Having turned off mod_speling, I still had trouble. I narrowed the bug down to three simple conditions. Usually /projects/foo would redirect properly to /projects/?project=foo. If there was a file in the /projects/ directory called foo.html then /projects/foo would redirect to /projects/?project=foo.html. Finally, if there was a file called foo.html.inc in that directory (and there was), it would crap out and give a 404 response.

This was caused by the aforementioned mod_negotiate, which basically guesses what file to serve if it's asked for a URL with no file extension. The default is to serve whatever file it can that it recognises the extension for. Thus it will serve foo.html because it knows the .html extension, but it doesn't play well with foo.html.inc because .inc is just something I made up for my own convenience.

The problem is that this module can't be turned off! Ridiculous. The only thing I could do to stop it from breaking my site was to set it to be as promiscuous as it can be, to try to serve up any file with any extension that seems to match. This got me to the stage that /projects/foo would redirect to /projects/?project=foo.html.inc. A small change to the relevant scripts now strips the unwanted ".html.inc" from the parameter it's given.

Why is there no off switch on that module?

• Permanent Link
• Toggle Comments [2]

You probably won't have noticed much if anything different about Soylent Red in the last day or so—I think the new URLs are the most visible change—but I did make quite a significant code change yesterday evening. The diff weighed in at over 2000 lines in fact. All of the code worked on my local testing setup, but I had a problem as soon as I tried to post a new entry on the live site. Only the date, the tags and the comment status (open or closed) were saved.

After a bit of wrangling—I had to check in a new copy of the code every time I wanted to test something—I discovered that the problem lay in the use of the mysql_real_escape_string() function, which is used to protect against injection attacks when putting text in a database. It was news to me that this PHP function actually calls an SQL function to do the work for it, so it requires an existing database connection.

After my big code changes there was one part of my code where I happened to try to use the function to create my query before the code to establish a connection to the database server. In this event PHP will try to connect as if calling mysql_connect() with no arguments, and failing that it does the only thing it can do: it throws a warning while returning an empty string. I have my live site set not to display errors, so it looked to be silently failing. Meanwhile my local testing setup never came across the warning because it is always possible to establish a connection (to a server on localhost with no username or password).

I love these bugs that lead to new learning.

• Permanent Link
• Toggle Comments [2]