Jan has been working hard on lighttpd. With version 1.4.12 he added a new module called mod_magnet.

At first glance, mod_magnet might be mistaken for a newer version of mod_cml – and it may seem a better version. So what makes mod_magnet so cool?

  • Firstly, you can modify most request parameters
  • You can avoid creating our own DSL within mod_rewrite to implement -d/-f or even more complex conditionals?
  • lua is really fast! See this example from jan and his benchmarks.

What is mod_magnet not?

Be Warned:This plugins is not meant for something like “lua server pages”. This is because The magnet script is executed within the lighty core, therefore EVERY long-running operation is blocking ALL connections in the server.

For time-consuming or blocking scripts use mod_fastcgi and friends.

the problem

The scenario I use mod_magnet is to provide simple (clean) URLs. With mod_rewrite, you can achieve them rather trivially with:

  # taken from the drupal .htaccess file
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

So far, lighttpd has lacked this feature….

.... unlike apache mod_rewrite. To quote http://httpd.apache.org/docs/2.2/rewrite/:

`` Despite the tons of examples and docs, mod_rewrite is voodoo. Damned cool voodoo, but still voodoo.’‘

Brian Moore

Do we want another voodoo language?

Back to our scenario – with Dr Magneto to the rescue!

Anyone spent any time dealing with rails will be used to:

2006-06-18 21:29:04: (connections.c.1407) Warning: Either the error-handler returned status 404 or the error-handler itself was not found: /dispatch.fcgi
2006-06-18 21:29:04: (connections.c.1409) returning the original status 404
2006-06-18 21:29:04: (connections.c.1411) If this is a rails app: check your production.log

The infamous 404 handler trick rears its ugly head again. So I started with the example shown in the mod_magnet docs. However, like many other users I had problems with luafilesystems. After multiple iterations of the script and a few more changes to mod_magnet, we have a nice and slick version of cleanurl.lua for rails. =)

-- the magic ;)
attr = lighty.stat(lighty.env["physical.path"])
if (not attr) then
  -- file does not exist. check if we have a cached version
  lighty.env["physical.path"] = lighty.env["physical.path"] .. ".html" 
  attr = lighty.stat(lighty.env["physical.path"])

  if (not attr or (attr and not attr["is_file"])) then
    -- file still missing. pass it to the fastcgi backend
    lighty.env["uri.path"] = "/dispatch.fcgi" 
    lighty.env["physical.rel-path"] = lighty.env["uri.path"]
    lighty.env["physical.path"] = lighty.env["physical.doc-root"] .. lighty.env["physical.rel-path"]
-- fallthrough will put it back into the lighty request loop
-- that means we get the 304 handling for free. ;)

-- debugging code
-- print ("final file is " ..  lighty.env["physical.path"])


To use this script, you need a lighttpd build from the 1.4.x branch (in svn). 1 SuSE users can just grab the lighttpd14-snapshot rpms from the server:http build service project. 2

Jan added lighty.stat() for me. My first 2 versions (v1, v2) cost us 2 uncached stat() calls. The same applies to the complex rewrites example using luafilesystems. lighty.stat() uses the stat-cache of lighttpd. Ideally, the stat() is hit once and than cached. Version two of my script has the advantage of working on plain 1.4.12 already.

The return value of lighty.stat() cache allows to distinguish files from directories. My next project will be a customizable mod_dirlisting replacement in lua – with caching of rendered directory indeces and all those funny icon stuff people want.

As many people complained about the length of the cleanurl.lua script, i add the minimalistic version:

attr = lighty.stat(lighty.env["physical.path"])
if (not attr) then
   lighty.env["uri.path"] = "/dispatch.fcgi" 
   lighty.env["physical.rel-path"] = lighty.env["uri.path"]
   lighty.env["physical.path"] = lighty.env["physical.doc-root"] .. lighty.env["physical.rel-path"]


So how much slower does this make lighttpd? The short answer is not much. I benchmarked my lua script and got about 3200r/s. Without the the mod_magnet script i can get my server to 4200r/s. Does this cost performance? Yes. A whole bunch- but if your server is already pushing over a couple thousand requests a second, you’re definitely going to have other issues already. Even the powered by lighttpd list in the wiki lists only 100r/s as top.

mod_magnet can do that without sweating: R.I.P. Mr. 404 handler

To build from SVN

you need at least a checkout of rev 1378 or greater, and lua 5.1.

$ svn co svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x/
$ ./autogen.sh
$ ./configure <your params here>
$ make
$ make install

SuSE notes

The user documentation can be found at the opensuse wiki. Additionally you need the lua51-libs rpm from devel:languages:lua.

Thanks to imajes for making this article a pleasure to read for native english speakers