Mongrel Cluster on Mac OS X

Mongrel is a HTTP server that is well suited for serving Rails web applications, often in conjunction with Apache as a 'front end' server.

Mongrel works fine as is for development applications but if you have any real number of users its performance will degrade rapidly. In this case you want to use a cluster of mongrel servers, each running on a different port and have Apache balance the load between all of them

The mongrel_cluster gem is a convenient way to set up and manage multiple mongrels. HOWEVER I do not recommend this on Mac OS X. Because of the way Mac OS X handles processes at startup and/or because I couldn't figure it out, I was not able to get mongrel_cluster to work correctly at startup on my system. Some of the guides on the web that claimed to do this did not work for me. But don't worry... mongrel_cluster is really a simple utility and you can do without it at the price of a little more configuration. That is what I will show here.

These instructions relate to Mac OS X 10.4 and they assume that you already have Apache, Mongrel and your Rails application up and running.

1. Make sure that your Rails app is using Active Record to store Session data in the database, as opposed to storing it in files, which is the default. Instructions for that can be found HERE.

2. Setup a launchd plist file for each mongrel instance that you want to set up.
The preferred way to start programs automatically under Mac OS X is through launchd instead of the traditional init process in other Unix variants. launchd takes a bit of getting used to but you should use it (and don't try and mimic init scripts using StartupItems...)

You can learn about launchd is this Apple developer note and by doing a man launchd and man launchd.plist

3. In this example I am going to setup up 4 instances of mongrel on ports 8000, 8001, 8002 and 8003

In /Library/LaunchDaemons create the file net.mongrel80000.plist with contents similar to this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/Prop
ertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>net.mongrel8000</string>
  <key>ProgramArguments</key>
  <array>
          <string>/usr/local/bin/mongrel_rails</string>
          <string>start</string>
          <string>-e</string>
          <string>development</string>
          <string>-p</string>
          <string>8000</string>
          <string>-c</string>
          <string>/Users/jones/myapp</string>
          <string>-P</string>
          <string>/Users/jones/myapp/tmp/pids/mongrel.8000.pid</string>
          <string>-l</string>
          <string>/Users/jones/myapp/log/mongrel.8000.log</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>ServiceDescription</key>
  <string>Mongrel Rails Application Server</string>
</dict>
</plist>

This ugly block of XML breaks down into the following components:
Label
This is a UNIQUE label for this launch item. In our case make this the same as the name of the file, less the '.plist' extension (net.mongrel8000)
ProgramArguments
An array of strings which, when joined together, create the command that you would run to start this instance of mongrel. Change the paths, etc to suit your application. Be VERY careful to set the port numbers to the one used in this file (8000 in this case)
RunAtLoad
This is set to true and tells launchd to run this item once when the system starts up.
ServiceDescription
An optional string that describes what this item represents.

chown/chmod this to have these permissions and ownership:

-rw-r--r--   1 root  wheel  836 Feb 19 14:53 net.mongrel8000.plist

4. Clone and modify this file for each mongrel instance
In my case I copied this into net.mongrel8001.plist, etc. and changed each instance of the port number 8000 to 8001, 8002, or 8003 as appropriate. These are marked in red in the above XML. Make absolutely sure the Label is correct and unique otherwise it won't work.

5. Test it out
Clean out any editor backup files in /Library/LaunchDaemons, check your file permissions and restart your machine. If things are setup correctly, when it restarts it will have started four instances of Mongrel that will drive your application.

Test these by using a browser on that machine and going to each port in turn, in other words these four URLs should all work:

http://localhost:8000
http://localhost:8001
http://localhost:8002
http://localhost:8003

Note that in one of the guides on the web that I saw they used a single .plist file and put the command strings for all the mongrel instances in a single ProgramArguments section. This did not work for me at all...

If my instructions don't work for you then double check the format of the XML file and your paths.

6. Configure Apache to direct requests to the Mongrel servers
Your Apache installation needs to have the mod_proxy_balancer module installed. 'httpd -l' will list the compiled in modules and hopefully you will find it there. You can find out how to compile in modules in Apache 2.2 and read my grumbles about that HERE.

Edit your Apache httpd.conf file to create a virtual host that it will respond to and a set of proxy and load balancing instructions. You can do this in various ways and get a lot more complex than this, but here is a single block that you can put at the bottom of httpd.conf.

<VirtualHost *:80>
ServerName myserver.craic.com

# Enable URL rewriting
RewriteEngine On

# Rewrite index to check for static pages
RewriteRule ^/$ /index.html [QSA]

# Rewrite to check for Rails cached page
RewriteRule ^([^.]+)$ $1.html [QSA]

# Redirect all non-static requests to cluster
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L]

# You could also add a bunch of deflate rules etc here

# This is the path to static content as opposed to your Rails app
DocumentRoot  "/Users/jones/html"
<Directory "/Users/jones/html">
     Options Indexes FollowSymLinks

     AllowOverride None
     Order allow,deny
     Allow from all
 </Directory>

</VirtualHost>

# This block tells the load balancer to pass requests
# onto these four mongrels

<Proxy balancer://mongrel_cluster>
  BalancerMember http://127.0.0.1:8000
  BalancerMember http://127.0.0.1:8001
  BalancerMember http://127.0.0.1:8002
  BalancerMember http://127.0.0.1:8003
</Proxy>

All the Rewrite lines tell Apache how to handle requests for static versus dynamic (from Rails) content. The important lines are the two that rewrite requests for non-static content into ones with a prefix of balancer://mongrel_cluster. These are passed to the <Proxy> block at the end where the load balancer distributes these among the members listed here. These members are the four mongrel instances that we set up earlier. Apache doesn't care that these are mongrel servers. It just sees them as URLs and efficiently passes on the requests.

Your Apache is most likely started by a launchd plist file. Assuming it starts automatically then any reboot of your server will start it and the mongrel instances.

Giving a URL like http://myserver.craic.com/myapp will now get forwarded to one of the mongrels.


There you have it... If you read this and have a better mongrel launchd configuration please let me know. There has to be a cleaner way than what I have here...

Using Active Record for Session Storage in Rails

This setup appears in all sorts of pages but I'm including the basic steps here for my own benefit - and hopefully yours...

The default way that Rails (1.2.3) stores session data is in files in your application tmp directory. This mechanism is referred to as CGI::Session::PStore. This is fine for development but becomes a problem as you move to a real production environment.

One problem with it is that unless you actively clean out old files with a cron job and 'rm' you can end up with a massive number of old session files.

It is also a problem if you use Apache and Mongrel to serve your application and want to scale things up with mongrel_cluster. Various resources warn of bad things happening with multiple mongrels and session files.

The next step up from files is to use a database table and have Active Record store session data in that. This is easy to setup.

1: Create a migration to set up the table and run that

$ rake db:sessions:create
    exists  db/migrate
    create  db/migrate/027_add_sessions.rb
$ rake db:migrate
== AddSessions: migrating =====================================================
-- create_table(:sessions)
 -> 0.4298s
-- add_index(:sessions, :session_id)
 -> 0.2914s
-- add_index(:sessions, :updated_at)
 -> 0.0727s
== AddSessions: migrated (0.8001s) ============================================

2: Edit your app's config/environment.rb file and uncomment this line
config.action_controller.session_store = :active_record_store

3: Start your app server, interact with it and look in mysql
mysql> select * from sessions;

You will see a hexadecimal encoded session_id and a big block of encoded session data. Everything else should just work normally.

4: To avoid sessions accumulating in applications where you have users login and logout, you can call reset_session in the logout action - something like this:

  def logout
  reset_session
  flash[:notice] = "Logged out"
  redirect_to :action => "index"
end

This clears out your current session row in the table and initializes a new session object. I'm not sure why it does the latter.