A collection of computer systems and programming tips that you may find useful.
 
Brought to you by Craic Computing LLC, a bioinformatics consulting company.

Tuesday, April 10, 2007

Setting up a Subversion Repository

Subversion (svn) is a leading software version control system that has a myriad of features. It is heavily documented, however the basic steps involved in setting it up and accessing it are somewhat obscured by descriptions of the many configuration choices.

Here is a concise description of setting a svn repository on a linux server (which I'll call remote) and accessing it from a client computer (local).

1: Creating a Repository

Download the software from http://subversion.tigris.org/ or, on a Fedora system :

[remote ~]#
yum install subversion.

Create a directory for the repository, with whatever name you choose

[remote ~]$ mkdir /proj/svn_repository

Create the svn repository in that directory

[remote ~]$ svnadmin create /proj/svn_repository

Create subdirectories for each project using svn (not regular mkdir!).
There are many ways to do this but a commmon approach is to have a svn directory for each project and three subdirectories called trunk, branches, tags - with trunk containing the primary version of your code.

[remote ~]$ svn mkdir file:///proj/svn_repository/project1 -m 'New directory'
[remote ~]$ svn mkdir file:///proj/svn_repository/project1/trunk -m 'New directory'
[remote ~]$ svn mkdir file:///proj/svn_repository/project1/branches -m 'New directory'
[remote ~]$ svn mkdir file:///proj/svn_repository/project1/tags -m 'New directory'

There is no need for a svn commit command. And, of course, you can add project directories at any time.

Import a test body of code from a directory on that server as a test of the installation

[remote ~]$ cd ~/mytestproj
[remote ~]$ svn import . file:///proj/svn_repository/project1/trunk

Check that the files are in the repository

[remote ~]$ svn list file:///proj/svn_repository/project1/trunk

2. Set up Remote Access to the Repository

There are several ways to do this. Assuming that your users have accounts on the server with the repository, then using svn+ssh is probably the easiest to setup and maintain.

First, install the svn client on the client(local) machine. Look for this on http://subversion.tigris.org/

Without making any changes to the server, connect to it using svn+ssh with the remote hostname and the full path to the repository directory.

[local ~]$ svn list svn+ssh://svn.craic.com/proj/svn_repository/project1/trunk

This will prompt you for your user account password on the server and then return the list of files in the repository. So this just works... but it will ask you for your password for every outgoing connection, which quickly becomes a royal pain.

To fix this you need to set up an RSA private/public key pair and copy the public key to your directory on the remote server. Key pairs may look a little cryptic but the basic is pretty simple.

(Note, that if you already have an RSA keypair for another purpose then you will need to give your new pair a different name - see below for the tweak needed to use this - for now I'll use the default name)

First, create the key pair on your client machine (Do NOT enter a passphrase when prompted - just hit Return)

[local ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/jones/.ssh/id_rsa): /Users/jones/.ssh/id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/jones/.ssh/id_rsa.
Your public key has been saved in /Users/jones/.ssh/id_rsa.pub.
The key fingerprint is:
[...]

Create a .ssh directory in your home directory on the remote server if
it doesn't already exist, then copy the the public key (id_rsa.pub) to the remote machine that you want to connect to.

[local ~]$ scp .ssh/id_rsa.pub jones@svn.craic.com:/home/jones/.ssh/tmp.pub

Now go to the remote server and install the key in ~/.ssh.
[remote ~]$ cd .ssh

Look for a file called authorized_keys, create it if it does not exist
[remote .ssh]$ mv tmp.pub authorized_keys
Or append the new key on to the end of it.
[remote .ssh]$ cat tmp.pub >> authorized_keys

Now, back on the local client, you should be able to use the svn command I gave above without having to give a password.
[local ~]$ svn list svn+ssh://svn.craic.com/proj/svn_repository/project1/trunk

In addition, you can now ssh into the remote server without having to give a password. In some cases this is a nice side benefit, but some sites may wish to separate svn access from ssh access to a server. Doing this can have several benefits, especially if you have multiple private/public keypairs for different purposes, plus it lets you to simplify your svn client commands a little.

On the client, rename your private key to something that indicates it is specifically for svn.
[local ~]$ cd .ssh
[local ~]$ mv id_rsa id_rsa_svn

Add a line to your shell settings file, such as ~/.bashrc or ~/.bash_login, that tells svn which key to use when it runs ssh.

export SVN_SSH="ssh -i $HOME/.ssh/id_rsa_svn"

Now go back to the remote server and customize the authorized_keys file so that this public key can only be used with svn. Add the string
command='/usr/bin/svnserve -t -r /proj/svn_repository'
to the start of the line for the public key for your client, thus:


command='/usr/bin/svnserve -t -r /proj/svn_repository' ssh-rsa AAFaB3NzaC1[...]2oT+HQ== jones@client.craic.com

What this does is to start up an instance of the SVN server (svnserve) whenever you connect from your client via ssh. But more than that, the string
-r /proj/svn_repository specifies the root of the repository path. That means that the URL that you specify on your client becomes quite a bit shorter.

So instead of using this command to list files in a project:
[local ~]$ svn list svn+ssh://svn.craic.com/proj/svn_repository/project1/trunk

You can use this:
[local ~]$ svn list svn+ssh://svn.craic.com/project1/trunk

That is all you should need to access your remote repository. Check out the svn book and other resources for details on how you actually work with the files stored therein.




Thursday, April 5, 2007

Perl CPAN on Mac OSX

Two things to remember when using CPAN to install or upgrade a Perl module on a Mac OS X system:

1. Set FTP to use Passive Mode
Otherwise it will just spin its wheels trying to find a cpan repository that it can download from.

In your bash shell, or your ~/.bash_login file add this line:
export FTP_PASSIVE=1

2. Run the CPAN interface under sudo
You need root privileges to install the modules after they have compiled.
sudo perl -MCPAN -e "shell"

The CPAN may recommend updating your Bundle::CPAN module. While this is a good thing to do, it does take a long time to install/update all the supporting modules (10-15 minutes), so bear that in mind.

Tuesday, January 30, 2007

Using Oracle as a Backend to Rails

In a recent project I was asked to use Oracle as the database for a Rails application rather than MySQL. I am somewhat of an Oracle newbie and so setting this up was a bit of a learning experience. This note describes the steps that it took to accomplish the task.

For testing purposes I installed a copy of Oracle Database Express Edition (XE) on one of my servers. This is a free download from Oracle that offers much of the core functionality, albeit with some restrictions. I've written up the steps needed to Install Oracle XE on Fedora Core 5 Linux in a separate note. The following steps assume that you have this up and running.

My installation had the Rails application and web server on one machine and the Oracle database on another.

Oracle has its own way of handling client/server comunication over a network. For a Rails application to talk to a remote database it needs to access all that machinery and the preferred way of doing that in Rails is to use a Ruby interface to the Oracle Call Interface. This resides on the machine that hosts the Rails application.

So you need to install an Oracle client interface, the Ruby OCI interface and tell Rails how to connect to the database.

The Ruby OCI interface is called ruby-oci8 and comes in two variants, depending on the Oracle client software that you installed. In the Oracle/Linux installation note I installed the Full Client, however I ran into problems getting ruby-oci8 to work with the libraries included with that client (it works fine with the Server libraries). So I have to recommend the other route using the Oracle Instant Client.

1. Install Oracle Instant Client

You can download the Instant Client packages for free from Oracle, although you will need to create an account for yourself first. Pick your platform, agree to their license and you'll get a page full of packages in different versions. All you need are these two from Version 10.2:
Instant Client Package - Basic
instantclient-basic-linux-x86-64-10.2.0.3-20070103.zip (36 MB)
Instant Client Package - SDK
instantclient-sdk-linux-x86-64-10.2.0.3-20070103.zip (0.6 MB)
The ruby-oci8 install guide tells you to install these into /opt/oracle but that is probably not mandatory. I had downloaded the files into /proj/downloads.
# mkdir /opt/oracle
# cd /opt/oracle
# unzip /proj/downloads/instantclient-basic-linux-x86-64-10.2.0.3-20070103.zip
# unzip /proj/downloads/instantclient-sdk-linux-x86-64-10.2.0.3-20070103.zip
# cd instantclient_10_2
Then create the following symbolic link
# ln -s libclntsh.so.10.1 libclntsh.so
Edit /etc/bashrc to include the directory in your LD_LIBRARY_PATH environment variable by adding these lines
LD_LIBRARY_PATH=/opt/oracle/instantclient_10_2:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH
The start a new shell, or source /etc/bashrc, in order to see the new value for the variable. Installing ruby-oci8 won't work without it!

2. Install ruby-oci8

Download version 1 of the interface from rubyforge:
# wget http://rubyforge.org/frs/download.php/16630/ruby-oci8-1.0.0-rc1.tar.gz
Unpack, make and install the package
# tar -zxvf ruby-oci8-1.0.0-rc1.tar.gz
# cd ruby-oci8-1.0.0-rc1
# make
# make install
The make calls ruby setup.rb to configure and then compile the code. It should all go smoothly provided the Oracle libraries are in place and it knows where to find them.

The official ruby-oci8 install guide may help if you have problems.

3. Test out the Connection to the Remote Database

This assumes that the remote database is running and that you have enable the test HR database that ships with the server. Test the connection with a simple Ruby one-liner. Note that this specifies the remote host using Oracle's Easy Connect naming scheme (bit like a URL). Alternatively you can set up a TNSNAMES.ORA files if you know about all that. Here the username is hr, the password is hr, the remote host is testbed.int.craic.com and the Oracle SID is XE. If the call is successful then you should get back a bunch of lines listing various jobs in the database.
# ruby -r oci8 -e "OCI8.new('hr', 'hr', '//testbed.int.craic.com/XE').exec(
'select * from jobs') do |r| puts r.join(','); end"
AD_PRES,President,20000,40000
AD_VP,Administration Vice President,15000,30000
AD_ASST,Administration Assistant,3000,6000
FI_MGR,Finance Manager,8200,16000
[...]
If that works then you have finished the installation and can now move on to creating your Rails application. If, like me, you are not very familiar with setting up a database in Oracle here are some steps to get you started.

4. Create a 'Database' in Oracle

First of all, Oracle does not use that term in the way you might expect if you are coming from MySQL!

Oracle has an instance, identified by an SID (which in the above case is XE). Within that instance you need to create a new User and in doing so you create a new tablespace (I think that is the right term) which is equivalent to creating a database in MySQL.

You need to access the Oracle Server web site on your server, login as system and create the new user. Use the name of your Rails app for convenience. I use myapp in this example. Enter a password and check the boxes Connect and Resource in the Roles section. Don't worry about the Directly Granted System Privileges. Then click Alter User to set it up.

5. Create Your Rails Application

Back to you client machine. Assuming that you have Rails already setup, go to your target directory and create the subdirectories for your app.
# cd /proj/rails
# rails myapp
Then configure your database.yml file. You will actually want three Oracle users, one for the development, production and test databases, but just consider the development version right now. A suitable block for that file would be:
development:
adapter: oci
# database: myapp_development
host: testbed.int.craic.com/XE
username: myapp
password: myapp
Note that the database: key is not relevant for Oracle, as I mentioned. The hostname is this Oracle-specific combination of the real hostname followed by a slash and the SID (which is always XE for Oracle Express Edition)

(Update: you can create your rails app with this option:
# rails myapp --database=oracle
Which specifies the adapter to be oracle instead of oci, but when doing so I got an error telling me that the TNS service name was not properly specified. But if you use the oci adapter as above it should work fine.)

From here on out you are working in regular Rails. You can create and modify your tables using migrations, you can CRUD your data and everything should just work.

If you are coming from MySQL, like me, then be aware that Oracle uses different underlying data types than MySQL and it appears that can be an issue with regard to dates and times in certain cases. Don't use the datatype :tinyint as it will barf - use :integer, :limit => 4 instead.

Note that in Rails 1.2.1 there is a bug that, with Oracle, will put the text empty_clob() in to empty textareas in a form. This has been noted and fixed but is not yet released. A simple work around is to set that field explicitly to blank string in the new action in your controller. For example:
def new
@blog_entry = BlogEntry.new
@blog_entry.text = ''
end

Monday, January 29, 2007

Configuring Oracle Database XE on Linux

I do all my database work in MySQL but I recently needed to setup a Rails application to use Oracle as a backend database. I didn't want to spend any of my money to do this, so I was pleased to see that Oracle make Oracle Database Express Edition (Oracle Database XE) available as free download.

You can find the software HERE but you will need to create an Oracle account first. Be warned that the Oracle site can be a little frustrating. You will want to download Oracle Database 10g Express Edition (either Western European or Universal version depending on your language requirements), which represents the Server, and Oracle Database 10g Express Client, which allows remote access to the server from another machine.

Oracle provides detailed documentation on these, including an Installation Guide, a Getting Started Guide and an Online Tutorial. While these are welcome, they tend to cover more options than you really need and so they can be a bit confusing. This note trims off that excess verbage and explains what I needed to do to install the database on a Linux Fedora 5 server.


Oracle XE Server Installation

1. Check the prerequisites in the Installation Guide
For hardware these are basically 512 MB memory, 1.5 GB disk.
You'll need at least 1024 MB of swap space. Check that with:
# free -m
If the total for the Swap line is less that 1024 then you need to add a swapfile of an appropriate size. I'll try and add a short note about that to this site when I get the chance - otherwise google for it.

You'll need the linux packages glibc (2.3.2 or higher) and libaio (0.3.96 or higher). You should have glibc already there but you'll have to install libaio (as root):
# yum install libaio
There are also a bunch of kernel parameters listed in the installation guide. I didn't have to mess with these.

2. Install the Downloaded RPM
# rpm -ivh oracle-xe-10.2.0.1-1.0.i386.rpm
Preparing... ########################################### [100%]
1:oracle-xe ########################################### [100%]
Executing Post-install steps...
You must run '/etc/init.d/oracle-xe configure' as the root user to
configure the database.
If you've met all the prerequisites, the rpm should install smoothly

3. Run the Configuration Script
# /etc/init.d/oracle-xe configure
Accept the defaults for the two questions about ports, provide an administrator password and set the database to start on boot. The script will then do a load of work in the background to configure things properly.

This can take quite a while. Run top in another window if you want reassurance that things are happening.

4. Access the Database via its Web Site

On the server with the database, open up a browser and go to http://localhost:8080/apex.
Login as system and give the password that you set earlier. If you want to access this site from other machines on your network then go Administration->Manage HTTP Access and select Available from Local Server and Remote Clients.

You can enable a test database by going to Administration->Database Users->Manage Users and clicking on HR. Provide a password (such as 'hr') and change Account Status to unlocked and click the Alter User button. This test database is handy for testing later on.

5. Set Some Environment Variables

As root, open /etc/bashrc in an editor and add this line at the bottom of the file:
. /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/bin/oracle_env.sh
When you create a shell this will set a few environment variables that are required in order to access the database. These include LD_LIBRARY_PATH, ORACLE_HOME and ORACLE_SID. Look for them in the output of printenv when you create a new shell.

That's it for installing the server...


Oracle XE Client Installation

You would install this on a remote machine that wants to access the database server over the network.

1. Check the prerequisites

Nothing significant here, except the glibc and libaio packages, as above:
# yum install libaio
2. Install the Downloaded RPM

This gave me a couple of errors but they didn't appear to be a problem...
# rpm -ivh oracle-xe-client-10.2.0.1-1.0.i386.rpm
Preparing... ########################################### [100%]
df: `/usr/lib/oracle': No such file or directory
expr: syntax error
/var/tmp/rpm-tmp.86099: line 23: [: -lt: unary operator expected
1:oracle-xe-client ########################################### [100%]
Executing Post-install steps...
3. Setup the Environment Variables

As root, open /etc/bashrc in an editor and add this line at the bottom of the file:
. /usr/lib/oracle/xe/app/oracle/product/10.2.0/client/bin/oracle_env.sh
4. Access the Remote Server

If your paths are setup correctly then you can access sqlplus from a shell. Here I am accessing the HR test database on server testbed. The connect syntax is username/password@host
# sqlplus /nolog

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Jan 29 10:21:21 2007

Copyright (c) 1982, 2005, Oracle. All rights reserved.

SQL> connect hr/hr@testbed.int.craic.com
Connected.

SQL> select * from jobs;
You see a load of output returned by the server, indicating that everything is set up correctly.

That's it... fairly simple installations... you'll need to look elsewhere for guidance on actually creating tables and entering data. I'll add a guide to using Oracle as a backend database with Rails shortly.

Thursday, January 25, 2007

Configuration of Web Servers and Rails

When it comes to setting up a web server to host a Rails application, there are a number of choices. Unfortunately, the documentation and HOWTOs that describe these options can be confusing, often getting bogged down in the details of compiling pieces of code, etc.

This note an attempt to present the range of options and to guide you through the installation of each. In summary, your choices are:
1. WEBrick - built into Rails
2. Apache
3. Apache with FastCGI - higher performance
4. Lighttpd - a fast alternative to Apache
5. Lighttpd with FastCGI
6. Mongrel - a server written in Ruby
7. A Hybrid Solution

Lighttpd with FastCGI is a common solution for production Rails servers. Apache is the most common server on the Internet but as you will see there are issues in using it compared to the alternatives. Mongrel is a relative newcomer that is getting a lot of attention.

The installation steps given here are for Linux Fedora Core 5 but these will likely apply to other current Unix variants.

1: WEBrick

The simplest option is to use the WEBrick server that is built into each Rails installation. You don't have to install anything over and above regular Rails.

From a terminal, cd to the top level directory of your application and run:
# script/server
By default this runs a server on localhost (i.e. locally to that machine) on port 3000. Point a browser on that machine to http://localhost:3000 and you should see your application. There is no configuration file.

This is great for development, but doesn't cut it if you want anyone else to use your application.

2: Apache

Most Linux installations will already have the Apache httpd server installed and ready to run. Perhaps the easiest way to see if it is running is (as root):
# /sbin/service httpd status
You can start, stop or restart it with
# /sbin/service httpd start
# /sbin/service httpd stop
# /sbin/service httpd restart
The default configuration file for Apache (/etc/httpd/conf/httpd.conf) needs to be modified in order to run a Rails application. Make a backup copy of the file before you make any changes!

Open the file in an editor, look for these two LoadModule lines and make sure they are not commented out:
LoadModule env_module modules/mod_env.so
LoadModule rewrite_module modules/mod_rewrite.so
Then go to the bottom of the file and add the following block, replacing MYAPP with the path to your application directory:
<VirtualHost *:80>
SetEnv RAILS_ENV development

DocumentRoot /MYAPP/public/
ErrorLog /MYAPP/log/apache.log

<Directory "/MYAPP/public">
Options ExecCGI FollowSymLinks
AddHandler cgi-script .cgi
AllowOverride all
Order allow,deny
Allow from all
</Directory>
</VirtualHost>
Now when you restart Apache and point your browser to http://localhost (you don't need the :3000 here), or its real hostname from a browser on a different machine, you should get the Rails welcome page.

If you get a '403' page telling you that you don't have permission then make sure that all the directories including and above the application directory are world-readable.

So now you have the application running under Apache. Apache is great but does not give you the highest performance possible. You can improve its response by adding in the FastCGI module.

3: Apache + FastCGI

This is typically where the HOWTOs get really confusing, really quickly...

The bottom line is that, for now, you might find another one of the server options an easier path than this one.

FastCGI is an extension to CGI that greatly improves server performance. You can learn more and download it from http://www.fastcgi.com.

You will need to download and compile two components - fcgi, which is the 'core' FastCGI software - and mod_fastcgi, the Apache module component that links fcgi and the web server.

You should be root when you install all the following packages.

Before doing this, make sure you have these Apache development packages installed. They may not all be necessary, but play it safe. Install them like this:
# yum install httpd-devel apr apr-devel apr-util-devel
The following assumes that you can use wget to fetch remote files. If not, fetch them with your browser. Download and compile in a temporary or scratch directory. You might want to check for newer versions of these files, but they have not changed since 2003.

Install the software under /usr/local to isolate it from general Linux software.

To install fcgi:
# wget http://www.fastcgi.com/dist/fcgi-2.4.0.tar.gz
# tar -zxvf fcgi-2.4.0.tar.gz
# cd fcgi-2.4.0
# ./configure --prefix=/usr/local
# make
# make install
If this all works as expected then you'll be able to see a bunch of libfcgi files in /usr/local/lib.

Now you need to install a ruby gem that allows it to use fcgi. You need to tell it where the fcgi libraries and include files are installed.
# gem install fcgi -r -- --with-fcgi-lib=/usr/local/lib --with-fcgi-include=/usr/local/include


Next you install the mod_fastcgi module for Apache... and here's the problem...

At the time of writing (January 2007) the current version mod_fastcgi-2.4.2 does NOT COMPILE with Apache 2.2. The solution requires you to patch the mod_fastcgi source.. messy, very messy... I've written a separate post that will guide you through the process: Compiling mod_fastcgi-2.4.2 for Apache 2.2. I will do my best to update this post if and when the problem is fixed.

Those instructions show you how to patch the source, compile the source and update the httpd.conf file.

If you have been brave, or foolhardy, enough to go through all that rigmarole then there is just two more steps left before your Rails app can utilize the performance boost that FastCGI offers.

In your Rails application directory, go to public and edit the .htaccess file. Change the line:
RewriteRule ^(.*)$ dispatch.cgi [QSA,L]
to
RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]
Restart Apache and you should be good to go...

Finally, check that the log directory in your application is world-writeable so that fcgi can create a file called fastcgi.crash.log

4: Lighttpd

Lighttpd (aka LightTPD and pronounced 'lighty') is an alternative to Apache that offers many of the same features but with smaller resource requirements and higher performance. It has been long favored by the Rails community, although Mongrel is challenging that status.

Before you start, make sure that you have /usr/local/sbin in your path. You can put this line in the .bash_login file for each user, or in /etc/profile to set up all users:
export PATH="/usr/local/sbin:$PATH"
. Start a new shell to make sure you pick up the new PATH.

Installation is straightforward, but it does have a prerequisite in the PCRE library which is used for regular expression pattern matching. Install PCRE and Lighttpd into /usr/local. You will see a ton of compile commands and warnings but don't worry.

Note the CFLAGS option. This was included in Dan Benjamin's install instructions for Mac OS X and is needed for Intel Macs. You may not need it, but it'll do no harm. The option is written as 'Oh-One' not 'Zero-One'.
# wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-6.6.tar.gz
# tar -zxvf pcre-6.6.tar.gz
# cd pcre-6.6
# ./configure --prefix=/usr/local CFLAGS=-O1
# make
# make install
Now fetch the lighttpd distribution (check for a more recent version first) and install it thus:
# wget http://www.lighttpd.net/download/lighttpd-1.4.13.tar.gz
# tar -zxvf lighttpd-1.4.13.tar.gz
# cd lighttpd-1.4.13
# ./configure --prefix=/usr/local --with-pcre=/usr/local
# make
# make install
The lighttpd executable is located at /usr/local/sbin/lighttpd

Assuming that you already had Apache installed on your system, you will need to replace references to it in your system startup scripts, and you will need to modify the lighttpd config file to use your Rails application.

On Linux the httpd startup script is /etc/rc.d/init.d/httpd. Make a backup copy of this before you start messing with it. You need to replace all references to the Apache httpd with the lighttpd equivalents. To avoid cluttering this post, I've put a lighttpd specific startup file on my site HERE. You can also look at the script rc.lighttpd included in the doc subdirectory of the lighttpd source distribution.

The one line that you will need to change is the location of the lighttpd config file. If you only plan to run a single Rails app on your system then you can use the one in your application directory tree. Otherwise you need to place one in a suitable location such as /etc/httpd/conf/lighttpd.conf.

Detailed help on the config file can be found HERE, but for a Rails application you should start with the lighttpd.conf file in your application config directory.

What's that? You don't have lighttpd.conf in your config directory? Rails will take care of that for you!

Assuming you have /usr/local/sbin in your PATH, go to the Rails App directory and start the built-in server thus:
# script/server
=> Booting lighttpd (use 'script/server webrick' to force WEBrick)
=> config/lighttpd.conf not found, copying from
/usr/lib/ruby/gems/1.8/gems/rails-1.2.1/configs/lighttpd.conf
=> Rails application starting on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server (see config/lighttpd.conf for options)
Rails detects lighttpd in your path and uses that in preference to WEBrick. Not only that but it will copy a lighttpd.conf file into your config directory if it doesn't see one. Pretty neat...

To serve this application to users on other machines you will want to copy this local config file and modify it.

The two most important lines in that file are at the beginning:
server.bind = "0.0.0.0"
server.port = 3000
These tell the server to just respond to requests on that machine (localhost) on port 3000. This is the default for the development server. Change these to the real hostname of the machine and port 80.

Next you need to change all references to CWD to the real path to that directory (/proj/rails/myapp for example).

Reference this file in the lighttpd startup script, restart the server and go to that URL in a browser on this or another machine. You should see the Rails app welcome page. You're up and running with lighttpd!

5: Lighttpd + FastCGI

You can add performance to a lighttpd server using FastCGI, just like you can with Apache, although setting it up is much easier. Before you install lighttpd, install the fcgi software as described in the Apache + FastCGI section, install the fcgi gem but do not go through all the mod_fastcgi nonsense! The lighttpd installation comes with its own mod_fastcgi module.

Lighttpd picks up on the fact that you have fcgi installed and makes use of it. You'll see it referenced in the lighttpd.conf file that Rails sets up for you.

6: Mongrel

Mongrel is a web server written in Ruby (with C) that is extremely easy to get running and that plays well with Rails.

Installation involves nothing more than installing a gem.
# gem install mongrel
But note that at the time of writing (01/25/2007) Mongrel 1.0.1 requires Ruby 1.8.4 and does not install on Ruby 1.8.5.

You can run mongrel as a local development server. Instead of running script/server:

# cd myrailsapp
# mongrel_rails start
Its performance is apparently faster than WEBrick and it does not need a configuration file in many cases.

One advantage it offers over other solutions in production systems is a simple way to setup clusters of servers that can help load balancing. But that is beyond the scope of this note.

7: A Hybrid Solution

You would not use Mongrel by itself as a production server. Instead you combine it with Apache (or lighttpd). Apache acts as the front end, proxying certain requests on to mongrel. This hybrid solution may appear complex but it allows you to utilize the best features of both servers. Apache is better at serving static content like stylesheets and images, whereas mongrel will be better at serving dynamic content coming from your Rails application. Setting this up seems fairly straightforward. Check the mongrel web site for details and examples.

Similarly you can combine Apache and lighttpd by adding ProxyPass directives to an Apache configuration. You might want to do this if your server already hosts a site under Apache that you don't want to change. You can add in a Rails application linked to lighttpd on a different port with Apache as the front end, redirecting requests or processing them as appropriate.

An example configuration block is:
<VirtualHost *:80>
ServerName myapp.com
ServerAlias www.myapp.com

ProxyPass / http://www.myapp.com:8000/
ProxyPassReverse / http://www.myapp.com:8000
ProxyPreserveHost on
</VirtualHost>
This redirects all requests to the virtual host to a second server running on port 8000.



A Note About Sources of Information

This note is the result of reading a lot of HOWTOs and notes about web server installation, a lot of experimentation and quite a bit of frustration.

The Ruby on Rails site lists many such documents, but some of these are confusing and/or outdated - caveat emptor.

Some of the useful documents for me have been:
Rails on lighttpd - Duncan Davidson
Building Ruby, Rails, LightTPD, and MySQL on Tiger - Dan Benjamin
Ruby on Rails (with FastCGI) Howto - "goldenratio"
Thanks to those authors and others who have helped guide me through some of the thorny installation issues.

mod_fastcgi-2.4.2 and Apache 2.2

This is a description of the steps needed to compile the mod-fastcgi module for Apache 2.2.

If you are trying to improve the performance of you web server or install the Rails web framework with Apache then this tech tip could save you a lot of frustration.

At the time of writing (01/25/2007) the distribution of this module (mod_fastacgi-2.4.2) will not compile for Apache 2.2 without patching the source. The problem stems from Apache dropping some definitions from an include file that mod_fastcgi relied upon. mod_fastcgi developers have known about the problem, and have had a workaround, since late 2005 but unfortunately the fix has not made it into a new release. Users are required to patch the source themselves... furthermore, to find this out you have to read through a bunch of messages on the fastcgi developers mailing list and be familiar with the patch command, which many of us are not.

So I've written up the necessary steps here with the goal of lessening this unnecessary pain for others. Hopefully the problem will be fixed properly soon but for now here is what you need to do...

This is written for a Linux Fedora Core 5 system but should be applicable to other Unix variants. You will want to be root when you install the software.

1. Install the Apache development libraries and include files
# yum install httpd-devel apr apr-devel apr-util-devel
On Fedora these will install the libraries into /usr/lib/httpd

2. Fetch the mod_fastcgi distribution and unpack
# wget http://www.fastcgi.com/dist/mod_fastcgi-2.4.2.tar.gz
# tar -zxvf mod_fastcgi-2.4.2.tar.gz
# cd mod_fastcgi-2.4.2
3. Fetch the patch

For convenience I've put a file containing the patch on my site
# wget http://www.craic.com/rails/installation/mod_fastcgi-2.4.2-apache2.2.patch
The original patch was created by Daniel Smertnig and can be found HERE.

4. Patch the distribution

Go to the directory that contains the mod_fastcgi distribution and apply the patch:
# patch -p0 < mod_fastcgi-2.4.2-apache2.2.patch
patching file mod_fastcgi-2.4.2/fcgi.h
patching file mod_fastcgi-2.4.2/Makefile.AP2
Hunk #1 succeeded at 20 with fuzz 1.

5. Fix the Makefile

Now cd into the mod_fasta-2.4.2 directory. Copy the Apache 2 specific Makefile into place:
# cp Makefile.AP2 Makefile
Open Makefile in an editor. Change top_dir to this:
top_dir = /usr/lib/httpd

Then uncomment the INCLUDES line and change it to this:
INCLUDES=-I /usr/include/httpd
Close the file and compile the module thus:
# make
# make install
You will see a bunch of warnings and commands that (hopefully) you can ignore. If the make suceeded you will see the module in the Apache modules directory:
# ls -l /usr/lib/httpd/modules/mod_fastcgi*
-rwxr-xr-x 1 root root 202193 Jan 24 18:07 /usr/lib/httpd/modules/mod_fastcgi.so

6. Update httpd.conf

Edit the Apache config file /etc/httpd/conf/httpd.conf add this line at the end of the LoadModules section:
LoadModule fastcgi_module modules/mod_fastcgi.so

Add this block towards the end of the file - just before the Virtual Hosts section. It has to be after the lines the begin User and Group.
<IfModule mod_fastcgi.c>
FastCgiIpcDir /tmp/fcgi_ipc/
AddHandler fastcgi-script .fcgi
</IfModule>
Save the file and restart Apache
# /sbin/service httpd restart


You should not have to jump through all these hoops to install this. FastCGI is a great piece of software and it is a shame that whoever maintains it has not been able to keep the distribution up to date.

Monday, January 22, 2007

Installing Rails 1.2.1 on Fedora Linux Core 5

There is a proliferation of installation guides at rubyonrails.org and elsewhere. Unfortunately some of these add more confusion than clarity, particularly if the author had to use some funky hack to get things working.

Here are the steps I've taken to install Rails on Fedora Core 5. The Fedora installation included Mysql and its development tools, as well as an installation of Ruby.

You should be root when you install the software. The following steps will install into /usr/bin. Installing into /usr/local/bin protects you from future linux upgrades, but involves additional steps.

1. Update all the basic Ruby packages
# yum update ruby ruby-libs ruby-mode ruby-devel ruby-rdoc ruby-irb ruby-ri ruby-docs

2. Install rubygems
- Get the tar file from http://rubyforge.org/projects/rubygems
- Pick up the latest (at least Rel_0_9_1)
- Download the file (e.g. rubygems_0.9.1.tgz) into a temporary directory
- Extract the file (tar xzvf rubygems_0.9.1.tgz) and cd into that directory
- Run the setup script
# ruby setup.rb

3. Use the gem package manager to install Rails
# gem install rails --include-dependencies

4. At this point you can test out the Rails installation
- cd to a scratch directory and run
# rails myapp

- This will create a directory tree with myapp as its root
# cd myapp
# script/server

- That starts the built-in web server. Point your browser to http://localhost:3000 and you should see the 'Welcome Aboard' home page for the project. If so then your basic installation is good - but you can't actually do anything with it yet.

5. Install Mysql if you don't already have it
# yum install mysql mysql-devel mysql-server

6. Start the mysqld and give the root account a password
# /sbin/service mysqld start
# mysqladmin -u root password <your_password>

7. Install the Ruby / MySQL Bindings
- This has caused confusion in some of the install guides - this works for me.
- Be very careful not to mix up your hyphens and underscores!
- This command does have two hyphens by themselves after the mysql
# gem install mysql -- --with-mysql-config=/usr/bin/mysql_config

You will be prompted for the version of the gem to install. Pick the latest MySQL version for Ruby (e.g. mysql 2.7 (ruby))

At this point you can create a basic Rails application with associated databases and access it from your browser via the built in Webrick server.

The installation is complete with the exception of linking Rails to a production web server like Apache. That can be its own can of worms and so that will be covered in its own posting on this site.

Archive of Tips