Linking Github and Lighthouse

Github is a hosted source code repository for projects managed with the git SCM system.

Lighthouse is a hosted bug/issue tracking system.

You can link the two so that you can embed Lighthouse ticket numbers in your Git commit messages and automatically change the status of Lighthouse tickets when you commit.

The steps involved in linking the two applications are listed on the site but they are a little difficult to find. Here are the steps that I used to link the sites.

I'm using private repositories on github, but that shouldn't matter.

1: On Lighthouse, generate a token to use from github.

1.1: Go to 'My Profile' in the top right corner of each page. In the right hand side bar you will see 'Create a Token' and a 'choose an account' pulldown. Select the Account you want to send messages to - I just have one account.

1.2: A panel will appear asking you to enter a label - this will identify the token in a list on Lighthouse. I call mine something like myproject_github. Select 'full access' for github linkage as it has to update your Lighthouse database. And then select the Project that you want to link to. This can be a single project or all of them. You can have multiple tokens so I would recommend one token per project.

1.3: Create the token and a record will appear in the right hand panel with the project name, your label and a grayed out hexadecimal string.

2: On Github go to the linked repository in your account

2.1: In the light yellow panel, click on the 'edit' Button next to the repository name (not the other edit links)

2.2: Above the light yellow panel is a gray bar with 'General', 'Collaborators' and 'Services' on it. Click on Services.

2.3: The returned page will have multiple panels for Twitter, Lighthouse, etc. Scroll down to the Lighthouse one.

2.4: This panel has entry boxes for 'Subdomain', 'Project id' and 'Token'. This is confusing as these names don't match up with Lighthouse!

Subdomain is the same as your Lighthouse Account name. So if your Lighthouse URL is something like http://acme.lightouseapp.com then 'acme' is what you put in subdomain.

Project Id is NOT your Lighthouse project Name - it's the NUMERIC Project Id - when you go to your Project page on Lighthouse the URL will look something like:

http://acme.lighthouseapp.com/projects/12345-myproject/overview

Your Project Id is that number - in this case 12345

Token is that hexadecimal string you just created on Lighthouse. Cut and paste that into this entry box.

Check the 'active' box and click update settings.

3: Test it out...

3.1: For the purposes of testing, create a dummy ticket in your project on Lighthouse and make a note of the ticket number - let's say ours is #21

3.2: Make some change in your local clone of your github repository and commit that to your local repository, giving it a message that includes an embedded message for Lighthouse, and then push that up to github. For example:
$ git commit -m 'Made a trivial change [#21 state:resolved]'
$ git push

The text in the square brackets is the message for Lighthouse. The project is implicit in the token you created, #21 identifies the ticket and 'status:resolved' tells Lighthouse that this code change fixes that issue. The Lighthouse docs list out the full range of messages that can be passed.

3.3: Now go back to your Lighthouse project and refresh and you should see that ticket #21 is now marked as 'Resolved' - brilliant! Click on the Ticket message and you'll see a link back to your Github repository that details the fix.

3.4: Github does not seem to retain a link to the matching Lighthouse records - but I bet that will come in due course.

That's it...

Github and Lighthouse are both pretty nice sites. I think Lighthouse still has a few wrinkles but I'm sure those will get sorted out. Both sites need better documentation/help. New features are getting reported in their blogs before any documentation appears and the laudable goal of keeping the user interface simple has led to a few things being 'implicit' when they should be 'explicit' - but overall I think the services are great and they have already made a big impact in the way I interact with some of my clients.

Loading CSV data from Excel into MySQL on a Mac

Loading data from a CSV file into MySQL requires that your file has the same number of columns in the same order as the MySQL table.

That means that you need to populate any column with an auto incrementing primary key, as well as any timestamp column that might be generated by your application. This is particularly relevant if you want to populate a tables used in a Rails web application.

1: Build your table in Excel, adding a primary key column if necessary and populating it using the Edit->Fill->Series... feature in Excel.

2: Save it in CSV (Comma delimited) format - don't bother with the CSV Windows, etc options. Don't worry about Excel's warnings about losing formatting (unless of course you have important formatting...)

3: Optionally load it into an editor like TextMate (turn on 'show invisibles') and do any fine tuning you might want or need to do.

4: Run the Mysql client and from the mysql> prompt load in the data using a command like this:

mysql> load data local infile 'products.csv' into table products fields terminated by ',' lines terminated by '\r';

I needed the '\r' line termination loading the file from Mac OS X. It always takes me a couple of tries before I get the right \n and/or \r combination...

Using Older Versions of Rails

You want to develop with the latest version of Rails but you have an existing application that
uses an older version that you are not ready to bring up to date. What do you do?

By default gem will keep older versions of installed gems until you tell it not to. To see what you have installed:
$ gem list --local
[...]
rails (2.1.0, 1.2.3, 1.2.1)

To use a specific version in your application add a line like this to the bottom of your config/environment.rb file
RAILS_GEM_VERSION = '1.2.3'

That should just work.

You can get rid of old versions of gems with this command:
$ gem cleanup

If you remove an old version by mistake you can always reinstall it with this gem command:
$ sudo gem install rails --version 1.2.3

When you are ready to move your application to the current version of rails then remove the line from environment.rb and bring your application files up to date with:
$ rake rails:update

Now, let's say you have installed the current version of Rails (say, 2.1.0) but you need to build an application that uses an older version. According to 'rails --help' there is no way to specify a version to use. It turns out that there is a hidden option available that does this:
$ rails _1.2.6_ myapp

Simple! Why this isn't explicitly documented I don't know. I found out about it from this post: http://rubybook.ca/2008/08/06/downgrade-older-version-rails/. Obviously you need to have the older version of Rails available on your system.

Upgrading to Rails 2.1 on Mac OS X

I've just upgraded to Rails 2.1 on my Mac and ran into some problems.

I was upgrading from Rails 1.2.3 and gem 1.0.1 on a Mac OS X 10.5.2 system and I keep my own ruby installation in /usr/local instead of using the Apple supplied version. Rails installed just fine but running 'rails myapp' generated an error.

$ rails myapp
/usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:379:in `report_activate_error': RubyGem version error: rake(0.7.1 not >= 0.8.1) (Gem::LoadError)

My version of gem is supposed to include any dependencies by default and in fact adding the --include-dependencies argument gives a message that it will be ignored. In reality it does not upgrade all the gems that rails depends on. You only find out which ones are missing when you run rails and even then you only reveal them one at a time. This is a mess. I find it hard to believe this bug slipped through the release process so maybe I'm doing something wrong. Please tell me if I am! Part of the problem may be that I jumped from Rails 1.2.3 to 2.1 without installing 2.0

Here is what I did to get Rails 2.1 up and running:
1: Update gem - to 1.1.1 in my case
$ sudo gem update --system

2: Update rails
$ sudo gem update rails

3: Update all of these:
$ sudo gem update rake
$ sudo gem update activesupport
$ sudo gem update activerecord
$ sudo gem update actionpack
$ sudo gem update actionmailer
$ sudo gem update activeresource

Note that I had to install activeresource as that is, I believe, new with Rails 2.0 and so I did not have it to update.

4: Check that you have everything you need by trying to create a dummy rails app
$ rails dummyapp

If that dumps out an error message then install or update that gem and try it again until it generates the application directory tree that you expect.

Selenium IDE - Testing Web Applications in the Browser

Selenium is a suite of tools for testing web applications through web browsers. It mimics the user clicks, text entries, pulldowns, etc. and can be an important component of your testing arsenal. Selenium is a product of openqa.org.

This note walks you through the setup and basic operation of Selenium IDE, a plugin for Firefox that lets you record and playback tests of any web application. The plugin does not work with other browsers, but you can still use other components of the larger Selenium suite with IE, Safari, etc.

You can download the plugin from http://selenium-ide.openqa.org/download.jsp. Download it within Firefox and follow the installation and restart steps.

My setup uses Selenium IDE v1.0 Beta 1 in Firefox 2.0.0.14 on Mac OS X 10.5.2

The following examples use Google as the target site. The instructions will apply equally well to your own site.

1: Visit google.com in your Firefox browser and open up the IDE window by going to the Tools menu -> Selenium IDE

2: The IDE window allows you to record the browser events that you invoke as you type and click in the main browser window. In fact, the IDE starts out in Record mode so anything that you do in the browser will be echoed in the IDE.

3: In the main browser type in a Google query term, e.g. 'Craic Computing Tech Tips' then click the Google Search button. You'll get a page of search results and at the same time several lines will appear in the 'Table' panel of the IDE.

First you can see the 'Base URL' entry box at the top of the IDE now has the Google URL. Then in the table panel you will see 3 lines that show the actions you took in the browser.

• open - you opened '/' relative to the base url, i.e. the Google home page
• type - you typed 'craic computing tech tips' into the query box, named 'q' in the form
• clickAndWait - you submitted the form through the 'btnG' button (which is the name Google uses in its form)

If you were to run these commands via the IDE you would recreate the steps that you just did manually.

4: Where Selenium gets interesting is the ability to create test assertions using the page that was returned.

On the Google results page you should see one or two links to this blog, followed by links to craic.com and to various pages with the word 'craic' in them.

Select the phrase 'craiccomputing.blogspot.com', right-click and look in the popup menu for the line 'assertTextPresent craiccomputing.blogspot.com'. Clicking on this adds a new line to IDE with a 'assertTextPresent' command. Running the IDE commands will now test whether that text is present in the page.

You can add multiple assertions to the test and the 'Show All Available Commands' in the right-click menu displays all your options. For now we'll just use this one assertion.

5: Click on the 'Craic Computing Tech Tips' link on the Google results page. You'll be directed to this blog and you'll see another clickAndWait line appeara in the IDE. Add another test by selecting 'Archive of Tips' in the blog page, right-clicking and assertTextPresent as before.

6: Stop the recording by going back to the IDE and click the red button in the right hand corner.

7: You now have a set of commands that will test out two linked web pages. You can rerun this 'Test Case' by clicking either of the two green arrows on the left of the IDE toolbar (leave the one in a square box for now).

Watch your Firefox window when you do this and you will see Selenium mimic what you did before. At the same time the lines in the IDE panel will become colored as each command completes. Assertions that are true are colored a darker shade of green. Any that fail will turn red. You'll also see a bunch of logging messages appearing in the lower panel of the IDE.

Seeing Selenium run complex tests can be pretty cool. If things are moving too quickly you can slow down the events using the 'Fast Slow' slider in the IDE.

8: You can save the Test Case from the IDE menu (File -> Save Test Case). Selenium tests are stored in HTML format so name your test something like 'google_test_1.html'. Take a look at the file to see the format, or click the 'Source' panel in the IDE.

9: Go back to Google's home page again and 'Shift Reload' it to get a blank query box.

10: Create a new test case (File -> New Test Case) and you will see a 'Test Case' list panel appear in the IDE, along with a blank Table panel.

11: Click the red Record button in the IDE, enter a search term in Google and build your own test on some other site like the one you just did, with one or more assertTextPresent commands.

12: Stop recording, run the test to make sure it works and save it to a file with a .html extension as before (e.g. google_test_2.html) in the same directory as the first test.

13: With two test cases we can now save a Test Suite that will allow us to run both of them. Test Suites are also html format files with table rows for each Test Case (File -> Save Test Suite). Remember to give this a .html extension and store it in the same directory as the Test Cases. Take a look at the file in an editor to see the format.

14: With a Test Suite you can now make use of another Selenium interface, the TestRunner. You invoke this by clicking the small green arrow in a square box in the IDE toolbar.

This replaces your main browser contents with the 'Selenium Functional Test Runner' which has 4 panels. In the top left is a list of the component test cases. Clicking on any of these will bring up the contents of the test case in the center panel. The top right panel contains buttons to run the tests and shows the summary results from doing so.

The left most button with a green arrow will run all tests, the next one will run an individual test case. Try running all of them and you will see the various pages appearing in the lower panel of the browser. Green lines and test indicate the tests passed, red indicates failure.

With these simple examples you should not have any failures but you can create one by editing one of the test cases and changing an assertion line to some random piece of text.

The format used for Test Cases is straightforward and the Selenium documentation will show you a wide range of assertions and commands that you can use in creating your own tests in either the IDE or by editing test files. But starting out I would suggest using the record feature of the IDE and walking through your own or other complex applications, perhaps with multiple linked pages, to get a good sense of how Selenium works.

Things can get complicated if you are testing a site with pages that refresh or involve Ajax. Selenium is capable of handling these but get familiar with it on relatively simple sites first. Google Maps would not be a good idea...

Also, be aware that you should type text into entry boxes. If you use completion pulldowns with previous values then Selenium is not able to capture that text, at least at the moment.

Once you've mastered the basic you should look into other aspects of Selenium that will allow you to test IE and Safari and to integrate it with other testing frameworks in Ruby, Rails, etc.

In summary, Selenium IDE is a great way to automate testing of web applications from the perspective that really matters to your users - from the web browser itself. It can save you a great deal of pain in your development process - plus it is a really impressive way to show your clients how thorough your testing process is.

Thanks to everyone involved in producing this amazing piece of code.