General approach to building Rails applications with Devise

I have built a number of complex Rails applications that include user authentication. I want to describe my general approach to getting these up and running.

In most of them I chose the Devise gem to handle authentication and added a number of custom fields to the User model (first name, last name, etc). The gem documentation and this Railscast can help you get started with all that.

One approach is to build your core application without authentication and then add it in at a later stage but I don't like that.

Authentication can quickly get very complicated once you start adding user confirmation via email, an admin role and other custom fields. I prefer to get all that machinery up and running in the context of a very simple application and only then add my real application code into it.

Similarly, when I'm starting with a new application I don't usually hit on the right data architecture right away. That leads to me adding new columns to my database tables and models over time and it always involves quite a lot of 'rake db:rollback / rake db:migrate' operations. I prefer to go through this phase without the extra baggage of authentication, and often, without worrying a lot about CSS etc. I just need to try several alternate views of the problem before I settle on a final design.

So may approach is to build two applications side by side. The first is a minimal application with, say, a single table (usually called 'posts') to which I add Devise and configure all the options that I want.

This is a very useful piece of code for other applications, so I always create a good README and store a copy of the code for future use.

Separately I work through my core application and, once that is working, I condense all the migrations that add/remove/rename columns into a few 'clean', simple migrations.

Finally I add the core application code to the minimal authenticated application. That way it is easier for me to update the controllers, etc. to use authentication.

The bottom line is that you want to minimize the number of moving parts in the early stages of a new application design. This is a simple approach that works very well for me.

Street Intersections in Google Maps

I need to display Google Maps with markers placed at street intersections, starting with an address that looks like '13 Av E / E John St, Seattle, WA'.

Google Maps doesn't know how to parse this and gives me a location on John Street but it's not correct.

Various sources on the web tell you use 'at' as the conjugation between the two streets - so '13 Av E at E John St, Seattle, WA'

This works fine in most cases but in some, like this one, it fails - this gives the intersection of 13th Ave East and E Prospect (7 blocks away).

What seems to work much better is to use 'and' ... '13 Av E and E John St, Seattle, WA' is spot on.

Jquery snippet to disable / enable links on a page

Here are two approaches to hiding all the anchor (a) tags on a page while retaining the html that was contained within the tags. They both use Jquery.

Approach #1 - Remove all the 'a' tags while retaining their content - no way to retrieve the links, etc.

This lets you effectively disable all the links without losing any text. Using the .html() call instead of simply .text() means that any images or other formatting is preserved.

    $("a").each(function() {
      var t = $(this).html();
      $(this).replaceWith(t);
    });

Approach #2 - Reversibly Hide and Show the tags

Replace the 'a' tags with 'span' tags and store the original tag pair in a custom data attribute

Custom data attributes all have the prefix 'data-' followed by a unique name. You can store arbitrary data in these. With this approach I am storing the original 'a' tag pair and its contents.

In the example, my custom data attribute is called 'data-craic' and note that the 'span' tags are given a unique class (I'm using 'hidden-link') that allows you to identify them

To hide/disable the links:

  $("a").each(function() {
    // Get the html within the 'a' tag pair
    var t = $(this).html();

    // Get the entire tag pair as text by wrapping in an in-memory 'p' tag and fetching its html
    var original_tag = $(this).clone().wrap('<p>').parent().html();

    // Replace the 'a' tag with a 'span' - put the original tag in the data attribute
    $(this).replaceWith("<span class='hidden-link' data-craic='" + original_tag + "'>" + t + "</span>"); 
  });

To show/enable the links:

  $(".hidden-link").each(function() {
    // Retrieve the original tag from the data attribute
    var original_tag = this.dataset.craic;

    // Replace the 'span' with it
    $(this).replaceWith(original_tag);
  });

At least in my use cases I have not had to encode the original 'a' tag pairs but base64 encoding might be a good idea to avoid any possible unintended consequences.

Custom data attributes are extremely useful and were the perfect to solution to this problem

You can find a self contained web page with these scripts at https://gist.github.com/craic/4987192

JSON versus JSONP Tutorial

I recently found myself a bit confused on how to convert a JQuery $.getJSON() call to use JSONP (JSON with Padding).

JSONP is a way to get around the Same Origin Policy restriction when a script in one domain wants to fetch JSON data from a server in a different domain.

The approach is simple enough but I had trouble finding a clear explanation. So I wrote up a simple example in both JSON and JSONP with server and client code.

The live demo is at http://json-jsonp-tutorial.craic.com and all the code is on Github at https://github.com/craic/json_jsonp_tutorial

I hope that you find it useful...

Tool to help Count Features in Images

Craic has released a new tool to help count features in images. This can be used in a range of applications such as counting buildings in satellite images, bacterial colonies and cell types in histology images.


It takes the form of a Ruby Sinatra application that fetches remote images and Javascript in the client web page that implements the marking and counting functions.

The user can choose the shape and color of the marker. The current total is updated as features are clicked and the coordinates of each point are stored internally.

The image with the user's marks can then be saved to a PNG format file and the list of coordinate pairs can be displayed in a separate window.

The graphics and interaction are implemented with Canvas and Javascript and make use of the wonderful Canvas2Image code from Jacob Seidelin which allows the current state of a canvas element to be saved to an image file.

The live application is hosted at Heroku and can be accessed at http://counter.craic.com

The code is distributed freely under the terms of the MIT license and is archived at Github

Counting Temporary Shelters in Satellite Images using OpenCV

A number of areas of data analysis are dominated by sophisticated algorithms, intensive computation and, in some cases, limited access to raw data. Examples include the analysis of satellite imagery, feature extraction from video, protein structure analysis and language processing.

I am interested in how simple, approximate methods can be used to extend the application of these technologies. While simple approaches will clearly not match the accuracy and resolution of complex methods, they can, by nature of their simplicity, be implemented and deployed more easily and hence more widely.

I have just posted the code for one of these projects to the Craic Github site. The application involves image processing of satellite images downloaded from Google Maps in order to estimate the number of temporary dwellings or shacks in the slums of Port-au-Prince in Haiti.

The satellite images of Port-au-Prince show large areas covered with small white, blue and rust colored squares. These are the slums of the city in areas such as St.Martin, Cite Soleil and others. 

The blue features are very distinctive and most likely represent the ubiquitous blue plastic tarpaulins the you can find at any hardware store. 

Using the Python interface to the wonderful OpenCV image processing library, I wrote a simple application that identifies blue features below a cutoff size and then computes their area. By fetching adjacent squares that tile across the city, and calculating the area of blue features, I can come up with an matrix showing the relative density of the slums.

The approach is undoubtedly simplistic (the code is only about 35 lines of Python) but it demonstrates how simple approaches can be successfully applied to complex and sophisticated types of data. You do not always need access to expensive commercial software and proprietary datasets in order to work in these fields.

The code is made freely available at https://github.com/craic/count_shelters and you can read my write up of the work so far, with example images HERE.

Tutorial on JS/JQuery Bookmarklets

I recently needed to implement a JavaScript bookmarklet for a new project. There are many web resources available but even so I had a bit of a learning curve to contend with.

In the hope of lessening that burden on future users, I've written up a tutorial on two types of bookmarklets with simple, annotated examples and posted all the code on our Github page.


The simplest form of bookmarklet is a chunk of JS code contained in the Bookmarklet URL that invokes some action. This is a direct approach - you select some text, you click the bookmarklet and you get a result. If you want to repeat the process with some other text on the page then you have to click the bookmarklet link again.

However, for many applications you need to invoke an action multiple times on a single page - looking up words in a dictionary would be a good example. Having to click on a Bookmarklet link every time is not a good solution. Instead you want to modify the behaviour of the page by injecting a custom JS script. A bookmarklet can be used to initiate this. So you end up with three scripts, two of which are slightly modified, general purpose scripts.


The bulk of the code for this second approach comes from how-to-create-a-jquery-bookmarklet, written by Brett Barros of latentmotion.com, with some code input from Paul Irish.

To fully benefit from the tutorial, you should download the project, run 'bundle install' and 'rackup -p 4567' to set up a local Sinatra server, then point your browser to http://localhost:4567