Rails 3.2 Asset Pipeline - 'require_tree' is evil !

I have been battling the Asset Pipeline in Rails 3 for a couple of days - and finally I've got it working

An important part of the Asset Pipeline is the use of Manifest files in app/assets/javascripts and stylesheets which determine the specific files to include and, in some cases, the order in which they are processed.

The default javascripts Manifest file has these lines:

//= require jquery
//= require jquery_ujs
//= require_self
//= require_tree .

That reads as:
- process jquery, then jquery_ujs
- process anything in this file (self)
- process all the files in this directory tree (require_tree)

I don't have many JS files in my app - three custom JS files written by myself and a couple of third party libraries.

I try and follow defaults as far as possible but require_tree has two BIG problems.

First is that it processes ALL the files in the directory tree.
Second is that the ORDER is UNDEFINED.

It is common for JS libraries to distribute their code along with examples and minimized versions of the code. In the past I would just drop these distributions in the javascripts directory and then specify which of the files should be included in my app.

So one problem with require_tree is that it will process both a 'native' and minimized version of the same library is both are present. That results in the same functions being defined multiple times - and that seems to be a real problem if a library includes it's own copy of jQuery.

Not specifying the order of processing is the other BIG issue. For libraries with a 'core' JS file along with others will typically require that the core library is processed first.

The solution is to avoid require_tree at all costs. Put only the JS files that you absolutely need in app/assets/javascripts and specify them EXPLICITLY in the order they should be processed.

In my example, my application.js file becomes:

//= require jquery
//= require jquery_ujs
//= require ./tabs_timeline_simple
//= require ./tandem_select
//= require ./raphael-min
//= require ./facebox

I strongly recommend that you start out with everything clearly defined like this. Once you have it all working then you can remove unused files and simplify application.js - if not, you risk going through the same debugging misery that I just went through.

Upgrading from Rails 3.0 to 3.1 - Issues with Javascripts

I have been upgrading a Rails app from 3.0.10 to 3.1.5.

The BIG difference with 3.1 is the use of the Assets pipeline which requires that you move images, stylesheets and javascripts from public/* to app/assets/*. In the case of Stylesheets and Javascripts you also put these under the control of a Sprockets Manifest file: application.js or application.css.scss

That Manifest file caused me problems with my javascripts...

I followed the advice in the Railscasts Episode 282 (which I recommend you look at) and put these lines into my application.js

//= require jquery
//= require jquery_ujs
//= require_self
//= require_tree .

But when I ran my app none of my javascript functions were working. Looking at the source for one of the pages I saw this link

  <script src="/assets/application.js" type="text/javascript"></script>

Clicking on brought up a message, instead of a JS file,  saying that Sprockets was unable to find the file 'jquery'

Jquery is supposed to 'just be there' if you have the jquery-rails gem installed.

I think there were a two, maybe three, issues.

1: Sprockets requires that there are NO BLANK LINES in the Manifest file BEFORE the end of the lines that should be processed. In Rails 3.2 the default file says that explicitly but if you are creating your own that is an easy thing to overlook.

2: Look at the 4 lines shown above. The first two are 'require', followed by a SPACE and then the name of the javascript file.

The Third is 'require_self' - underscore, no space - that means process any JS code in this file.
The Fourth is 'require_tree' - underscore, no space - and that means process ALL JS files in this directory.

If you type these lines in and happen to use a space instead of the underscore then Sprockets will look for JS file called 'self' and 'tree' - and won't find them. So be very careful tying those in.

3: The main problem was the require_tree line

I have a number of JS files that are included as needed in certain views. At the top of each view page I specify those with a content_for block like this:

<% content_for :custom_javascript_includes do %>
  <%= javascript_include_tag "raphael.js" %>
  <%= javascript_include_tag "tabs_timeline_simple.js" %>
<% end %>

and in my application.html file I have

  <%= javascript_include_tag "application" %>
  <%= yield :custom_javascript_includes %>

That has worked just fine in the past and I kept all the custom JS files in the javascripts directory. But require_tree will load ALL of them on EVERY page. I assume that there are some variable/function names that are duplicated and/or conflicting and so with all that loaded, nothing worked.

The solution is to get rid of the require_tree line completely.

If I had JS files that should be included in each page then I would 'require' each of them explicitly. This is the same approach that Ryan Bates suggests for CSS files. It just gives you more flexibility.

Uploading contents of a PostgreSQL database to Heroku without db:push

I just created a new Rails app on my local machine and want to deploy it to Heroku. I've done this before and the uploading of the code typically works great - and it did today.

Then I wanted to upload the contents of a small PostgreSQL database to Heroku - and that's where I ran into issues.

Heroku has recently switched from a Heroku gem to Heroku Toolbelt which I installed onto my Mac OS X machine.

I should be able to run 'heroku db:push' from my Rails app and the data would get moved over using the taps gem. But it failed with this error:

$ heroku db:push
! Taps Load Error: cannot load such file -- taps/operation
! You may need to install or update the taps gem to use db commands.
! On most systems this will be:
!
! sudo gem install taps

I have taps installed, so what is going on? Well, I'm not sure but I run my rubies under rbenv which allows you to manage multiple versions - and Heroku's client doesn't seem able to figure out where the gems are located with rbenv.

I dug around in /usr/local/heroku and hacked /usr/local/heroku/lib/heroku/command/db.rb to tell me what it thought the load path for ruby gems was, using this line:

STDERR.puts "DEBUG: load path  #{$:}"

The paths given in that were all in /usr/local/heroku/vendor/gems and my gem paths were nowhere to be found. I tried installing the taps, sqlite3 and sequel gems to that directory and I was able to get a bit further but I still couldn't get all the way - so I gave up on that path.

And then I found mention of a page on Heroku's site that deals with the pgbackups addon. Part of that describes how to import from a PostgreSQL backup directly, without having to use the heroku clients or taps.

I followed that and it worked great - here are the steps involved. Your starting data must be in PostgreSQL and you need to have an account on Amazon S3 (and know how to use it)

1: On your machine create a backup of your PostgreSQL database - 'mydb' is your database name and 'myuser' is your database user.

$ pg_dump -Fc --no-acl --no-owner -h localhost -U myuser mydatabase > mydb.dump

2: Transfer that to an Amazon S3 bucket and either make it world readable or set a temporary access url.

3: Check that you can get the file back out of S3 using curl

$ curl 'https://s3.amazonaws.com/mybucket/mydb.dump' > tmp
$ ls -l tmp

4: Enable the pgbackups addon in Heroku

$ heroku addons:add pgbackups

5: Get the name of the Heroku database that you will restore to, by capturing a backup from the empty heroku db:

$ heroku pgbackups:capture

HEROKU_POSTGRESQL_CRIMSON_URL (DATABASE_URL)  ----backup--->  b001

Capturing... done
Storing... done

6: Restore to that database from the S3 URL

$ heroku pgbackups:restore HEROKU_POSTGRESQL_CRIMSON_URL 'https://s3.amazonaws.com/mybucket/mydb.dump'

7: Test it out - my app was already up so I just had to browse to the appropriate page and there were all my records.

8: Remove your S3 file

DONE !