Speeding Up Rails Rake
On a brand new rails project (this article is rails 3, but the same principle applies to rails 2), rake --tasks takes about a second to run. This is just the time it takes to load all the tasks, as a result any task you define will take at least this amount of time to run, even if it is has nothing to do with rails. Tab completion is slow. That makes me sad.
The issue is that since rails and gems can provide rake tasks for your project, the entire rails environment has to be loaded just to figure out which tasks are available. If you are familiar with the tasks available, you can hack around things to wring some extra speed out of your rake.
WARNING: Hacks abound beyond this point. Proceed at own risk.
Below is my edited Rakefile. Narrative continues in the comments below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
# Rakefile def load_rails_environment require File.expand_path('../config/application', __FILE__) require 'rake' Speedtest::Application.load_tasks end # By default, do not load the Rails environment. This allows for faster # loading of all the rake files, so that getting the task list, or kicking # off a spec run (which loads the environment by itself anyways) is much # quicker. if ENV['LOAD_RAILS'] == '1' # Bypass these hacks that prevent the Rails environment loading, so that the # original descriptions and tasks can be seen, or to see other rake tasks provided # by gems. load_rails_environment else # Create a stub task for all Rails provided tasks that will load the Rails # environment, which in will append the real definition of the task to # the end of the stub task, so it will be run directly afterwards. # # Refresh this list with: # LOAD_RAILS=1 rake -T | ruby -ne 'puts $_.split(/\s+/)[1]' | tail -n+2 | xargs %w( about db:create db:drop db:fixtures:load db:migrate db:migrate:status db:rollback db:schema:dump db:schema:load db:seed db:setup db:structure:dump db:version doc:app log:clear middleware notes notes:custom rails:template rails:update routes secret stats test test:recent test:uncommitted time:zones:all tmp:clear tmp:create ).each do |task_name| task task_name do load_rails_environment # Explicitly invoke the rails environment task so that all configuration # gets loaded before the actual task (appended on to this one) runs. Rake::Task['environment'].invoke end end # Create an empty task that will show up in rake -T, instructing how to # get a list of all the actual tasks. This isn't necessary but is a courtesy # to your future self. desc "!!! Default rails tasks are hidden, run with LOAD_RAILS=1 to reveal." task :rails end # Load all tasks defined in lib/tasks/*.rake Dir[File.expand_path("../lib/tasks/", __FILE__) + '/*.rake'].each do |file| load file end |
Now rake --tasks executes near instantaneously, and tasks will generally kick off faster (including rake spec). Much nicer!
This technique has the added benefit of hiding all the built in tasks. Depending on your experience this may not be a win, but since I already know the rails ones by heart, I’m usually only interested in the tasks specific to the project.
I don’t pretend this is a pretty or permanent solution, but I share it here because it has made my life better in recent times.
Capturing output from rake
Rake has an annoying habit of putting it’s own diagnostic line on the first line of output. You can strip that out with tail.
1 |
rake my_report:xml | tail -n+2 > output.xml |
Rake tab completion with caching and namespace support
UPDATE: It now invalidates the cache if you touch lib/tasks/*.rake, for those using it with rails (like me)
There’s a few articles on the net regarding rake tab completion, I had to combine a few of them to get what I wanted:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
#!/usr/bin/env ruby # Complete rake tasks script for bash # Save it somewhere and then add # complete -C path/to/script -o default rake # to your ~/.bashrc # Xavier Shay (http://rhnh.net), combining work from # Francis Hwang ( http://fhwang.net/ ) - http://fhwang.net/rb/rake-complete.rb # Nicholas Seckar <nseckar@gmail.com> - http://www.webtypes.com/2006/03/31/rake-completion-script-that-handles-namespaces # Saimon Moore <saimon@webtypes.com> require 'fileutils' RAKEFILES = ['rakefile', 'Rakefile', 'rakefile.rb', 'Rakefile.rb'] exit 0 unless RAKEFILES.any? { |rf| File.file?(File.join(Dir.pwd, rf)) } exit 0 unless /^rake\b/ =~ ENV["COMP_LINE"] after_match = $' task_match = (after_match.empty? || after_match =~ /\s$/) ? nil : after_match.split.last cache_dir = File.join( ENV['HOME'], '.rake', 'tc_cache' ) FileUtils.mkdir_p cache_dir rakefile = RAKEFILES.detect { |rf| File.file?(File.join(Dir.pwd, rf)) } rakefile_path = File.join( Dir.pwd, rakefile ) cache_file = File.join( cache_dir, rakefile_path.gsub( %r{/}, '_' ) ) if File.exist?( cache_file ) && File.mtime( cache_file ) >= (Dir['lib/tasks/*.rake'] << rakefile).collect {|x| File.mtime(x) }.max task_lines = File.read( cache_file ) else task_lines = `rake --silent --tasks` File.open( cache_file, 'w' ) do |f| f << task_lines; end end tasks = task_lines.split("\n")[1..-1].collect {|line| line.split[1]} tasks = tasks.select {|t| /^#{Regexp.escape task_match}/ =~ t} if task_match # handle namespaces if task_match =~ /^([-\w:]+:)/ upto_last_colon = $1 after_match = $' tasks = tasks.collect { |t| (t =~ /^#{Regexp.escape upto_last_colon}([-\w:]+)$/) ? "#{$1}" : t } end puts tasks exit 0 |
Packaging with Rake
Automated the packaging process for winchester this morning use rake, the ruby build system. A few hurdles to jump, but I can now package up a release on either linux or windows with one line.
First trick was to determine the output executable of rubyscript2exe, since I couldn’t find a way to configure it, and also the desired extension for the platform:
1 2 3 4 5 6 7 8 9 10 |
if RUBY_PLATFORM =~ /linux/ insuffix = '_linux' outsuffix = '' elsif RUBY_PLATFORM =~ /mswin32/ insuffix = '.exe' outsuffix = '.exe' else puts 'Unsupported platform!' exit end |
I decided to get fancy and automagically determine the release suffix based on the current directory (trunk, dev-r1). This can be overriden by an environment variable. I’d like to add some special processing here so trunk builds also get the subversion revision number attached to them.
1 2 3 4 5 6 7 8 |
class String def tail key i = self.reverse.index(key) return nil if i == nil return self[-1 * i, self.length - i] end end release_suffix = ENV["RELEASE_SUFFIX"] ? ENV["RELEASE_SUFFIX"] : '-' + Dir.getwd.tail('/') |
And finally I used the ruby-zip package to create a zip file, in the process adding a convenient ‘add_dir’ method to ZipFile to recurse a directory and add the contents.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
require 'zip/zip' module Zip class ZipFile def add_dir entry, src self.mkdir(entry) Dir.foreach(src) do |fn| if fn[0] != '.'[0] if File.directory?(src + fn) self.add_dir(entry + '/' + fn, src + fn + '/') else self.add(entry + '/' + fn, src + fn) end end end end end end Zip::ZipFile.open('build/' + app_name + release_suffix + '.zip', Zip::ZipFile::CREATE) do |zf| zf.add(app_name + outsuffix, 'build/tmp/' + app_name + outsuffix) zf.add_dir('res', 'build/tmp/res/') end |