Deploying A Thor Application With Vlad, From Github, Run As A Cron Job

The previous 4 blog posts, in combination with a few others, have basically been a series of posts all leading up to this one. If you read them all in the right order (and possibly a few of the links in each), you should be able to step away from your browser / rss reader and build a complete command line ruby application that is executed through a cron job, and deployed to your various servers and environments.

This final post in the somewhat-series of posts will cover the deployment topic and what I had to do to get the whenever gem to play nice with my thor script, configure each environment’s settings for the app, and push it all to the server with vlad.

But first, the index of posts in the order I think you should read them:

1. Getting Started With Thor
2. Writing A Thor Application
3. The Whenever Gem: Making Cron Easy
4. Cleaning Up Log Files On Linux, With Logrotate
5. Using Mongo And Mongoid Without Rails
6. Solving Some SSH Issues For Deploying Rails Apps
7. Joey Beninghove’s Vlad, RVM and Bundler sittin’ in a tree
8. … this one

How To Use RVM And Bundler With Vlad The Deployer

I had written half of this information as another blog post at the start of this series. In the middle of that effort, though, I did a google search to find a blog post describing how to trust an rvmrc file with no user interaction. Well, googling “rvm trust rvmc” happened to turn up Joey Beninghove’s blog post on using Vlad, Bundler and RVM to deploy sinatra applications – nearly exactly what I was writing, minus the sinatra portion. Rather than duplicate what he’s already said so well, I would recommend that you start there. It’s a great post.

I’ll try not to repeat too much of what Joey has said already. I’m deploying a thor application that needs to be run from a cron job, instead of a web app, as well. There are some specific things I needed to do to get this work correctly, regarding thor and cron. I also want to talk about some of the general vlad and deployment related topics.

Release Folders And Symlinks For Fast App Version Switching

There are several deployment systems out there that make use of release folders and symlinks to provide fast app version switching, including roll back to previous versions. Both Vlad and Capistrano come to mind, initially, but I’ve also worked with .NET teams that did this, including my previous job at TrackAbout.

The idea is simple. Every time you do a release of your web app, you create a new folder for that release, on the server. You can spend all day uploading files and setting permissions and getting everything ready to go without affecting the production code. Once the new folder structure is ready to go, you set a symlink (a Symbolic Link) that points the web server to the correct version of the application. (Yes, this is even supported on Windows – just google “windows symlinks”).

To keep everything organized, you typically need a project home on the server – a folder that will contain all releases, the symlink to the current release, and any support files that the application needs. Vlad will handle the release folders and symlink for you, but you still need to create the home folder, first. You’ll want to configure vlad to point to your app’s home folder, as well. This way vlad does it’s ssh into the server, it will know where to go and do the deploy, etc.

Even though my app is not actually a web app, I still want to take advantage of this setup. If I happen to deploy any changes while the app is running – and that’s very likely, since I will be running the app on a very frequent schedule – then I won’t interrupt it with file changes that could possibly break the current execution. The already in-memory app will continue to run to completion and the next time it’s queued up, the symlink will have changed to point to the new version and the new version will be run.

Vlad-Git And Vlad Configuration For A Specific Environment

We’re using a privte Github repository to host the application, which makes it easy for us. We provided an ssh key to our github repository as a “deploy key”, which let’s that ssh key read the private repository but not update it.

We also need to use vlad-git to support this setup. This little extension to Vlad tells it to use Git as the source control system instead of the default (which I think is Subversion, but am not 100% certain of that). It’s easy to use, too. You only need to call the “vlad:update” rake task after setting the repository location.

Here’s an example of our vlad configuration, housed in our ‘deploy.rb’ files for each environment folder that we have (you can see the folder structure in the Writing A Thor Application blog post).

set :application, "app_name"
set :repository, "[email protected]:username/repositoryname.git"
set :domain, "dev.example.com"
set :deploy_to, "/home/app_staging_account/app_name"

Vlad and Vlad-Git know how to read these configuration settings and use them to do the deploy. We have multiple copies of this settings file, too – one for each environment that we are deploying to. This lets us change the configuration to point to the correct server and folder locations, as well as any other changes we might want to make for vlad’s configuration. Each of the configuration files for the environments is housed in a ./config/environmentname folder and the proper configuration is included via the rake task that we call to do the deploy.

Organizing Vlad Tasks And Rake Tasks

In addition to the environment specific configuration, we have a generic deploy.rb file in our app that sits in the ./config/ folder, directly. This file contains all of the actual vlad task definitions. We include this file in our rakefile with a ‘require config/deploy’ line. We don’t provide a desc (description) for any of the vlad tasks directly, so they are not advertised when you call rake -T from the command line. Instead, we provide a “vlad:deploy” task in the rakefile, which knows how to handle the environment targeting.

Here’s the rake task that I created to do kick off the deploy:

require 'vlad/core'
require 'vlad/git'
require "./config/deploy"

namespace :vlad do
  desc "Deploy and start the import job"
  task :deploy, [:environment] do |t, args|
    args.with_defaults :environment => :staging
    require "./config/#{args.environment}/deploy"
    Rake::Task["bioref:deploy"].invoke(args.environment)
  end
end

Note the use of the :environment task argument and the args variable in the code block definition. This let’s us call the deploy task with an environment specified. I’ve also defaulted the argument to “staging” so that we can run the task without an environment specified and it will deploy to staging.

This task is also responsible for requiring the appropriate environment configuration. The require statement in the task uses the environment argument to include the correct deploy configuration. It them calls the “bioref:deploy” task, which we have defined in our ./config/deploy.rb file. This task makes use of the configuration values that were set in the environment specific configuration.

Our deploy.rb file is set up with a namespace and various tasks, to organize it better. It also calls the vlad:update task that vlad includes, to update our source code on the server.

namespace :bioref do
  remote_task :bundle do
    # run bundle install with explicit path and without test dependencies
    run "#{goto_app_root} && bundle install $BUNDLE_PATH --without development test"
  end

  remote_task :clear_cron do
    run "#{goto_app_root} && whenever -c"
  end

  remote_task :set_cron do
    run "#{goto_app_root} && whenever -w"
  end

  remote_task :copy_cron, [:env] do |t, args|
    run "#{goto_app_root} && cp config/#{args.env}/schedule.rb config/"
  end

  task :deploy, [:env] do |t, args|
    Rake::Task['vlad:update'].invoke
    Rake::Task['bioref:bundle'].invoke
    Rake::Task['bioref:copy_cron'].invoke(args.env)
    Rake::Task['bioref:clear_cron'].invoke
    Rake::Task['bioref:set_cron'].invoke
  end
end

def goto_app_root
    # loads RVM, which initializes environment and paths
    init_rvm_cmd = "source /usr/local/rvm/scripts/rvm"

    # automatically trust the gemset in the .rvmrc file
    trust_rvm = "rvm rvmrc trust #{release_path}"

    # ya know, get to where we need to go
    # ex. /var/www/my_app/releases/12345
    "#{init_rvm_cmd} && #{trust_rvm} && cd #{release_path}"
end

Most of this is just re-posting what Joey already showed and explained in his post. At a high level, though, the :deploy task will ssh into the server, update the source code from github, run bundler against the code that was pulled down, copy the environment specific schedule.rb (which I’ll explain next), clear any old cron jobs using the whenever gem, and then create the cron jobs specified in the schedule.rb file using the whenever gem again.

(I’m sure there are ways that we can improve this file, too. If you have any suggestions, let me know. 🙂

Handling Whenever / Cron On The Remote Machine

This part of the process was a little tricky to get right. I went through several iterations of my whenever script before it worked. I need to have the whenever script called through ssh, on the remote machine. I also need it to set up a cron job that will call the correct version of the script – the one in the release folder that represents the current release.

From my previous post on writing a thor application, I have a ./bioref executable script set up and I decided it would be easier for me to call this than to call the thor tool directly. I don’t really have any reasoning for saying that, but it’s what I decided and I stuck with it. To make sure that I am calling the correct ./bioref script, I needed to get the full path of the executable when the whenever script was called. This turned out to be easy with the __FILE__ macro and File.expand_path. Here’s what my final schedule.rb, from the ./config/staging/ folder looks like:

dir = File.expand_path(File.dirname(__FILE__))

set :output, {:standard => '/var/log/bioref.import.log', :error => '/var/log/bioref.import.errors.log'}

every 1.minutes do
  command "cd #{dir} && ./bioref import /home/bioref/labs/ -d app_staging -k"
end