Having a process that automatically monitors your system and restarts applications when things get out of wack is pretty important to have. The 2 popular free options are God and Monit. This article is about not only setting up God to monitor your mongrels, but setting up God only ONCE to do this. If you add another mongrel cluster to your system, all you have to do is restart God.

Here is a sample god configuration file for monitoring all the mongrels on your system. This script assumes that your mongrel configs are all in /etc/mongrel_cluster and that your pid_file attribute in your config files point to an absolute path. I lifted the majority of this script from the example in thin. (Thin is awesome FYI)

I have a few mongrel config files in /etc/mongrel_cluster that look similar to:

--- 
user: deploy
group: www-data
cwd: /var/www/apps/sampleapp/current
port: "8200" 
environment: production
address: 127.0.0.1
pid_file: /var/www/apps/sampleapp/shared/pids/mongrel.pid
servers: 2

When I want to deploy another application on the same machine, I add the config file there and then restart god. It will then begin automatically monitoring the new application.

My /etc/god.conf:

require 'yaml'
config_path = "/etc/mongrel_cluster" 

Dir[config_path + "/*.yml"].each do |file|
  config = YAML.load_file(file)
  num_servers = config["servers"] ||= 1

  (0..num_servers-1).each do |i|
    number = config['port'].to_i + i

    God.watch do |w|
      w.group = "mongrel-" + File.basename(file, ".yml")
      w.name = w.group + "-#{number}" 
      w.interval = 30.seconds

      w.uid = config["user"]
      w.gid = config["group"]

      w.start = "mongrel_rails cluster::start -C #{file} --only #{number}" 
      w.start_grace = 10.seconds
      w.stop = "mongrel_rails cluster::stop -C #{file} --only #{number}" 
      w.stop_grace = 10.seconds
      w.restart = "mongrel_rails cluster::restart -C #{file} --only #{number}" 

      # assemble the pid file, pid files look like mongrel.8000.pid, mongrel.8001.pid etc
      pid_path = config["pid_file"]
      ext = File.extname(pid_path)
      w.pid_file = pid_path.gsub(/#{ext}$/, ".#{number}#{ext}")
      w.behavior(:clean_pid_file)

      w.start_if do |start|
        start.condition(:process_running) do |c|
          c.interval = 5.seconds
          c.running = false
          c.notify = 'team'
        end
      end

      w.restart_if do |restart|
        restart.condition(:memory_usage) do |c|
          c.above = 150.megabytes
          c.times = [3, 5] # 3 out of 5 intervals
          c.notify = 'team'
        end

        restart.condition(:cpu_usage) do |c|
          c.above = 50.percent
          c.times = 5
          c.notify = 'team'
        end
      end

      # lifecycle
      w.lifecycle do |on|
        on.condition(:flapping) do |c|
          c.to_state = [:start, :restart]
          c.times = 5
          c.within = 5.minute
          c.transition = :unmonitored
          c.retry_in = 10.minutes
          c.retry_times = 5
          c.retry_within = 2.hours
          c.notify = 'team'
        end
      end
    end

  end
end

God::Contacts::Email.message_settings = {
  :from => 'god@example.com'
}

God::Contacts::Email.server_settings = {
  :address => "localhost",
  :port => 25,
  :domain => "example.com" 
}

God.contact(:email) do |c|
  c.name = 'team'
  c.email = 'team@example.com'
end

One thing that the god install doesn’t do is create a nice init.d script, but you can find one here which I found from this post about using God.

The God website has an example with explanations of what each of those sections mean. Check it out.