Customizing our Cookbooks

Now that we have a working structure to load and run cookbooks, we need to start customizing them to make the servers ready to receive our application deployments. This includes setting up up accounts, passwords, firewall rules, and directories.

MySQL Password/Binding Address
Create DB User/Database
Enable MySQL Port in Firewall
Enable Apache Port in Firewall
Create Website Directories
Create Website Conf File
Create Website Database Config
Commit Our Work

Set MySQL Root Password and Binding Address

First, open up the following file in your mysql cookbook: chef/cookbooks/mysql/attributes/server.rb

1 default['mysql']['bind_address'] = attribute?('cloud') ? cloud['local_ipv4'] : ipaddress
2 default['mysql']['port']         = 3306

Chef cookbooks and recipes are built around the concept of setting attributes during the installation. (See the Chef Attributes wiki page for more.). These two lines towards the top of the file tell the server recipe that unless we specify otherwise, bund to the defined ipaddress on port 3306.

Now open up the mysql server recipe: chef/cookbooks/mysql/recipes/server.rb

1 node.set_unless['mysql']['server_root_password']   = secure_password

Somewhere towards the top you’ll see the line above. This tells the recipe that if we haven’t specified a root password, randomly generate one.

So, how do we tell the recipes what root password we want to use and what ip address to bind to? Why, the Vagrantfile of course!. Open instances/cluster/Vagrantfile and add the following code to the db instance chef solo config block:

 1 db_config.vm.provision :chef_solo do |chef|
 2   chef.cookbooks_path = "../../chef/cookbooks"
 3   chef.roles_path     = "../../chef/roles"
 4   chef.add_role "db"
 5 
 6   chef.json = {
 7     :mysql => {
 8       :server_root_password => "abcd1234",
 9       :bind_address => "10.10.10.11"
10     }
11   }
12 end

)
bq. From the top: After add_role, we set the json property of the chef object with a hash of attributes to pass into Chef Solo when the recipes are run. In this case, we’re setting the root password to abcd1234 and binding mysql to the 10.10.10.11 address.

Now, let’s re provision our db server instance and check our new settings:

1 $ cd instances/cluster
2 $ vagrant up

Once vagrant up completes and chef solo has completed, let’s login and check our handywork:

 1 $ vagrant ssh db
 2 Last login: Sun Sep  2 18:51:55 2012 from 10.0.2.2
 3 
 4 [vagrant@db ~]$ grep bind /etc/my.cnf
 5 bind-address            = 10.10.10.11
 6 
 7 [vagrant@db ~]$ mysql -u root -p
 8 Enter password: abcd1234
 9 
10 Welcome to the MySQL monitor.  Commands end with ; or \g.
11 Your MySQL connection id is 5
12 Server version: 5.0.95-log Source distribution
13 
14 Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
15 
16 Oracle is a registered trademark of Oracle Corporation and/or its
17 affiliates. Other names may be trademarks of their respective
18 owners.
19 
20 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
21 
22 mysql> exit
23 
24 exit

Success! Got now, go ahead and leave the db machine instance running this time.

Create a Database and User

To create a database and user for our application, we’re going to need to import another cookbook from github. Before we do that, we need to commit our work in git. While we’re at it, let’s ignore a file that vagrant uses to store instance state .vagrant:

1 $ cd ../..
2 $ echo ".vagrant" >> .gitignore
3 $ git add .
4 $ git commit -m "Added mysql password/binding. Ignore .vagrant file."

Now, let’s import the database cookbook, which knows how to create and maintain databases and database users using the knife cookbook github install command from before:

 1 $ knife cookbook github install opscode-cookbooks/database
 2 Installing database from git://github.com/opscode-cookbooks/database.git to /Users/claco/devops_toolbox/chef/cookbooks
 3 Checking out the master branch.
 4 Creating pristine copy branch chef-vendor-database
 5 Removing pre-existing version.
 6 Found master amoung heads.
 7 1 files updated, committing changes
 8 Creating tag cookbook-site-imported-database-549c441c0064b986b1fc369ed23a2c1fbe421ec3
 9 Checking out the master branch.
10 Updating e190921..dda093a
11 Fast-forward
12  chef/cookbooks/database/.gitignore                                             |   1 +
13  chef/cookbooks/database/CHANGELOG.md                                           |  46 +++++++++++++++++++
14  chef/cookbooks/database/CONTRIBUTING                                           |  29 ++++++++++++
15  chef/cookbooks/database/LICENSE                                                | 201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
16  chef/cookbooks/database/README.md                                              | 468 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
17  chef/cookbooks/database/libraries/provider_database_mysql.rb                   | 103 +++++++++++++++++++++++++++++++++++++++++
18  chef/cookbooks/database/libraries/provider_database_mysql_user.rb              |  76 ++++++++++++++++++++++++++++++
19  chef/cookbooks/database/libraries/provider_database_postgresql.rb              | 131 ++++++++++++++++++++++++++++++++++++++++++++++++++++
20  chef/cookbooks/database/libraries/provider_database_postgresql_user.rb         |  83 +++++++++++++++++++++++++++++++++
21  chef/cookbooks/database/libraries/provider_database_sql_server.rb              | 109 +++++++++++++++++++++++++++++++++++++++++++
22  chef/cookbooks/database/libraries/provider_database_sql_server_user.rb         | 106 ++++++++++++++++++++++++++++++++++++++++++
23  chef/cookbooks/database/libraries/resource_database.rb                         | 119 +++++++++++++++++++++++++++++++++++++++++++++++
24  chef/cookbooks/database/libraries/resource_database_user.rb                    |  90 ++++++++++++++++++++++++++++++++++++
25  chef/cookbooks/database/libraries/resource_mysql_database.rb                   |  34 ++++++++++++++
26  chef/cookbooks/database/libraries/resource_mysql_database_user.rb              |  34 ++++++++++++++
27  chef/cookbooks/database/libraries/resource_postgresql_database.rb              |  35 ++++++++++++++
28  chef/cookbooks/database/libraries/resource_postgresql_database_user.rb         |  35 ++++++++++++++
29  chef/cookbooks/database/libraries/resource_sql_server_database.rb              |  34 ++++++++++++++
30  chef/cookbooks/database/libraries/resource_sql_server_database_user.rb         |  34 ++++++++++++++
31  chef/cookbooks/database/metadata.rb                                            |  22 +++++++++
32  chef/cookbooks/database/recipes/default.rb                                     |  20 ++++++++
33  chef/cookbooks/database/recipes/ebs_backup.rb                                  |  89 +++++++++++++++++++++++++++++++++++
34  chef/cookbooks/database/recipes/ebs_volume.rb                                  | 204 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
35  chef/cookbooks/database/recipes/master.rb                                      |  78 +++++++++++++++++++++++++++++++
36  chef/cookbooks/database/recipes/mysql.rb                                       |  20 ++++++++
37  chef/cookbooks/database/recipes/postgresql.rb                                  |  20 ++++++++
38  chef/cookbooks/database/recipes/snapshot.rb                                    |  62 +++++++++++++++++++++++++
39  chef/cookbooks/database/templates/default/app_grants.sql.erb                   |   8 ++++
40  chef/cookbooks/database/templates/default/aws_config.erb                       |   3 ++
41  chef/cookbooks/database/templates/default/chef-solo-database-snapshot.cron.erb |   6 +++
42  chef/cookbooks/database/templates/default/chef-solo-database-snapshot.json.erb |   1 +
43  chef/cookbooks/database/templates/default/chef-solo-database-snapshot.rb.erb   |   6 +++
44  chef/cookbooks/database/templates/default/ebs-backup-cron.erb                  |   2 +
45  chef/cookbooks/database/templates/default/ebs-db-backup.sh.erb                 |   8 ++++
46  chef/cookbooks/database/templates/default/ebs-db-restore.sh.erb                |  10 ++++
47  chef/cookbooks/database/templates/default/s3cfg.erb                            |  27 +++++++++++
48  36 files changed, 2354 insertions(+)
49  create mode 100644 chef/cookbooks/database/.gitignore
50  create mode 100644 chef/cookbooks/database/CHANGELOG.md
51  create mode 100644 chef/cookbooks/database/CONTRIBUTING
52  create mode 100644 chef/cookbooks/database/LICENSE
53  create mode 100644 chef/cookbooks/database/README.md
54  create mode 100644 chef/cookbooks/database/libraries/provider_database_mysql.rb
55  create mode 100644 chef/cookbooks/database/libraries/provider_database_mysql_user.rb
56  create mode 100644 chef/cookbooks/database/libraries/provider_database_postgresql.rb
57  create mode 100644 chef/cookbooks/database/libraries/provider_database_postgresql_user.rb
58  create mode 100644 chef/cookbooks/database/libraries/provider_database_sql_server.rb
59  create mode 100644 chef/cookbooks/database/libraries/provider_database_sql_server_user.rb
60  create mode 100644 chef/cookbooks/database/libraries/resource_database.rb
61  create mode 100644 chef/cookbooks/database/libraries/resource_database_user.rb
62  create mode 100644 chef/cookbooks/database/libraries/resource_mysql_database.rb
63  create mode 100644 chef/cookbooks/database/libraries/resource_mysql_database_user.rb
64  create mode 100644 chef/cookbooks/database/libraries/resource_postgresql_database.rb
65  create mode 100644 chef/cookbooks/database/libraries/resource_postgresql_database_user.rb
66  create mode 100644 chef/cookbooks/database/libraries/resource_sql_server_database.rb
67  create mode 100644 chef/cookbooks/database/libraries/resource_sql_server_database_user.rb
68  create mode 100644 chef/cookbooks/database/metadata.rb
69  create mode 100644 chef/cookbooks/database/recipes/default.rb
70  create mode 100644 chef/cookbooks/database/recipes/ebs_backup.rb
71  create mode 100644 chef/cookbooks/database/recipes/ebs_volume.rb
72  create mode 100644 chef/cookbooks/database/recipes/master.rb
73  create mode 100644 chef/cookbooks/database/recipes/mysql.rb
74  create mode 100644 chef/cookbooks/database/recipes/postgresql.rb
75  create mode 100644 chef/cookbooks/database/recipes/snapshot.rb
76  create mode 100644 chef/cookbooks/database/templates/default/app_grants.sql.erb
77  create mode 100644 chef/cookbooks/database/templates/default/aws_config.erb
78  create mode 100644 chef/cookbooks/database/templates/default/chef-solo-database-snapshot.cron.erb
79  create mode 100644 chef/cookbooks/database/templates/default/chef-solo-database-snapshot.json.erb
80  create mode 100644 chef/cookbooks/database/templates/default/chef-solo-database-snapshot.rb.erb
81  create mode 100644 chef/cookbooks/database/templates/default/ebs-backup-cron.erb
82  create mode 100644 chef/cookbooks/database/templates/default/ebs-db-backup.sh.erb
83  create mode 100644 chef/cookbooks/database/templates/default/ebs-db-restore.sh.erb
84  create mode 100644 chef/cookbooks/database/templates/default/s3cfg.erb
85 Cookbook database version 549c441c0064b986b1fc369ed23a2c1fbe421ec3 successfully installed

Now this is where the fun begines We’re going to create a new cookbook for our application, and create a new db recipe to create the database and database user.

To create the cookbook, use the knife cookbook create command:

 1 $ knife cookbook create myapp
 2 ** Creating cookbook myapp
 3 ** Creating README for cookbook: myapp
 4 ** Creating metadata for cookbook: myapp
 5 
 6 $ tree chef/cookbooks/myapp
 7 chef/cookbooks/myapp
 8 ├── README.md
 9 ├── attributes
10 ├── definitions
11 ├── files
12 │   └── default
13 ├── libraries
14 ├── metadata.rb
15 ├── providers
16 ├── recipes
17 │   └── default.rb
18 ├── resources
19 └── templates
20     └── default
21 
22 10 directories, 3 files

From the top: After running the cookbook create command, a new chef/cookbooks/myapp folder is created. It contains the default structure for creating recipes including default attributes, metdata, and a README.

Now that we have a cookbook for our application, we need to add default attributes for the database/user we’re creating, and a recipe to use those attributes to actually create the database/user using the database cookbook we imported earlier. Since myapp is split into two server, the web role and the db role, we’re going to follow that pattern in the cookbook by creating a web recipe and a db recipe, each with their own attributes files.

First, create and open a new attributes file in chef/cookbooks/myapp/attributes/db.rb. In that file we’re going to set two attributes: the default database nane, and the default user name:

1 default['myapp']['database']['name'] = 'myapp'
2 default['myapp']['database']['user'] = 'myapp'

Next, let’s create the recipe itself. Create and open a new recipe file in chef/cookbooks/myapp/recipes/db.rb with this content:

 1 include_recipe "database::mysql"
 2 
 3 mysql_connection_info = {
 4   :host => "localhost",
 5   :username => 'root',
 6   :password => node['mysql']['server_root_password']
 7 }
 8 
 9 mysql_database node['myapp']['database']['name'] do
10   connection mysql_connection_info
11   action :create
12 end
13 
14 mysql_database_user node['myapp']['database']['user'] do
15   connection mysql_connection_info
16   password node['myapp']['database']['password']
17   database_name node['myapp']['database']['name']
18   host '%'
19   action :grant
20 end

From the top: Line 1, we include the database::mysql recipe, which contains methods that know how to manage mysql databases and users. Lines 3-7 we setup our current mysql auth information that the recipe will use to manage mysql, including using the root password we’ve already set earlier. Lines 9-12 create the database with the default name (myapp), and Lines 14-20 creates a database user and grants them access to the new database.

As you may have noticed, the one thing we didn’t declare a default attribute for is the database users password. So, let’s add that to the recipe settings in @Vagrantfile:

 1 db_config.vm.provision :chef_solo do |chef|
 2   chef.cookbooks_path = "../../chef/cookbooks"
 3   chef.roles_path     = "../../chef/roles"
 4   chef.add_role "db"
 5 
 6   chef.json = {
 7     :mysql => {
 8       :server_root_password => "abcd1234",
 9       :bind_address => "10.10.10.11"
10     },
11     :myapp => {
12       :database => {
13         :password => 'myapp_1234'
14       }
15     }
16 
17   }
18 end

Now that we have a db recipe ready to go, we just need to add it to our db role in chef/roles/db.rb:

1 name "db"
2 description "Database Server Role"
3 run_list(
4   "recipe[mysql::server]", \
5   "recipe[myapp::db]"
6 )

Now, let’s run chef solo again so it can create our database/user. Since our vagrant instance is still running, we can just use the provision command:

 1 $ cd instances/cluster
 2 $ vagrant provision db
 3 [db] Running provisioner: Vagrant::Provisioners::ChefSolo...
 4 [db] Generating chef JSON and uploading...
 5 [db] Running chef-solo...
 6 [2012-09-27T02:52:00+01:00] INFO: *** Chef 10.12.0 ***
 7 [2012-09-27T02:52:01+01:00] INFO: Setting the run_list to ["role[db]"] from JSON
 8 [2012-09-27T02:52:01+01:00] INFO: Run List is [role[db]]
 9 [2012-09-27T02:52:01+01:00] INFO: Run List expands to [mysql::server, myapp::db]
10 ...
11 [2012-09-27T02:53:33+01:00] INFO: Processing mysql_database[myapp] action create (myapp::db line 9)
12 [2012-09-27T02:53:33+01:00] INFO: Processing mysql_database_user[myapp] action grant (myapp::db line 14)
13 [2012-09-27T02:53:33+01:00] INFO: mysql_database_user[myapp]: granting access with statement [GRANT all ON myapp.* TO 'myapp'@'%' IDENTIFIED BY 'myapp_1234']

From the top: This time, line 9 has our myapp::db recipe in the run list, via the db role, and towards the end you can see the myapp database being created, the myapp user being created, and the user being granted access to the new database.

Let’s log into the db instace just to make sure things worked properly:

 1 $ vagrant ssh db
 2 Last login: Thu Sep 27 02:57:26 2012 from 10.0.2.2
 3 
 4 [vagrant@db ~]$ mysql -u myapp -p -D myapp -h 10.10.10.11
 5 Enter password: myapp_1234
 6 Welcome to the MySQL monitor.  Commands end with ; or \g.
 7 Your MySQL connection id is 14
 8 Server version: 5.0.95-log Source distribution
 9 
10 Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
11 
12 Oracle is a registered trademark of Oracle Corporation and/or its
13 affiliates. Other names may be trademarks of their respective
14 owners.
15 
16 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
17 
18 mysql> exit
19 $ exit

Fantastic!

We’re going to be importing a few more cookbooks, so let’s save our work:

$ cd ../..
$ git add .
$ git commit -m "Added myapp cookbook. Added db recipe. Updated db role."
[master e7f4f0c] Added myapp cookbook. Added db recipe. Updated db role.
 7 files changed, 55 insertions(+), 1 deletion(-)
 create mode 100644 chef/cookbooks/myapp/README.md
 create mode 100644 chef/cookbooks/myapp/attributes/db.rb
 create mode 100644 chef/cookbooks/myapp/metadata.rb
 create mode 100644 chef/cookbooks/myapp/recipes/db.rb
 create mode 100644 chef/cookbooks/myapp/recipes/default.rb

Enable MySQL Port in the Firewall

Now that we have configured database, we still can’t talk to it from other machines, namely the web server we’re going to be creating. Luckily for us, there’s an iptables cookbook that knows how to add/update firewall rules. Unfortunately for me, and fortunately for you, the first time I applied rules using the iptables cookbook, it added the mysql port, but also disabled the ssh port. This was because we were now loading iptable rules explicitely and the default ssh port that was enabled by KickStart OS install no longer applies. So, with that said, we will also import the openssh cookbook so we can encorporate its attributes and settings into the firewall rules.

Also, if you look around the apache/mysql/openssh cookbooks, you’ll find port_*.erb files. These are the template files the iptables cookbook uses to aply firewall rules.

First, inport both cookbooks:

knife cookbook github install opscode-cookbooks/iptables
knife cookbook github install opscode-cookbooks/openssh

Next, let’s call iptables and run the appropriate firewall rules for mysql (and ssh!) by adding the following to your myapp db recipe.

 1 include_recipe "database::mysql"
 2 include_recipe "iptables"
 3 
 4 iptables_rule "port_mysql" do
 5   cookbook "mysql"
 6 end
 7  
 8 iptables_rule "port_ssh" do
 9   cookbook "openssh"
10 end
11 
12 mysql_connection_info = {
13   :host => "localhost",
14   :username => 'root',
15   :password => node['mysql']['server_root_password']
16 }
17  
18 mysql_database node['myapp']['database']['name'] do
19   connection mysql_connection_info
20   action :create
21 end
22 
23 mysql_database_user node['myapp']['database']['user'] do
24   connection mysql_connection_info
25   password node['myapp']['database']['password']
26   database_name node['myapp']['database']['name']
27   host '%'
28   action :grant
29 end

From the top: On line 2, we include the iptables recipe. Lines 4-10 call the iptables rule helper, asking it to load the port_* rules from their respective cookbooks; in this case port_mysql from the mysql coobook and port_ssh from the openssh recipe.

Food for thought: This is the easiest way to open the ports in the firewall, but one could arue these decisions belong in the mysql::server recipe, or maybe in a new custom mysql::iptables recipe.

Lastly, let’s run chef solo again on the db instance to setup the firewall rules:

 1 $ vagrant provision db
 2 [db] Running provisioner: Vagrant::Provisioners::ChefSolo...
 3 [db] Generating chef JSON and uploading...
 4 [db] Running chef-solo...
 5 ...
 6 [2012-09-27T04:00:40+01:00] INFO: Processing template[/etc/iptables.d/port_mysql] action create (myapp::db line 23)
 7 [2012-09-27T04:00:40+01:00] INFO: template[/etc/iptables.d/port_mysql] mode changed to 644
 8 [2012-09-27T04:00:40+01:00] INFO: template[/etc/iptables.d/port_mysql] updated content
 9 [2012-09-27T04:00:40+01:00] INFO: template[/etc/iptables.d/port_mysql] not queuing delayed action run on execute[rebuild-iptables] (delayed), as it's already been queued
10 [2012-09-27T04:00:40+01:00] INFO: Processing template[/etc/iptables.d/port_ssh] action create (myapp::db line 23)

From the top: After running chef solo again, among the output you’ll see it now adding the mysql and ssh rules.

Just for giggles, we should be able to now connect directly to mysql using the external ip address (10.10.10.11) locally, as well as the web instance:

1 $ mysql -u myapp -p -D myapp -h 10.10.10.11
2 Enter password: myapp_1234
3 
4 $ vagrant ssh web
5 Last login: Sun Sep  2 18:51:55 2012 from 10.0.2.2
6 
7 [vagrant@web ~]$ mysql -u myapp -p -D myapp -h 10.10.10.11
8 Enter password: myapp_1234
9 Welcome to the MySQL monitor.  Commands end with ; or \g.

Enable Apache Port in the Firewall

Since we’re on a role with firewall rules, let’s move over to the web machien instance and configure the firewall to allow port 80. To do that, we need to create a new recipe our application in the web role. Create and open a new recipe in chef/cookbooks/myapp/recipes/web.db. In it, load the iptables cookbook and call the port_apache rule from the apache2 cookbook:

Remember, since we’re setting up custom firewall rules, we need to remember to include the ssh port too!

 1 include_recipe "iptables"
 2 
 3 iptables_rule "port_apache" do
 4   cookbook  "apache2"
 5   variables :port => 80
 6 end
 7 
 8 iptables_rule "port_ssh" do
 9   cookbook "openssh"
10 end

From the top: Just as with the db recipe, we include the iptables cookbook. Next we call the port_apache rule from the apache2 cookbook and give it a port [as a template variable]. Then we call the port_ssh rule from the openssh cookbook.

So why the “variables” thingy? The port_apache file uses a “template variable” to assign the port instead of hard coding the port in the file, like port_ssh and port_mysql do:

# Port <%= @port %> 
-A FWR -p tcp -m tcp --dport <%= @port %> -j ACCEPT

Next, let’s add this recipe to our web role. Open chef/roles/web.rb and add the myapp::web recipe:

1 name "web"
2 description "Web Server Role"
3 run_list(
4   "recipe[apache2]", \
5   "recipe[passenger_apache2::mod_rails]", \
6   "recipe[mysql::client]", \
7   "recipe[myapp::web]"
8 )

Now, let’s provision our web instance with the firewall changes. If your instance is still running, use vagrant provision web, otherwise, run vagrant ip web:

 1 $ cd instances/cluster
 2 $ vagrant up web
 3 
 4 [web] Running provisioner: Vagrant::Provisioners::ChefSolo...
 5 [web] Generating chef JSON and uploading...
 6 [web] Running chef-solo...
 7 [2012-09-27T13:04:53+01:00] INFO: *** Chef 10.12.0 ***
 8 [2012-09-27T13:04:53+01:00] INFO: Setting the run_list to ["role[web]"] from JSON
 9 [2012-09-27T13:04:53+01:00] INFO: Run List is [role[web]]
10 [2012-09-27T13:04:53+01:00] INFO: Run List expands to [apache2, passenger_apache2::mod_rails, mysql::client, myapp::web]
11 ...
12 [2012-09-27T13:04:55+01:00] INFO: Processing template[/etc/iptables.d/port_apache] action create (myapp::web line 23)
13 [2012-09-27T13:04:55+01:00] INFO: Processing template[/etc/iptables.d/port_ssh] action create (myapp::web line 23)

From the top: Just like the db provisioning, the myapp::web is in the run list, and towards the end we see it adding rules to the firewall for ssh and apache.

Now, we should be able to see the default Apache page from the external ip address. In your browser, open http://10.10.10.10:80/

Create Webite Directories

Now we need to create all of the directories where we want out application to live and a user to own those directories.

First, like the db recipe, let’s set some default attributes for our web recipe. Create and open chef/cookbooks/myapp/attributes/web.rb and add the following:

1 default['myapp']['website']['approot'] = '/var/www/myapp'
2 default['myapp']['website']['docroot'] = '/var/www/myapp/current/public'
3 default['myapp']['website']['server_name'] = ipaddress
4 default['myapp']['website']['server_aliases'] = [ipaddress]
5 default['myapp']['website']['rails_env'] = 'development'

From the top: By default, we set the approot to /var/www/myapp, the docroot to the current/public folder in docroot, set the server_name/aliases to empty, and the rails environment to development. These will be used to create the directories as well as configure apache/passenger for the rails applicaiton we’ll be deploying later.

Now, open chef/cookbooks/myapp/recipes/web.rb and add the following:

 1 include_recipe "apache2"
 2 include_recipe "passenger_apache2"
 3 
 4 approot = node['myapp']['website']['approot'] 
 5 
 6 group "myapp" do
 7   gid 1000
 8 end
 9 
10 user "myapp" do
11   comment "MyApp User"
12   uid 1000
13   gid "myapp"
14   home "/home/myapp"
15   shell "/bin/bash"
16   password "$1$wp3nGnOs$NYH9NB05fC504YH2UsAW61"
17 end
18 
19 directory approot do
20   owner "myapp"
21   group "myapp"
22   mode "0755"
23   recursive true
24 end
25 
26 directory "#{approot}/shared" do
27   owner "myapp"
28   group "myapp"
29   mode "0755"
30 end
31 
32 directory "#{approot}/releases" do
33   owner "myapp"
34   group "myapp"
35   mode "0755"
36 end
37 
38 directory "#{approot}/releases/20110101010101" do
39   owner "myapp"
40   group "myapp"
41   mode "0755"
42 end
43 
44 directory "#{approot}/releases/20110101010101/public" do
45   owner "myapp"
46   group "myapp"
47   mode "0755"
48 end
49 
50 directory "#{approot}/releases/20110101010101/public/system" do
51   owner "myapp"
52   group "myapp"
53   mode "0755"
54 end
55 
56 execute "maintenance.html" do
57   command "echo \"System Maintenance In Progress\" > #{approot}/releases/20110101010101/public/system/maintenance.html"
58 end
59 
60 link "#{approot}/current" do
61   to "#{approot}/releases/20110101010101"
62   owner "myapp"
63   group "myapp"
64   not_if do
65     File.exists?("#{approot}/current")
66   end
67 end
68 
69 directory "#{approot}/current/public" do
70   owner "myapp"
71   group "myapp"
72   mode "0755"
73 end

That’s a pretty big change, so let’s break it down a bit.

Line 1-2 we’re including the apache2/passenger recipes. We’ll be using those in a little bit.
Lines 6-16, we’re create a user and group called myapp with the shadow password “myapp_1234”
Lines 13-54, we’re creating a capistrano style directory layout. We’ll be using Capstrano later to deploy our sample Rails application.
Line 56-48, we’re creating a “System Maintenance In Progress” page to display by default until we deploy the application.
Lines 60-73, we create a symlink for the current directory and create a public folder.

Let’s reprovision our web instance again and verify the changes:

 1 $ cd instances/cluster
 2 $ vagrant provision web
 3 ...
 4 [2012-09-27T13:54:24+01:00] INFO: Processing user[myapp] action create (myapp::web line 20)
 5 [2012-09-27T13:54:24+01:00] INFO: user[myapp] created
 6 [2012-09-27T13:54:24+01:00] INFO: Processing directory[/var/www/myapp] action create (myapp::web line 29)
 7 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp] created directory /var/www/myapp
 8 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp] owner changed to 1000
 9 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp] group changed to 1000
10 [2012-09-27T13:54:24+01:00] INFO: Processing directory[/var/www/myapp/shared] action create (myapp::web line 36)
11 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/shared] created directory /var/www/myapp/shared
12 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/shared] owner changed to 1000
13 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/shared] group changed to 1000
14 [2012-09-27T13:54:24+01:00] INFO: Processing directory[/var/www/myapp/releases] action create (myapp::web line 42)
15 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases] created directory /var/www/myapp/releases
16 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases] owner changed to 1000
17 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases] group changed to 1000
18 [2012-09-27T13:54:24+01:00] INFO: Processing directory[/var/www/myapp/releases/20110101010101] action create (myapp::web line 48)
19 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101] created directory /var/www/myapp/releases/20110101010101
20 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101] owner changed to 1000
21 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101] group changed to 1000
22 [2012-09-27T13:54:24+01:00] INFO: Processing directory[/var/www/myapp/releases/20110101010101/public] action create (myapp::web line 54)
23 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101/public] created directory /var/www/myapp/releases/20110101010101/public
24 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101/public] owner changed to 1000
25 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101/public] group changed to 1000
26 [2012-09-27T13:54:24+01:00] INFO: Processing directory[/var/www/myapp/releases/20110101010101/public/system] action create (myapp::web line 60)
27 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101/public/system] created directory /var/www/myapp/releases/20110101010101/public/system
28 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101/public/system] owner changed to 1000
29 [2012-09-27T13:54:24+01:00] INFO: directory[/var/www/myapp/releases/20110101010101/public/system] group changed to 1000
30 [2012-09-27T13:54:24+01:00] INFO: Processing execute[maintenance.html] action run (myapp::web line 66)
31 [2012-09-27T13:54:24+01:00] INFO: execute[maintenance.html] ran successfully
32 [2012-09-27T13:54:24+01:00] INFO: Processing link[/var/www/myapp/current] action create (myapp::web line 70)
33 [2012-09-27T13:54:24+01:00] INFO: link[/var/www/myapp/current] created
34 [2012-09-27T13:54:24+01:00] INFO: link[/var/www/myapp/current] owner changed to 1000
35 [2012-09-27T13:54:24+01:00] INFO: link[/var/www/myapp/current] group changed to 1000
36 ...

From the top: Now when chef solo is run, you should now see a bunch of lines about creating users, group, and all of the directories.

Now, let’s login to the web instance and check things out:

 1 $ vagrant ssh web
 2 Last login: Thu Sep 27 13:58:48 2012 from 10.0.2.2
 3 
 4 [vagrant@web ~]$ su myapp
 5 Password: myapp_1234
 6 
 7 [myapp@web vagrant]$ cd /var/www
 8 
 9 [myapp@web www]$ ls -lta
10 total 32
11 drwxr-xr-x  7 root  root  4096 Sep 27 13:54 .
12 drwxr-xr-x  4 myapp myapp 4096 Sep 27 13:54 myapp
13 drwxr-xr-x 21 root  root  4096 Sep 27 01:34 ..
14 drwxr-xr-x  3 root  root  4096 Sep 27 01:34 error
15 drwxr-xr-x  3 root  root  4096 Sep 27 01:34 icons
16 drwxr-xr-x  2 root  root  4096 Jun  6 15:04 cgi-bin
17 drwxr-xr-x  2 root  root  4096 Jun  6 15:04 html
18 
19 [myapp@web www]$ tree myapp
20 myapp
21 |-- current -> /var/www/myapp/releases/20110101010101
22 |-- releases
23 |   `-- 20110101010101
24 |       `-- public
25 |           `-- system
26 |               `-- maintenance.html
27 `-- shared
28 
29 6 directories, 1 file

Create Website Httpd Conf File

Now that we have a website directory structure and a “System Maintenance In Progress” page, let’s configure Apache to server up this website instead of the default Apache website.

Add the following to the bottom of your chef/cookbooks/myapp/recipes/web.db:

1 web_app "myapp" do
2   cookbook "passenger_apache2"
3   docroot "#{approot}/current/public"
4   server_name node['myapp']['website']['server_name']
5   server_aliases node['myapp']['website']['server_aliases']
6   rails_env node['myapp']['website']['rails_env']
7 end

From the top: We’re using the web_app helper from the Apache cookbook to creete a “debian style” enabled site configuration with the defaults we set in chef/cookbook/myapp/attributes/web.rb

Now, open your instances/cluster/Vagrantfile, and add the following to the web instance configuration:

 1 web_config.vm.provision :chef_solo do |chef|
 2   chef.cookbooks_path = "../../chef/cookbooks"
 3   chef.roles_path     = "../../chef/roles"
 4   chef.add_role "web"
 5 
 6   chef.json = {
 7     :myapp => {
 8       :website => {
 9         :server_name => "10.10.10.10",
10         :server_aliases => ["10.10.10.10"],
11       }
12     }
13   }
14 end

From the top: Just like we added attribute settings to the db instance configuration, here we’re telling the web instance recipes what server name/aliases to send to the apache conf file we’re creating.

LEt’s once again, reprovision the web instance:

 1 $ vagrant provision web
 2 ...
 3 [2012-09-27T14:23:11+01:00] INFO: Processing template[/etc/httpd/sites-available/myapp.conf] action create (myapp::web line 29)
 4 [2012-09-27T14:23:11+01:00] INFO: template[/etc/httpd/sites-available/myapp.conf] mode changed to 644
 5 [2012-09-27T14:23:11+01:00] INFO: template[/etc/httpd/sites-available/myapp.conf] updated content
 6 [2012-09-27T14:23:11+01:00] INFO: Processing execute[a2ensite myapp.conf] action run (myapp::web line 24)
 7 [2012-09-27T14:23:11+01:00] INFO: execute[a2ensite myapp.conf] ran successfully
 8 [2012-09-27T14:23:11+01:00] INFO: execute[a2ensite myapp.conf] not queuing delayed action restart on service[apache2] (delayed), as it's already been queued
 9 [2012-09-27T14:23:11+01:00] INFO: execute[a2enmod rewrite] sending restart action to service[apache2] (delayed)
10 [2012-09-27T14:23:11+01:00] INFO: Processing service[apache2] action restart (apache2::default line 219)
11 [2012-09-27T14:23:12+01:00] INFO: service[apache2] restarted
12 ...

From the top: This time, chef solo created our myapp.conf file in the sites-available directory structure and reloaded apache.

Let’s login and check out that config file:

 1 $ vagrant ssh web
 2 Last login: Thu Sep 27 14:03:25 2012 from 10.0.2.2
 3 
 4 [vagrant@web ~]$ cat /etc/httpd/sites-enabled/myapp.conf
 5 <VirtualHost *:80>
 6   ServerName 10.10.10.10
 7   ServerAlias 10.10.10.10 
 8   DocumentRoot /var/www/myapp/current/public
 9 
10   RailsBaseURI /
11   RailsEnv development
12 
13   <Directory /var/www/myapp/current/public>
14     Options FollowSymLinks
15     AllowOverride None
16     Order allow,deny
17     Allow from all
18   </Directory>
19 
20   LogLevel info
21   ErrorLog /var/log/httpd/myapp-error.log
22   CustomLog /var/log/httpd/myapp-access.log combined
23 
24   RewriteEngine On
25   RewriteLog /var/log/httpd/myapp-rewrite.log
26   RewriteLogLevel 0
27   # Canonical host
28   RewriteCond %{HTTP_HOST}   !^10.10.10.10 [NC]
29   RewriteCond %{HTTP_HOST}   !^$
30   RewriteRule ^/(.*)$        http://10.10.10.10/$1 [L,R=301]
31 
32   RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
33   RewriteCond %{SCRIPT_FILENAME} !maintenance.html
34   RewriteRule ^.*$ /system/maintenance.html [L]
35 
36 </VirtualHost>

From the top: Within the VirtualHost block in the conf file, you see our default rails environment, the docroot, the server name/aliases, and towards the end, the check for the system/maintenance.html page.

And now that we have told apache to load our website, let’s check it in a browser again. Open http://10.10.10.10:80/

Woohoo!

Create Website Database Config"

The last thing we need to do before we deploy our application is to have a database config file ready for Rails to use after deployment.

First, we need to create a template file. In your editor, create and open chef/cookbooks/myapp/templates/default/database.yml.erb with the folowing content:

1 <%=@rails_env%>:
2   adapter: mysql2
3   encoding: utf8
4   reconnect: true
5   database: <%=@database%>
6   pool: 5
7   username: <%=@username%>
8   password: <%=@password%>
9   host: <%=@host%>

Add the following to the bottom of your chef/cookbooks/myapp/recipes/web.db:

 1 template "#{approot}/shared/database.yml" do
 2   source "database.yml.erb"
 3   owner "myapp"
 4   group "myapp"
 5   mode "0755"
 6   variables(
 7     :database  => node['myapp']['database']['name'],
 8     :username  => node['myapp']['database']['user'],
 9     :password  => node['myapp']['database']['password'],
10     :host      => node['myapp']['database']['host'],
11     :rails_env => node['myapp']['website']['rails_env']
12   )
13 end
14 
15 gem_package "bundler" do
16   action :install
17 end

From the top: We’re using the template helper to turn an erb template into a real file in the file system. In this case, /var/www/myapp/shared/database.yml, with the specified database name, user, password, host, and rails environment. And just for giggles, let’s also ensure bundler is installed.

Lastly, before we reprovision the web instance to create this file, we need to add more attributes to Vagrantfile

 1 web_config.vm.provision :chef_solo do |chef|
 2   chef.cookbooks_path = "../../chef/cookbooks"
 3   chef.roles_path     = "../../chef/roles"
 4   chef.add_role "web"
 5 
 6   chef.json = {
 7     :myapp => {
 8       :database => {
 9         :name => "myapp",
10         :host => "10.10.10.11",
11         :user => "myapp",
12         :password => "myapp_1234"
13       },
14       :website => {
15         :server_name => "10.10.10.10",
16         :server_aliases => ["10.10.10.10"],
17         :rails_env => "development"
18       }
19     }
20   }
21 end

Now, once more, let’s reprovision the web instance:

1 $ cd instances/cluster
2 $ vagrant provision web
3 ...
4 [2012-09-27T14:46:22+01:00] INFO: Processing template[/var/www/myapp/shared/database.yml] action create (myapp::web line 93)
5 [2012-09-27T14:46:22+01:00] INFO: template[/var/www/myapp/shared/database.yml] owner changed to 1000
6 [2012-09-27T14:46:22+01:00] INFO: template[/var/www/myapp/shared/database.yml] group changed to 1000
7 [2012-09-27T14:46:22+01:00] INFO: template[/var/www/myapp/shared/database.yml] mode changed to 755
8 [2012-09-27T14:46:22+01:00] INFO: template[/var/www/myapp/shared/database.yml] updated content
9 ...

Let’s login to the web instance and verify that file:

 1 $ vagrant ssh web
 2 Last login: Thu Sep 27 14:25:17 2012 from 10.0.2.2
 3 
 4 [vagrant@web ~]$ cat /var/www/myapp/shared/database.yml
 5 development:
 6   adapter: mysql2
 7   encoding: utf8
 8   reconnect: true
 9   database: myapp
10   pool: 5
11   username: myapp
12   password: myapp_1234
13   host: 10.10.10.11

Awesome! We are now ready to create a simple Rails application and deploy it to our local vagrant instances.

Commit Our Work

Let’s commit our cookbook/recipe customizations to the git repository.

1 $ cd ../..
2 $ git add .
3 git commit -m "Customized Recipes for application deployment"
4 [master 887f558] Customized Recipes for application deployment
5  6 files changed, 149 insertions(+), 1 deletion(-)
6  create mode 100644 chef/cookbooks/myapp/attributes/web.rb
7  create mode 100644 chef/cookbooks/myapp/recipes/web.rb
8  create mode 100644 chef/cookbooks/myapp/templates/default/database.yml.erb

To Continue…

Introduction – Introduction
Installing Prerequisites – XCode, CommandLineTools, Homebrew, RVM, Ruby, and VirtualBox
Project Setup – Create the git repository and directory structure for Vagrant, Chef, etc.
Vagrant/Veewee Installation – Install Vagrant/Vewee to create/control VirtualBox machines
Define/Create a Vagrant Box – Define and Create a Vagrant Box for use i VirtualBox
Provisioning Machines with Vagrant – Provision a cluster (Web/DB) of machines using Vagrant
Configuring Machines Using Chef Solo – Configuring our new machine instances using Chef Solo
Customizing Recipes for Our Application – Customize the recipes we have to prepare for our application deployment
Create and Deploy a Rails Applications – Create a simple Rails application and deploy it to our Vagrant instances
Migrate from Chef Solo to Hosted Chef – Migrate from using Chef Solo to hosted Chef at OpsCode
Migrate Servers to RackCloud – Migrate your servers from VirtualBox to “The Cloud” using Rackspace.