Introduction

With Chef, you can manage your servers by writing code, not by running commands; integrate tightly with your applications, databases, LDAP directories; easily configure applications that require knowledge about your entire infrastructure; and create perfect clones of QA environments, pre-production (staging) and production environments, etc.

One motivation in writing this document is that searching for information, even simple getting-started information, always seems to be heavy on the Ruby side of things with stuff like Gem. Everyone assumes Ruby/Gem competence. Many of us readily accept hacking together simple recipes in Ruby, but stop short of wanting to become Ruby programmers. It's a bit like writing Perl scripts without the added competence of CPAN.

Links

• Chef home
• Just enough Ruby for Chef
• Common errors
• Opscode Chef practice, pretty cool step-by-step walk through setting up a web server.
• Foodcritic: A lint tool for your Opscode Chef cookbooks, a nice idea I haven't tried.
• Chef best practices, old, but important reading as you gain experience.
• What is the idiomatic way to install a Debian package using Chef?
• A knife.rb for Our Time, great way to share this file with others.
• Getting Started with Chef
• How to write a good Chef cookbook.
• About Roles: Attribute Precedence. This is a really important page that discusses attribute precedence and gives a table that illustrates increasing precedence from left to right and top to bottom (i.e.: Role and and automatic are strongest).
• About the chef-client Run.
• About iknife-backup.

Setting up the Chef server and/or client

In the very recent past (May 2013), this has gotten easier. It is two downloads. It's supposed that disparate versions of Chef can be mixed.

1. Browse to http://www.opscode.com/chef/install.
2. Under Installing the Chef client, select OS, version and platform architecture.
 
If behind a proxy or firewall, don't use the Quick Install, but Downloads to download the Chef client*:
 
3. Choose desired Chef version.
4. Copy downloaded package to host to install.

^* Note that the Chef server install is always a download.

Step-by-step Chef configuration

Overview of steps.

- Set up a Chef server.
- Set up one or more Chef clients.
- Select a development host or workstation from which to "command" Chef.

Set up the Chef server and node software

These instructions assume that a Chef server is already installed and running, and that one or more Chef clients/nodes are installed in terms of having the relevant packages as noted above.

Set up the administrator/developer client

The following configuration file, knife.rb, is used in these steps. The Chef server URL below is the one you supplied during the installation of the server.

    log_level                :debug
    log_location             STDOUT
    node_name                'jack-client'
    client_key               '/home/jack/dev/chef-repos/.chef/jack-client.pem'
    chef_server_url          'http://16.86.192.111:4000/'
    cookbook_path            "/home/jack/dev/chef-repos/cookbooks"
    cookbook_email           "[email protected]"
    cache_type               'BasicFile'
    cache_options( :path => '/home/jack/dev/chef-repos/checksums' )

Please see Creating a client. Until you get this right (running knife with no errors), do not attempt to continue.

Node registration, Chef client(s)

1. Using NTP, synchronize time between clients and server. Chef will error out if the drift exceeds 15 minutes. (See Notes on NTP.)
2. Get root.
3. Create /etc/chef and make that the current working directory.
4. Run knife configure client, knife.rb:

       $ knife configure client ./
   

   This will generate client.rb and validation.pem. Edit client.rb and change log-level from "info" to "debug". (Note: you may need to adapt some filenames to the version of Chef you're setting up. See Files.)

5. Replace validation.pem with the one your Chef server as these must match perfectly.
6. Run chef-client.

       # chef-client
   

   Unless everything is magically perfect, this will produce numerous errors that must be fixed. Frequently, these are errors in the configuration file, client.rb, such as a failure to find the server. In my observation, Chef will not make use of anything in /etc/hosts, so hard IP addresses must be used. I don't know whether it uses DNS as my set-ups have always been using ad hoc internal ESXi VMs or scrap hardware, never formal hosts noted in DNS tables.

   Sometimes it's good to edit client.rb to log_level to :debug and other times the debug output is just plain confusing and alarming. Try it both ways before deciding whether there's an error or not.

7. Now go to the Chef web interface to make sure that node is registered. You can do this from a browser running on your development host, which would not typically be the Chef server or any client. It's at this point that you'll make use of the password you supplied when installing the Chef server. Here, the username/password are (from the installation instructions earlier on this page), admin/Escoffier.

       http://16.86.192.111:4040
   

8. If everything is cool, you should see your client node listed (by hostname) as a client known to the Chef server.

1. Create a new subdirectory on your development host such as /home/jack/dev/chef-repos and create a hidden .chef directory. Place a copy of knife.rb (above) there. Change the properties to match that host.
2. Go to the Chef web interface and create a user with admin rights. (See how to do this.). Copy the private key to a PEM file, e.g.: jack-client.pem. (Note: if you already had an admin user, but don't have access to the original PEM, i.e.: if you're inheriting maintenance of the Chef process, you'll have to re-create the private key. See illustration here.)
3. Create a cookbooks directory and add any cookbooks. For example:

   `— cookbooks
       `— helloworld
           `— default.rb
   

   See Retrieving an existing cookbook, which fetches an existing recipe known to the server's repository. Or, create new ones.
4. Use knife commands to upload to the repository the recipe (or recipes) which are Ruby files.
5. In the web interface, verify that the cookbook is loaded correctly into the cookbook section.

1. Go to Roles and create a new role. This is something like tomcat-app, database-server, blowfish-preparer—anything that is an agent or actor in this process. If you have multiple VMs running different Tomcat applications, maybe tomcat-app would be too generic or insufficiently granular nomenclature. If some nodes are running PostgreSQL while others MongoDB, make database-server pg-database-server and mongo-database-server. Etc. Don't pay too close attention to many of the examples of roles in Chef you find Googling; the names are just confusing.
2. In the menu assign it the cookbook from the steps above. Doing this will mean very simply that any node this role is associated with gets the cookbook in question, very specifically, gets all the software and settings that the recipes in this cookbook set up—no others, no less than these: this is where the Chef rubber meets the road!
3. Go to the web interface representation of the node being configured and add this role for it.
Do this next bit on the client/node...
4. Next go to the actual client node and run chef-client again.
5. Verify that the changes have taken effect. If you've set :debug in client.rb (see Client/node below), you should see output to indicate that this is working.

At this point in the step-by-step process, Chef should be set up and running on a server and at least one node, as administered by your development host.

What's missing from the above?

Chef is a lot more sophisticated than to be used to associate some lame, do-nothing recipe with a node. What really needs to happen is to set up roles to accomplish this, a separate role for each kind of node to be installed, perhaps multiple roles that set up individual contributions that make one or even more types of nodes. The rest of this page has the objective of explaining that, though an Opscode course would obviously be better.

Polishing the installation

If you have myriad client/nodes, you aren't going to want to visit each one physically to 1) log in, 2) get root and 3) run chef-client every time you wish to update a node after publishing a new recipe.

Instead, you probably want to set up, as root, a cron job that awakens and checks for new recipes (i.e.: runs chef-client) every so often, for example, 5 minutes.

/etc/chef/chef-cron.tab:

    # Every five minutes, run chef-client:
    */5 * * * * chef-client

To launch the cron job the first time:

    $ crontab /etc/chef/chef-cron.tab

For further polishing, check into Sophisticated development host for using Chef.

Creating a client

In Chef terminology, a bit mixed up, a client is...

• a VM that is a node to Chef, see grey boxes in illustration and The Chef client node
• a role
• a developer on a development host (see green box in illustration and below)

Creating a development host client¹

We're concerned with the last type of client. In the Chef web interface, running on the development host, do the following to create a new client:

1. Open the Chef web interface in a browser from the development host.
2. Click Clients tab.
3. Click Create tab.
4. Type in Name, 'jack-client'.²
5. Click Admin.
6. Click Create Client.
7. Copy the key from the paragraph labeled, "Please copy and save..." to the clipboard.
8. Open an editor on the path /home/jack/dev/chef-repos/.chef/jack-client.pem and paste the contents of the clipboard.
9. Ensure this PEM file is referenced from knife.rb.³
10. In a shell open, probably on the path /home/jack/dev/chef-repos/, run knife:

        $ knife node list
    

¹ See also Sophisticated development host for using Chef.
² Note: I've had much grief trying to set up and consume a Client in the Chef web interface that is the same as my user (in this case, 'jack'). It seems to work when I use something else, like 'jack-client' here.
³ This is the knife.rb placed usually on a path like /home/jack/dev/chef-repos/.chef. The path to the PEM file is indicated by client_key in knife.rb as shown here.

Errors

This is the most brittle part of using Chef. You will almost always get:

~/dev/chef-repos/.chef $ knife node list
ERROR: Failed to authenticate to http://16.86.192.111:4000/ as app02 with key /home/jack/dev/chef-repos/.chef/jack.pem
Response:  Failed to authenticate. Ensure that your client key is valid.

This is due to

1. The private key was miscopied, newlines introduced, etc.
2. The name chosen for client was mysteriously unacceptable to Chef—likely coinciding with a username or other.
3. Who knows?

Managing (uploading, deleting and listing) recipes

You can get cookbooks with recipes that others (like the Opscode community) have written for common things on the web, such as for installing Tomcat 6, MongoDB, apt, etc. and download (the entire contents, a directory structure of) them to your local host from which you intend to manage the Chef server and nodes. A real cookbook has a rather complex subdirectory structure. (Our "Hello World" cookbook consists only of a defaut recipe in default.rb.)

    ~/dev/chef-repos $ tree -d cookbooks
    cookbooks
    |— helloworld
    `— mongodb
        |— attributes
        |— definitions
        |— files
        |   `— default
        |— libraries
        |— recipes
        `— templates
            `— default

    13 directories

Once you have your cookbooks in this structure, you can "knife-up" cookbooks at will.

    ~/dev/chef-repos $ knife cookbook upload --all
    Uploading apt            [2.0.0]
    Uploading helloworld     [0.0.0]
    Uploading mongodb        [0.11.0]
    Uploaded all cookbooks.

As it turns out, the MongoDB recipes require the advanced packaging tool for installation and, therefore, depend on a second recipe for that, shown here, but not shown in the previous directory tree listing. If the recipes of a cookbok you're trying to upload depend on another, you'll get a slightly cryptic message such as:

    ~/dev/chef-repos $ knife cookbook upload mongodb
    Uploading mongodb        [0.11.0]
    ERROR: Chef::Exceptions::CookbookNotFoundInRepo: Cannot find a cookbook named apt; did you forget to add metadata\
           to a cookbook? (http://wiki.opscode.com/display/chef/Metadata)

Here, I deleted my "Hello World" recipe because I don't need it anymore. It is not, however, deleted from my cookbook subdirectory.

    ~/dev/chef-repos $ knife cookbook delete helloworld
    Which version(s) do you want to delete?
    1. helloworld 1.0.1
    2. helloworld 0.0.1
    3. helloworld 0.0.0
    4. All versions

    4
    Deleted cookbook[helloworld][1.0.1]
    Deleted cookbook[helloworld][0.0.1]
    Deleted cookbook[helloworld][0.0.0]

So, any time I wish to restore the "Hello World" recipe, it's easy. (My copy didn't have version information and now there's only one version on the Chef server.)

    ~/dev/chef-repos $ knife cookbook upload helloworld
    Uploading helloworld     [0.0.0]
    Uploaded 1 cookbook.

Finally, after adding a cookbook for Tomcat 6, not previously shown, I list the recipes my Chef server has:

    ~/dev/chef-repos $ knife cookbook list
      apt          2.0.0
      helloworld   0.0.0
      mongodb      0.11.0
      tomcat6      0.5.4

Every time a Chef client runs, typically on one of the managed nodes, the following occurs:

1. The node is built.
2. Cookbooks are synchronized.
3. The resource collection is compiled.
4. The node is configured.
5. Notices are given and exceptions handled.

The Chef node is a Ruby object that represents the host (or machine) that's being configured. It has attributes and a run list. The object is rebuilt every time the Chef client is run, merging input from the local host, the Chef API and attributes and run lists from roles.

Ruby can use .yml files to build .json files, but I haven't enough Ruby experience yet to do that.

    {
        "normal": { "mongodb" : { "replicanode" : { "port" : 37017 } } },
        "name": "acme-dev-db04",
        "override": { },
        "default": { },
        "json_class": "Chef::Node",
        "automatic": { },
        "run_list":
        [
            "recipe[apt]",
            "recipe[mongodb::replica]",
            "role[install_database_node]",
            role[install_replica_node]"
        ],
        "chef_type": "node"
    }

    default[ :mongodb ][ :replicanode ][ :port ] = 37017

    # =====================================================
    # This will erect the first, second, third, etc. node
    # of a replica set because the port number for MongoDB
    # to use is specified in each node.
    # =====================================================
    name "install_replica_node"
    description "Role for managing MongoDB database nodes"

    for_database_servers = %w{
	    recipe[apt]
	    recipe[mongodb]
	    recipe[mongodb::replica]
    }

    run_list for_database_servers

    template "/data/mongodb/mongodb.conf" do
      source "replica-mongodb.conf.erb"
      owner mongodb
      group mongodb
      mode 00755           # -rwxr-xr-x
    end

    # /data/mongodb/mongodb.conf
    port=<%= node[ :mongodb ][ :replicanode ][ :port ] %>
    replSet=<%= node[ :mongodb ][ :replset ][ :name ] %>
    dbpath=/data/mongodb
    logpath=/data/mongodb/mongodb.log
    fork=true
    logappend=true

It's crucially important for Chef programmers to understand the anatomy of a Chef client run. This is usefully documented here: About the chef-client Run, with a superb diagram and a list of stages.

Cooking with Chef, part 2.

Note: There is no direct relationship or dependency between roles and cookbooks. Cookbooks are collections of recipes; those recipes may be used by any role. Multiple cookbook subdirectories (in knife.rb) is being deprecated by Chef because it requires more maintenance and is confusing.

Chef environments are most obvious in the Chef server web interface. There are many ways to create them; see Create Environments.

The Chef server web interface

The Chef web interface is reached via any browser anywhere and served up by the Chef server. Most or all of what can be seen or done in it can be seen or done at the knife command line.

Don't use this interface other than to observe state (and get private keys, etc.) The disadvantage of using the management console for editing much of anything is that whatever you do will probably not be reflected in your local version control system unless you then also download them from the server.

Chef and time synchronization

Client nodes that are being configured (by executing chef-client) and the Chef server must be in close synchronization. A drift longer than 15 minutes will result in chef-client erroring out.

The best way to prevent this is to install NTP on the Chef server host and all client nodes.

Note that sometimes Chef will complain of drift when there isn't any or drift that you fix, but that, once fixed, does not correct the problem.

Once, I bounced a client node and, when it came back up, it was running on its hardware clock time. I waited, but it didn't correct itself. I became impatient and hand-corrected it, but that didn't clear the error: Chef still complained and refused to carry out the update. It was late Friday afternoon.

When I came back in on Monday, I was able to run chef-client. My conclusion is that patience is required. NTP probably didn't update something Chef was looking at until later.

Files

Development host/workstation files

Files on the workstation or development host that will be used to administer Chef:

• .chef/knife.rb
• .chef/username.pem
• .chef/validation.pem

A Chef repository is a folder on this workstation from which the Chef infrastructure is managed (administrated). Typically, a repository is created on a path like /home/username/dev/chef-repos and configuration subdirectory, .chef, is created under there.

The development (or Chef administration) host has an open-ended list of subdirectories and files on it. Above are just the ones that are basic. All of the definitions of nodes, roles, cookbooks/recipes, etc. are in subdirectories too. These are discussed in separate sections elsewhere on this page (under nodes, roles, etc.).

Client/node files

Files on the chef-client node:

    /etc/chef
    |— client.pem
    |— client.rb
    `— validation.pem

Specifically, ...

• client.rb, contains configuration for communicating with the Chef server when you run chef-client. You can also insist that Chef get verbose by changing :info to :debug as here:

      log_level        :debug
      log_location     STDOUT
      chef_server_url  'http://16.86.192.111:4000'
      validation_client_name 'chef-validator'
  

• validation.pem, the permission certificate, downloaded from the Chef server.
• client.pem, the private key created by the Chef server web interface at the time that the client node is created. (Copy the private key out of the UI and paste it into a file to copy to the chef-client node.)

These files are created using knife in a step-by-step process described elsewhere in this document:

    $ knife configure client ./

...on a workstation or development host, then copied to the chef-client node.

/var/chef is where the Chef client/node keeps its cache files. /opt/chef is where the Chef executables are kept.

You should keep a cron tab file on /etc/chef for use in launching a cron job to run chef-client periodically. See Polishing the installation. This file is not named or listed here because it's not part of Chef, but part of what you might do to improve your mileage.

Server files

Files on the chef-server host path /etc/chef (Chef version < 11):

• client.rb, contains some server configuration, not to be confused with this file on chef-client nodes.
• server.rb, contains some server configuration.
• validation.pem, permission certificate, to be copied to client/node.
• webui.pem, permission certificate.
• webui.rb, contains Chef web interface configuration including the administrator name and password.
• certificates, subdirectory containing cert.pem and key.pem.
• solr.rb, contains Apache Solr configuration.
• solr-jetty, subdirectory contain Solr and Jetty configuration files.

Beginning in Chef version 11, these files are de rigueur:

• admin.pem, permission certificate.
• chef-server-running.json, configuration.
• chef-server-secrets.json, configuration.
• chef-validator.pem, client-validator certification permission certificate, replaces old validation.pem, to be copied to client/node.
• chef-webui.pem, permission certificate.
• chef-server.rb, Chef server configuration file.^*

/var/lib/chef is where the Chef server keeps its working files. There is no /opt/chef subdirectory.

^* By default, this file isn't created when chef-server-ctl reconfigure is run. Docs imply that it is created, but it doesn't right off and need not as long as the default configuration is acceptable.

Data bags are global and must be "named". Installing a data bag can be done "from file" with the file living in the default location, i.e.: data_bags. But, this cannot be arbitrarily hierarchical—one subdirectory of depth only. Please see About Data Bags.

    $ knife data bag from file [subdirectory] mydatabag.json

For example, ...

    data_bags
        admins
            charlie.json

but not

    data_bags
        admins
            another-subdirectory
                charlie.json

Chef doesn't support putting one data bag in another.

Interfaces

From the Chef recipe, data_bag() and data_bag_item() are methods to fetch data bags. The following follows closely something Daniel DeLeto said in an answer in the Chef forum.

The way that data_bag[_item] works in recipes can be confusing. It's a very thin wrapper over the actual server ReST API for data bags.

In general, the ReST API serves content in a way very similar to how a basic (HTML) web site works. Whenever you do a GET from a container URL (e.g., /nodes, but not /nodes/specific-node), the API returns a list of the items and their URLs. To get the actual content of an item, you need to follow that item's URL. (Note that this aspect of the Chef API design is called HATEOAS).

Assume charlie.json contains

{
    "id" : "charlie",
    "login" : "chuck",
    "password" : "Test123"
    "shell" : "bash"
    "uid" : 1003
    "gid" : "ops"
}

Data bags are just user-definable containers, so when you list them via the API, you get a list of the IDs and their URLs back. Since the data_bag() method is just a thin wrapper over the API:

    x = data_bag :admins
    # => [ "charlie" ]

you have to call data_bag_item() on each one in order to load the actual item:

    x = data_bag_item( :admin, "charlie" )
    # => data_bag_item[ "admins", "charlie", { "id" => "charlie", "shell" => "bash", etc. # (more stuff)

In practice most people use the search functionality (method) to load everything in a data bag via a single call when they want to do something with all of the items. chef-solo differs from chef-client in that it loads every item when you use data_bag() in a recipe. This is for practical reasons, because chef-solo doesn't have search,

search()

Per DeLeto, search() is what traditional, Chef client-Chef server implementations use (mine). search() returns a Ruby hash (hashmap).

    charlies_bag = search( :admins, "id:charlie" )          # returns the whole bag as a hash
		charlies_id  = search( :admins, "id:charlie" ).first    # returns just chuck's id
		charlies_bag[ :uid ]                                    # chuck's user id
		# => 1003

Roles in Chef

A role is a way to define certain patterns and processes that exist across nodes as a single job function. A role consists of zero or more attributes and a run list. Each node can have zero or more roles assigned to it. Roles are unversioned.

Role Cookbooks and Wrapper Cookbooks

Roles can be as super-granular as "application-node" or "database-node" or, identifying common components between the needs of two nodes, for example, more granular like a "common-base" or "base-client" with a more modest collection of recipes associated with it that more than one type of node will need. Then, the node will have more than one role associated with it.

Remember (or note for the first time) that recipes are held in cookbooks.

In the roles subdirectory on the development (administering) host might be a file like:

application-host.rb:

    name "application-host"
    description "The cookbooks for the application-host role"

    for_application_servers = %w
    {
        recipe[chef-repos::apt]
        recipe[chef-repos::tomcat6]
    }

    run_list for_application_servers

This associates (potentially) all the recipes the role uses in performing its job creating an application server node's run list.

Roles are often the glue that makes a recipe work on a node. Please see the a discussion of the relationship between node and role files here.

Sophisticated development host for using Chef

Instead of setting (or, by way of polishing the process later), a sophisticated approach replaces the development host client with Git and Jenkins thus:

1. Recipes are coded and maintained on a local development just as if source code (well, precisely because they are source code).
2. Recipes are committed to a Git repository as updates and for safe-keeping.
3. A Jenkins project is created that monitors Git pushes to origin master, and uses a plug-in to knife them up to the Chef server.

Obviously, the Git repository must be carefully thought out, perhaps with master being considered extremely sacred since pushing there won't just break a build, but one or more servers running live product.

Chef Solo

Here is a tutorial on Chef Solo: http://www.youtube.com/watch?v=1G6bd4b91RU.

Chef Solo is where you don't necessarily install a server. Instead, you develop and test-deploy your recipes to your local development host. Later, you can use those recipes on any Chef server. On the Chef server, you invoke the chef-solo command and the server runs the recipes on all the nodes specified.

Appendix: Examples

These are sort of pointed illustrations of the effect that knife commands can have on what's seen in the web interface.

Before a new node has anything associated with it, it appears as here. This is a node with nothing...

...as compared to this node which has something on it:

The commands issued were these. A role and a recipe were associated with this node as seen in the image above under "Run List" and "Recipes".

    ~/dev/chef-repos $ knife node run_list add acme-dev-app01 'role[application-node]'
    run_list:  [role[application-node]]
    ~/dev/chef-repos $ knife node run_list add acme-dev-app01 'recipe[chef-repos::tomcat6]'
    run_list:
        role[application-node]
        recipe[chef-repos::tomcat6]

Here's "role from file".

    ~/dev/chef-repos $ knife role from file application-node.rb
    Updated Role application-app!
    ~/dev/chef-repos $ knife role from file database-node.rb
    Updated Role database-app!

roles/application-node.rb:

    name "application-app"
    description "Role for managing Tomcat application nodes"

    for_application_servers = %w{
	    recipe[chef-repos::tomcat6]
    }

    run_list for_application_servers

(Beware: stuff in Ruby like %w does not allow the use of a sane, Allman syntax.)

I see no effect in the web interface after doing the above.

Appendix: Chef web interface illustrations

For what they're worth, these illustrations don't match the on-going and expanding notes on this page (except maybe at one, early point). They're here just to orient and your mileage using them may vary.

(Client) nodes serviced by the Chef server...

Users...

Clients...

Lost client's private key? Here's how to get a new one: Just click Edit, then Regenerate Private Key.

Cookbook list: there's only one here.

Clicked helloworld cookbook:

Examine recipes:

Appendix: Some chef-client run scrapes...

root@acme-dev-app01:/etc/chef# chef-client
[2013-06-26T16:32:17-06:00] INFO: *** Chef 10.18.1 ***
[2013-06-26T16:32:17-06:00] INFO: [inet6] no default interface, picking the first ipaddress
[2013-06-26T16:32:17-06:00] INFO: Run List is [recipe[tomcat6], role[application-node]]
[2013-06-26T16:32:17-06:00] INFO: Run List expands to [tomcat6]
[2013-06-26T16:32:17-06:00] INFO: HTTP Request Returned 404 Not Found: No routes match the request: /reports/nodes/acme-dev-app01/runs
[2013-06-26T16:32:17-06:00] INFO: Starting Chef Run for acme-dev-app01
[2013-06-26T16:32:17-06:00] INFO: Running start handlers
[2013-06-26T16:32:17-06:00] INFO: Start handlers complete.
[2013-06-26T16:32:17-06:00] INFO: Loading cookbooks [tomcat6]
[2013-06-26T16:32:17-06:00] INFO: Storing updated cookbooks/tomcat6/recipes/default.rb in the cache.
[2013-06-26T16:32:17-06:00] INFO: Processing execute[mkdir] action run (tomcat6::default line 27)
[2013-06-26T16:32:17-06:00] INFO: execute[mkdir] ran successfully
[2013-06-26T16:32:17-06:00] INFO: Processing cookbook_file[/usr/share/tomcat6//apache-tomcat-6.0.37.tar.gz] action create (tomcat6::default line 33)
[2013-06-26T16:32:18-06:00] INFO: cookbook_file[/usr/share/tomcat6//apache-tomcat-6.0.37.tar.gz] mode changed to 644
[2013-06-26T16:32:18-06:00] INFO: cookbook_file[/usr/share/tomcat6//apache-tomcat-6.0.37.tar.gz] created file /usr/share/tomcat6//apache-tomcat-6.0.37.tar.gz
[2013-06-26T16:32:18-06:00] INFO: Processing execute[tar] action run (tomcat6::default line 39)
apache-tomcat-6.0.37/bin/catalina.sh
apache-tomcat-6.0.37/bin/digest.sh
apache-tomcat-6.0.37/bin/setclasspath.sh
apache-tomcat-6.0.37/bin/shutdown.sh
.
.
.
apache-tomcat-6.0.37/webapps/manager/images/fix.gif
apache-tomcat-6.0.37/webapps/manager/images/tomcat.gif
apache-tomcat-6.0.37/webapps/manager/images/update.gif
apache-tomcat-6.0.37/webapps/manager/images/void.gif
apache-tomcat-6.0.37/webapps/manager/status.xsd
apache-tomcat-6.0.37/webapps/manager/xform.xsl
[2013-06-26T16:32:18-06:00] INFO: execute[tar] ran successfully
[2013-06-26T16:32:18-06:00] INFO: Processing execute[rm] action run (tomcat6::default line 45)
[2013-06-26T16:32:18-06:00] INFO: execute[rm] ran successfully
[2013-06-26T16:32:18-06:00] INFO: Processing template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/context.xml] action create (tomcat6::default line 51)
[2013-06-26T16:32:18-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/context.xml] backed up to /var/chef/backup/usr/share/tomc...
[2013-06-26T16:32:18-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/context.xml] updated content
[2013-06-26T16:32:18-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/context.xml] owner changed to 0
[2013-06-26T16:32:18-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/context.xml] group changed to 0
[2013-06-26T16:32:18-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/context.xml] mode changed to 644
[2013-06-26T16:32:18-06:00] INFO: Processing template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/server.xml] action create (tomcat6::default line 57)
[2013-06-26T16:32:19-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/server.xml] backed up to /var/chef/backup/usr/share/tomc...
[2013-06-26T16:32:19-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/server.xml] updated content
[2013-06-26T16:32:19-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/server.xml] owner changed to 0
[2013-06-26T16:32:19-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/server.xml] group changed to 0
[2013-06-26T16:32:19-06:00] INFO: template[/usr/share/tomcat6//apache-tomcat-6.0.37/conf/server.xml] mode changed to 644
[2013-06-26T16:32:19-06:00] INFO: Chef Run complete in 1.392388888 seconds
[2013-06-26T16:32:19-06:00] INFO: Removing cookbooks/tomcat6/libraries/tomcat_manager.rb from the cache; it is no longer needed by chef-client.
[2013-06-26T16:32:19-06:00] INFO: Removing cookbooks/tomcat6/libraries/tomcat.rb from the cache; it is no longer needed by chef-client.
[2013-06-26T16:32:19-06:00] INFO: Removing cookbooks/tomcat6/metadata.json from the cache; it is no longer needed by chef-client.
[2013-06-26T16:32:19-06:00] INFO: Removing cookbooks/tomcat6/definitions/tomcat_app.rb from the cache; it is no longer needed by chef-client.
[2013-06-26T16:32:19-06:00] INFO: Running report handlers
[2013-06-26T16:32:19-06:00] INFO: Report handlers complete

This appeared to work. What did it do? It disembowled the Tomcat tarball in files/default/apache-tomcat-6.0.37.tar.gz over the top of what was there already. This is a bit messy, but we're behind a firewall. I need to investigate how better to do this. The recipe is, however, very simple and easy to understand.

Appendix: Some as-yet disorganized notes...

Here's how to create a configuration or other file from Chef. This is basically coping a file.

    # Template to edit the memcached.conf file...
    template "/etc/memcached.conf" do
      source "memcached.conf.erb"
      mode   "0644"
      owner  "root"
      group  "root"
    end

Under the templates subdirectory, there's memcached.conf.erb that's the contents (or subcontents?) of the file to be created (or edited?) on path /etc/memcached.conf. It uses this file to create the target one. Down inside this file might be something like:

    -l <%= node[:ipaddress] %>

This is a variable that's filled in by Chef at deployment. When you issue chef-client from any node, it's going to talk to the Chef server, cough up its IP address, and Chef will use this value in there. That is, in this case. Obviously other variables could exist.

    Insert <%= :gobbledeegoop %> here!

Other variables may be specified in data bags.

    chef-repos
    |— cookbooks
    |— databags
    |  `—application-node-bags.json (contains: { “gobbledeegoop” : “Bouger all” } )
    |— nodes
    `— roles

Here's how to use the advanced packaging tool (apt-get) to install something.

    # Installing fun-software...
    package "fun-software" do
      action :install
    end

Here's how to copy a file from the recipe's files subdirectory to its final resting place and install it using the Debian package manager:

    # Copy the Debian package to /tmp...
    cookbook_file "/tmp/xyz.deb" do
        source "xyz.deb"                  # xyz.deb must be at files/default
        mode   "0644"
        owner  "root"
        group  "root"
    end

    # Install the Debian package...
    dpkg_package "xyz" do
        source "/tmp/xyz.deb"
        action :install
    end

The commands I used as I edited, updated and ran (lather, rinse, repeat) through this were as follows. The delete step is probably unnecessary, but I was making sure.

On my development host...

    ~/dev/chef-repos $ knife cookbook delete xyz
    Do you really want to delete xyz version 1.0.0? (Y/N) y
    Deleted cookbook[xyz version 1.0.0]
    ~/dev/chef-repos $ knife cookbook upload xyz
    Uploading xyz     [1.0.0]
    Uploaded 1 cookbook.
    ~/dev/chef-repos $ knife node from file acme-dev-app01.json
    Updated Node acme-dev-app01!

On the node running the Chef client...

    root@acme-dev-app01:/etc/chef# chef-client
    (lots of output...)
    root@acme-dev-app01:/etc/chef# ll /tmp
    total 7396
    drwxrwxrwt  7 root    root       4096 Jun 27 17:14 ./
    drwxr-xr-x 23 root    root       4096 Jun 14 06:48 ../
    -rw-r--r--  1 root    root    7544562 Jun 27 17:14 xyz.deb
    drwxr-xr-x  2 root    root       4096 Jun 27 17:14 hsperfdata_root/
    drwxr-xr-x  2 tomcat6 tomcat6    4096 Jun 17 04:30 hsperfdata_tomcat6/
    drwxr-xr-x  2 tomcat6 root       4096 Jun 17 04:30 tomcat6-tomcat6-tmp/
    drwx------  2 root    root       4096 Jun 17 09:46 vmware-root/
    root@acme-dev-app01:/etc/chef# dpkg --list | grep xyz
    ii  xyz                       1.00.141                        XYZ is fun
    root@acme-dev-app01:/etc/chef# ll /var/lib/tomcat6/webapps
    total 11808
    drwxrwxr-x 5 tomcat6 tomcat6    4096 Jun 27 17:14 ./
    drwxr-xr-x 6 root    root       4096 Mar 27 10:31 ../
    drwxr-xr-x 5 tomcat6 tomcat6    4096 Jun 27 17:14 xyz/
    -rw-r--r-- 1 russ    russ    7572973 Jun 27 16:37 xyz.war
    drwxr-xr-x 3 tomcat6 tomcat6    4096 Mar 27 10:31 ROOT/