Combing through with a fine-toothed comb, this is an example of individual Chef clients set up for different purposes in a MongoDB data center. This table goes along with Transcription of Notes on Setting Up MongoDB and Chef on a VM in a Data Center.
Colors explore tightly coupled relationships. Below are set-ups for the following. By "rolling up", a second step that involves invoking the MongoDB shell for finalizing set-up is specified. The basic set-up bounces the MongoDB service (binary) so that it's running in its fully configured state which must happen prior to deeper MongoDB configuration. Implicit nodes missing explicit treatment are listed here.
Featured in table below | Implicit/suggested | |
---|---|---|
1. db01 --single, replica node 2. db03 --replica node that rolls up a replica set 3. db07 --configuration server 4. db10 --sharding router that rolls itself up 5. db12 --arbiter that rolls itself up |
db02 --single, replica node db04 --single, replica node db05 --single, replica node db06 --replica node that rolls up replica set 2 db08 --second configuration server db09 --third configuration server db11 --second shard that rolls up shard 2 |
1 |
Nodedb01.json:{ "normal" : { "port" : 27017 }, "name": "db01", "override": { }, "default": { }, "json_class": "Chef::Node", "automatic": { }, "run_list": [ "recipe[apt]", "recipe[mongodb]", "recipe[mongodb::replica]", "role[install-database-node]", "role[install-replica-node]" ], "chef_type": "node" } |
Recipereplica.rb:# Copy the upstart configuration file to /etc/init... template "/etc/init/mongodb.conf" do source "replica-upstart.conf.erb" owner "root" group "root" mode 00644 # -rw-r--r-- end template "/data/mongodb/mongodb.conf" do source "replica.conf.erb" owner "mongodb" group "mongodb" mode 00644 # -rw-r--r-- notifies :restart, "service[ $ip_address ]", :immediately end # Restart the service so that replica-set.rb can do its magic. service $ip_address do provider Chef::Provider::Service::Upstart action [ :enable, :start ] end |
Data bag(none for simple replica nodes) |
Rolerollups/install-replica-node.rb:
name "install-replica-node"
description "Role for managing MongoDB database nodes"
for_database_servers = %w{
recipe[apt]
recipe[mongodb]
recipe[mongodb::replica]
}
run_list for_database_servers
|
2 |
Nodedb03.json:
{
"normal": { "rollup" : "replicaset-1" },
"name": "db01",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::replica]",
"recipe[mongodb::replicaset]",
"role[install-database-node]",
"role[install-replica-node]",
"role[config-replicaset]"
],
"chef_type": "node"
}
|
RecipeUse replica.rb, plus... replicaset.rb:which = node[ :rollup ] # i.e.: "replicaset-1" bag = search( :rollups, which ) # (see this code in the appendix) load 'get-config-command.rb' configuration = get_configuration_command( bag ) # Create replica set. This works because the MongoDB service was bounced # just before we were invoked. execute "compose-replicaset-configuration" do command "mongo --eval '#{configuration}'" EOS retries 6 retry_delay 10 end # Initiate replica set... execute "initiate-replicaset" do command "mongo --eval 'rs.initiate( config )'" EOS retries 6 retry_delay 10 end |
Data bagrollups/replicaset-1.json:
{
"id" : "replicaset-1",
"name" : "rs-1",
"description" : "Shard 1 replica set",
"replica_1" :
{
"hostname" : "16.86.193.100",
"port" : 37017,
"node" : "db01"
},
"replica_2" :
{
"hostname" : "16.86.193.101",
"port" : 37018,
"node" : "db02"
},
"replica_3" :
{
"hostname" : "16.86.193.102",
"port" : 37019,
"node" : "db03"
}
}
|
Roleconfig-replicaset.rb:
name "config-replicaset"
description "Role for rolling up MongoDB replca sets"
for_database_servers = %w{
recipe[apt]
recipe[mongodb]
recipe[mongodb::replica]
recipe[mongodb::replicaset]
}
run_list for_database_servers
|
3 |
Nodedb07.json:
{
"normal": { "rollup" : "configsvr_1" },
"name": "db07",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::configsvr]",
"role[install-database-node]",
"role[install-configsvr]"
],
"chef_type": "node"
}
|
Recipeconfigsvr.rb:
# Copy the upstart configuration file to /etc/init.
cookbook_file "/etc/init/mongodb-configsvr.conf" do
source "mongodb-configsvr.conf"
owner "root"
group "root"
mode 00644 # -rw-r--r--
end
which = node[ :which ]
bag = search( :rollup, "configsvr" )
# (see this code in the appendix)
load 'get-hostname-and-port.rb'
hostname_port = get_hostname_and_port( which, bag )
hostname = hostname_port[ 0 ]
port = hostname_port[ 1 ]
# Copy the configuration server's /etc/mongodb.conf-equivalent to
# /data/mongodb.
template "/data/mongodb/configsvr.conf" do
source "configsvr.conf.erb"
owner "mongodb"
group "mongodb"
mode 00644 # -rw-r--r--
variables( {
hostname,
port
}
)
end
directory "/data/mongodb/configsvr" do
action :create
owner "mongodb"
group "mongodb"
mode 00755 # rwxr-xr-x
end
directory "/data/mongodb/configsvr/log" do
action :create
owner "mongodb"
group "mongodb"
mode 00755 # -rwxr-xr-x
end
|
Data bagrollups/configsvr.json:{ "id" : "configsvr", "description" : "Configuration server", "configsvr_1" : { "hostname" : "16.86.193.100", "port" : 37017, "node" : "db07" }, "configsvr_2" : { "hostname" : "16.86.193.101", "port" : 37018, "node" : "db08" }, "configsvr_3" : { "hostname" : "16.86.193.102", "port" : 37019, "node" : "db09" } } |
Roleinstall-configsvr.rb:
name "install-configsvr"
description "Role for erecting a MongoDB configuration server"
for_database_servers = %w{
recipe[apt]
recipe[mongodb]
recipe[mongodb::configsvr]
}
run_list for_database_servers
|
4 |
Nodedb10.json:
{
"normal": { "rollup" : "shard-1" },
"name": "db10",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::sharding]",
"recipe[mongodb::add-shard]",
"role[install-database-node]",
"role[install-sharding-router]"
],
"chef_type": "node"
}
|
Recipesharding.rb:# Sets ownership and privileges on the sharding subdirectories # which could be newly created. directory "/data/mongodb/sharding" do action :create owner "mongodb" group "mongodb" mode 00755 # -rwxr-xr-x end directory "/data/mongodb/sharding/log" do action :create owner "mongodb" group "mongodb" mode 00755 # -rwxr-xr-x end cookbook_file "/etc/init/mongodb-sharding.conf" do source "mongodb.conf" owner "root" group "root" mode 00644 # -rw-r--r-- end which_bag = node[ :rollup ] bag = search( :rollups, which_bag ) # Get port on which the sharding router will listen. port = bag[ :port ] # Get the bagname for the configsvr bag, then from it, get the # configuration server list (hostnames and ports). bagname = bag[ :configsvr_bag ) bag = search( :rollups, bagname ) # (see this code in the appendix) load 'get-config-server-list.rb' configsvr_list = get_configuration_server_list( bag ) # Copy the sharding router /etc/mongodb.conf-equivalent to /data/mongodb. template "/data/mongodb/mongodb.conf" do source "sharding.conf.erb" owner "mongodb" group "mongodb" mode 00644 # -rw-r--r-- variables( { :port => port, :configsvr_list => configsvr_list } ) notifies :restart, "service[ $ip_address ]", :immediately end # Restart the service so that add-shard.rb can do its magic. service $ip_address do provider Chef::Provider::Service::Upstart action [ :enable, :start ] end add-shard.rb:replSet_name = node[ :mongodb ][ :replicaset ][ :name ] # (see this code in the appendix) load 'get-add-shard-shell-command.rb' shell_command = get_add_shard_shell_command( replicaset_bag ) # Erect the mongos daemon (sharding router). This works because the # MongoDB service was bounced just before we were invoked. execute "add-shard" do command "#{shell_command}" EOS retries 6 retry_delay 10 end |
Data bagrollups/shard-1.json:
{
"id" : "shard-1",
"description" : "Shard 1",
"replicaset_bag" : "replicaset-1",
"configsvr_bag" : "configsvr"
}
|
Roleinstall-sharding-router.rb:name "install-sharding-router" description "Role for erecting a MongoDB sharding router" for_database_servers = %w{ recipe[apt] recipe[mongodb] recipe[mongodb::sharding] recipe[mongodb::add-shard] } run_list for_database_servers |
5 |
Nodedb12.json:
{
"normal": { "rollup" : "arbiter-1" },
"name": "db12",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::arbiter]",
"recipe[mongodb::add-arbiter]",
"role[install-database-node]",
"role[install-arbiter]"
],
"chef_type": "node"
}
|
Recipearbiter.rb:# Sets ownership and privileges on the arbiter subdirectories # which could be newly created. directory "/data/mongodb/arbiter" do action :create owner "mongodb" group "mongodb" mode 00755 # -rwxr-xr-x end directory "/data/mongodb/arbiter/log" do action :create owner "mongodb" group "mongodb" mode 00755 # -rwxr-xr-x end bagname = node[ :rollup ] bag = search( :rollup, bagname ) port = bag[ :port ] bag = search( :rollup, bag[ :replicaset_bag ] ) replSet_name = bag[ :name ] # Copy the upstart configuration file to /etc/init. Copy the # arbiter's /etc/mongodb.conf-equivalent to /data/mongodb. cookbook_file "/etc/init/mongodb.conf" do source "mongodb.conf" owner "root" group "root" mode 00644 # -rw-r--r-- end template "/data/mongodb/mongodb.conf" do source "arbiter.conf.erb" owner "mongodb" group "mongodb" mode 00644 # -rw-r--r-- variables( { :port => port :replSet => replSet_name } ) notifies :restart, "service[ $ip_address ]", :immediately end # Restart the service so that add-arbiter.rb can do its magic. service $ip_address do provider Chef::Provider::Service::Upstart action [ :enable, :start ] end add-arbiter.rb:bagname = node[ :rollup ] bag = search( :rollup, bagname ) hostname = bag[ :hostname ] port = bag[ :port ] # -------------------------------------------------------------------- # Add arbiter to replica set. This works because the MongoDB service # was bounced just before we were invoked. execute "add-arbiter" do command "mongo --eval 'rs.addArb( "#{hostname}":#{port}" )'" EOS retries 6 retry_delay 10 end |
Data bag
{
"id" : "arbiter-1",
"description" : "Shard 1 arbiter",
"hostname" : "16.86.192.99",
"port" : 27016,
"replicaset_bag" : "replicaset-1"
}
|
Roleinstall-arbiter.rb:name "install-arbiter" description "Role for installing MongoDB arbiter node" for_database_servers = %w{ recipe[apt] recipe[mongodb] recipe[mongodb::arbiter] recipe[mongodb::add-arbiter] } run_list for_database_servers |
These are Ruby code resources loaded and used by the recipes.
# ===================================================================== # Derive the MongoDB shell command to erect the replica set. Here's our # example, likely the hostnames will be IP addresses rather than human- # friendly names as in this example. # # config = # { # _id:"replicas", members: # [ # { _id:0, host:"db01:37017" }, # { _id:2, host:"db02:37018" }, # { _id:3, host:"db03:37019" }, # { _id:4, host:"db04:37020" } # ] # } def get_configuration_command( bag ) id = 0 found = false replica_id = "id : \"%s\"" % bag[ :name ] # We'll expect up to 4 replica nodes, though we'd really only expect 3. If # an even number, likely, there's going to be a replica, but we don't care # about that in here. replica_1 = bag[ :replica_1 ] replica_2 = bag[ :replica_2 ] replica_3 = bag[ :replica_3 ] replica_4 = bag[ :replica_4 ] replica_members = "" if !replica_1.empty? found = true hostname = replica_1[ :hostname ] port = replica_1[ :port ] replica_members += "{ _id:%d, host:%s:%s }" % [ id, hostname, port ] id += 1 end if !replica_2.empty? if found replica_members += ", " end found = true hostname = replica_2[ :hostname ] port = replica_2[ :port ] replica_members += "{ _id:%d, host:%s:%s }" % [ id, hostname, port ] id += 1 end if !replica_3.empty? if found replica_members += ", " end found = true hostname = replica_3[ :hostname ] port = replica_3[ :port ] replica_members += "{ _id:%d, host:%s:%s }" % [ id, hostname, port ] id += 1 end if !replica_4.empty? if found replica_members += ", " end hostname = replica_4[ :hostname ] port = replica_4[ :port ] replica_members += "{ _id:%d, host:%s:%s }" % [ id, hostname, port ] id += 1 end # Build the configuration statement: configuration = "config = { " + replica_id + ", members: [" + replica_members + "] }" return configuration end
# ===================================================================== # Get hostname/IP address and port number for the configuration server. # Coming in are which configuration server in the bag to use. # { # "id" : "configsvr", # "description" : "configuration server list--hostnames and ports", # "configsvr_1" : { "hostname" : "16.86.193.100", "port" : 37017 }, # "configsvr_2" : { "hostname" : "16.86.193.101", "port" : 37018 }, # "configsvr_3" : { "hostname" : "16.86.193.102", "port" : 37019 } # } def get_hostname_and_port( which, bag ) configsvr = bag[ which.to_sym ] hostname = configsvr[ :hostname ] port = configsvr[ :port ] return hostname, port end
# ===================================================================== # Set up configuration server list for launching the sharding router. # Gather the list of configuration servers from the known bag. def get_configuration_server_list( bag ) configsvr_list = nil configsvr_1 = bag[ :configsvr_1 ] configsvr_2 = bag[ :configsvr_2 ] configsvr_3 = bag[ :configsvr_3 ] if !configsvr_1.empty? configsvr_1_hostname = configsvr_1[ :hostname ] configsvr_1_port = configsvr_1[ :port ] end if !configsvr_2.empty? configsvr_2_hostname = configsvr_2[ :hostname ] configsvr_2_port = configsvr_2[ :port ] end if !configsvr_3.empty? configsvr_3_hostname = configsvr_3[ :hostname ] configsvr_3_port = configsvr_3[ :port ] end # We must have either one configuration server or three. We'll ignore # any more unless all three are known. TODO: tighten the logic and # error-handling here? if !configsvr_1_hostname.empty? and !configsvr_1_port.nil? configsvr_list = configsvr_1_hostname + ":" + configsvr_1_port.to_s else puts "Missing hostname or port number, we're screwed!" end if configsvr_2_hostname.empty? or configsvr_3_hostname.empty? # so, we'll ignore anything else halfway established... elsif configsvr_3_hostname.empty? # so, we'll ignore anything else halfway established... else # well, we'll give all three a shot... configsvr_list += "," + configsvr_2_hostname + ":" + configsvr_2_port.to_s configsvr_list += "," + configsvr_3_hostname + ":" + configsvr_3_port.to_s end return configsvr_list end
# ===================================================================== # Derive the MongoDB shell command to add the shard. def get_shard_shell_command( replicaset_bag ) replSet_name = replicaset_bag[ :name ] # Get the name of the replica set that will belong to this shard, plus # one of its hostname:port tuples. hostname = "" port = nil if !replicaset_bag[ :replica_1 ].empty? hostname = replicaset_bag[ :replica_1 ][ :hostname ] port = replicaset_bag[ :replica_1 ][ :port ] elsif !replicaset_bag[ :replica_2 ].empty? hostname = replicaset_bag[ :replica_2 ][ :hostname ] port = replicaset_bag[ :replica_2 ][ :port ] elsif !replicaset_bag[ :replica_3 ].empty? hostname = replicaset_bag[ :replica_3 ][ :hostname ] port = replicaset_bag[ :replica_3 ][ :port ] elsif !replicaset_bag[ :replica_4 ].empty? hostname = replicaset_bag[ :replica_4 ][ :hostname ] port = replicaset_bag[ :replica_4 ][ :port ] end shell_command = "mongo --eval 'rs.addShard( \"#{replSet_name}/#{hostname}:#{port}\" )'" return shell_command end