Second release of "Learning CFEngine 3"

CFEngine tip #004: How to bootstrap a CFEngine client

My apologies for the long delay since the last tip! Today we come back with a simple tip: how to bootstrap a CFEngine client.

First, some background

CFEngine is designed to operate in a fully distributed fashion – each CFEngine client (i.e. a machine running cf-agent) can operate fully autonomously, using only the policy files stored locally. For regular operation, cf-agent does not require any type of network connectivity.

In real deployments, of course, maintaining each machine’s policy independently would be impractical, so CFEngine has the concept of a policy hub. In CFEngine Community, the policy hub is simply a file server, a machine from which others can download policy files. This allows having a single point of distribution, so that changes made there can be distributed to a large number of clients. In CFEngine Enterprise, the hub has a more complex role, acting also as an aggregator of data from the clients, for the purposes of reporting and analysis.

All CFEngine clients will copy to their local /var/cfengine/inputs/ directory the contents of /var/cfengine/masterfiles/ in the policy hub, so that is where you should make any changes that you want distributed to all the machines.

Bootstrapping

When a new client is installed, it needs to be told which machine is the policy hub to which it should connect. This is done using the following command:

# cf-agent --bootstrap --policy-server=10.0.2.15

You should, of course, replace 10.0.2.15 with the actual IP address of the hub to which you want to bootstrap. The first machine you install will be the policy hub itself, and it should bootstrap to itself. In this case, you have to use its own external IP address, and not “localhost” or 127.0.0.1.

When you issue this command on the policy hub, you will see a message like this, that indicates the host recognizes itself as a policy hub:

# cf-agent --bootstrap --policy-server=10.0.2.15
** CFEngine BOOTSTRAP probe initiated

   @@@     
   @@@      CFEngine

 @ @@@ @    CFEngine Core 3.3.5
 @ @@@ @   
 @ @@@ @   
 @     @   
   @@@     
   @ @     
   @ @     
   @ @     

Copyright (C) CFEngine AS 2008-2012
See Licensing at http://cfengine.com/3rdpartylicenses

 -> This host is: precise32
 -> Operating System Type is linux
 -> Operating System Release is 3.2.0-23-generic-pae
 -> Architecture = i686
 -> Internal soft-class is linux
 -> No previous policy has been cached on this host
 -> Assuming the policy distribution point at: 10.0.2.15:/var/cfengine/masterfiles
 -> Attempting to initiate promised autonomous services...

 ** This host recognizes itself as a CFEngine Policy Hub, with policy distribution and knowledge base.
 -> The system is now converging. Full initialisation and self-analysis could take up to 30 minutes

R: This host assumes the role of policy distribution host
R:  -> Updated local policy from policy server
R:  -> Started the server
R:  -> Started the scheduler
-> Bootstrap to 10.0.2.15 completed successfully

If you issue it on a client, you will see a similar message, but indicating that it is bootstrapping from a different machine.

Regardless of what you use, after bootstrap you should see /var/cfengine/inputs/ populated with the default set of policy files, and both cf-execd and cf-serverd should be running:

# ps axw | grep [c]f-
16099 ?        Ss     0:00 /var/cfengine/bin/cf-execd
16102 ?        Ss     0:00 /var/cfengine/bin/cf-serverd

Once this has been done, CFEngine will start running every 5 minutes, updating its policy files from the hub, and executing them afterwards.

CFEngine tip #003: The distinction between bodies and bundles

This is an extract from Chapter 3 of "Learning CFEngine 3", where you can also find a much more comprehensive description of bodies and bundles in the CFEngine 3 policy language.

The distinction between bundles and bodies can be confusing at first. Remembering these points may help:

• Bodies are named groups of attributes, whereas bundles are collections of promises. Promises are the units that actually do something in CFEngine (for example, run a command or add a line to a file), whereas attributes specify characteristics of how things are done (for example, whether to run the command in a shell, or where in the file to add the line).

• The value of an attribute can be a basic data type (string, integer, list, etc.), it can be the name of a body, or it can be the name of a bundle.

• The type of an attribute’s value is fixed, and determined by the attribute itself (for example, the value of the depth_search attribute in a files: promise is always a body, and the value of an edit_line attribute is always a bundle).

• For bodies and bundles, their type is always the name of the attribute to which they correspond. For example, bodies to be used with the depth_search attribute are always declared as “body depth_search xyz”, where xyz is an arbitrary name of your choosing. The same goes for bundles: bundles to be used with the edit_line attribute are always declared as “bundle edit_line xyz".

  There are only four types of “top level” bundles that are not used as arguments to attributes: agent, server, knowledge and monitor.

• The promise types (sections) that can appear in a bundle are determined by the bundle type. For example, commands: promises can only appear in bundles of type agent. 

CFEngine tip #002: How to pass arguments to bundles using arrays

(This tip is based on a section from Chapter 5 of Learning CFEngine 3.)

Many system configuration tasks require groups of name-value pairs as arguments. For example:

• Editing configuration files in which parameters and their values need to be stored (ssh configuration files, Windows-style INI files, etc.)
• Setting user parameters. In this case, sets of name-value pairs (home directory, full name, shell, etc.) are associated with a single user, identified by name.

Having sets of related values in a single array has a number of advantages, since they can be manipulated by a single set of promises just by varying the indices used to access them. To make use of this array, you have to pass it as an argument to a bundle. One of the most useful functions in this technique is getindices(), which returns a list containing the indices of the given array, and can be used to produce an enumeration of the elements over which to iterate. The complementary function to get just the values is getvalues(). For example, consider this bundle:

To pass arrays as arguments we must pass a string with the name of the array, and then dereference it inside the function. The argument we are passing to set_config_values() is “configfiles.sshd”, which refers to the sshd array defined in the configfiles() bundle. The dereferencing happens in the set_config_values() bundle:

This bundle receives the name of the array as the “v” parameter, so we dereference the array and its values by using $(v) wherever we would normally use the array name. For example, to loop over the array elements using the indices stored in the $(index) list, we use $($(v)[$(index)]) instead of $(configfiles.sshd[$(index)]).

To group name/value sets into named groups, we can use two-dimensional arrays, as in this example:

In this case the dereferencing can get a little more complicated. For example, let us look at some of the code inside the create_users() bundle:

This bundle is being called from the methods: section of the manage_users() bundle, with the string "manage_users.users" as the value of $(info). We use getindices() directly on this value to get a list of the first-level indices of the array (the user names), which we store in @(user). Then we use implicit looping over @(user) to cycle through all those values, and we use the following construction to access individual elements of each user’s data: $($(info)[$(user)][field]). This expands to $(manage_users.users[$(user)][field]), on which implicit looping is applied through the $(user) variable. Remember that parenthesis (or curly braces, they mean the same) are required around the whole expression, so that CFEngine recognizes it properly as a variable reference.

While the syntax looks complicated, this data structure allows great flexibility in passing around and using data structures to be used in configuration operations.

Get a free copy of "Learning CFEngine 3" in exchange for a review

Day Against DRM at O'Reilly: 50% discount on "Learning CFEngine 3" and all other ebooks

Today, May 4th, 2012, O'Reilly Media is participating in support of the FSF's Day Against DRM. All O'Reilly ebooks and videos are DRM-free, and to celebrate this, you can get 50% off on any purchase by using the code DRMFREE.

CFEngine talks at PICC'12 conference

A full shelf

CFEngine tip #001: Modularize complex policies using methods: calls

As your CFEngine policies grow in complexity, they become harder to read if all your promises are in a single bundle, or if you split them among bundles, to have all those bundles listed in the bundlesequence declaration.

New "reviews" page