Limited Time Promotion! 50% off "Learning CFEngine 3" on Amazon

For one week only, until September 20th, 2017, you can buy the new ebook release of “Learning CFEngine 3” on Amazon at 50% off its list price, US$4.98 instead of the regular US$9.99.

Get it while it’s hot! Buying the ebook on Amazon gives you access to free updates to the book, so you will get all future versions as well.

Chapter 1 of “Learning CFEngine 3” is Online

I have always believed that knowledge is best when freely shared. For this reason I have for a long time wished I could make “Learning CFEngine 3” freely available, and it became possible when the rights of the book were reassigned to me. I am proud to announce that, as promised a few days ago, the first chapter of the book is now available online. Over the next few weeks I will post the next chapters.

Read it now at The Book.

New Release of "Learning CFEngine 3"

It has finally happened: a new release of “Learning CFEngine 3” is here!

Short version

  • The book is no longer published by O’Reilly Media - the new release is published directly by me, and distributed through Amazon as a Kindle ebook, available now. It is also available at a lower price (US$9.99) than the previous release.

  • The book now has an index! This has been a very frequent request since the book was first released.

  • The book will also be available for free in this website! Over the next few weeks, I will publish it one chapter at a time.

  • This release is still focused on CFEngine 3.5.2 (just like the last O’Reilly release), but being self-published, it will receive much more frequent updates as I update it for the most recent releases of CFEngine.

You can get the new release from Amazon immediately (disclaimer: affiliate link): Learning CFEngine 3: Automated System Administration for Sites of Any Size

Long version

Read on if you want the full, juicy details about the new release and the change of publisher.

Over the last few years, O’Reilly has shifted its focus from being primarily a book publisher, to being a much more diverse media publisher – its conferences, videos, tutorials, etc. have become just as important, if not more, than its books. The change was made official when O’Reilly announced that it was stopping the direct distribution of books, ebooks and videos, but it’s a shift many years in the making.

Since last year, O’Reilly decided they would no longer publish new releases of “Learning CFEngine 3”, but at my request, agreed to revert the rights of the book back to me. This means that I can self-publish the book, but I can no longer use the O’Reilly design elements, including the animal in the cover (I consider the current cover to be temporary, and am actively looking for a new cover design). However, among other things, it also means that I can make the book’s content available for free, which is something I have wanted to do for a long time. I have decided to make the full ebook available now, but over the next few weeks, I will publish the chapters of the book in this website. Stay tuned!

The departure from O’Reilly has been an extremely amicable one. O’Reilly is an amazing publisher, and it was always a pleasure working with them. I will always be particularly grateful to my editor, Andy Oram, who helped and supported me through the whole process, from the initial conception of the book until the rights reassignment a few weeks ago.

Why release the book now, before updating its contents to newer versions of CFEngine? The most important reason is: to get it out. I’ve been slowly working on updating the content, but I wanted to get the self-publishing process done, and to make the index available, which I think is a great resource for anyone reading the book. I have also fixed many of the errata in the last release of the book. Eventually I will produce a paperback version of the book, but only after I update the contents. If you want a paper version of the book, you can still get the O’Reilly release from Amazon and other book vendors. If you are a Safari Books Online subscriber, you can read also find it there.

I hope you enjoy this new release. I look forward to hearing from you! Please talk to me in the comments below, or send me a message.

Finally, an update about “Learning CFEngine 3”!

I know, I know. It’s been over two years since the last post, and other than my occasional comment on the IRC channel or mailing list, you couldn’t be blamed to think that I had disappeared from the face of the earth.

But I’m back! And I have a few updates for you:

  • I am finally working again on the new release of the book! It will focus on the upcoming CFEngine 3.7 release, and will of course also talk about all the goodies that were introduced in the 3.6 series.
  • You may notice the new, redesigned website. Most of the contents is the same as before, but the website is now being powered by my own Enwrite, which allows me to post and manage contents from Evernote, making it much easier to keep things up to date. Please let me know if you find anything broken.

I have a few other surprises coming up, but those are the big two for now. Follow @LearningCF3 on Twitter if you want to stay updated.

"Learning CFEngine 3" at LISA'12

Last week I attended the Usenix LISA'12 conference in San Diego, and the awesome people at O'Reilly organized a book signing while we were there. I'd like to share a couple of photos from the event. I had a great time!

The second release of the book has been out for a few weeks now. If you purchased the e-book from O'Reilly, make sure to check your "My books" page at oreilly.com to get the updated version!



Lisa_2012_diego_zamboni Lisa_2012_diego_zamboni_booksi

Second release of "Learning CFEngine 3"

I am happy to announce that as of today, the second release of "Learning CFEngine 3" is available. If you have purchased the e-book version of the book you can get the new release from the "Your Products" page at oreilly.com (pro tip: set up Dropbox synchronization, and you will always have the latest version with you!). And if you haven't purchased it, now it's an excellent time to do it! You get all the fresh content, and all future releases for free.

These are the main updates in this release:
  • The book has been updated to reflect changes introduced up to CFEngine 3.3.9 (the latest stable release as of this writing), including many features that were previously only available in CFEngine Enterprise, such as guest_environments promises, databases promises, and more.
  • The installation instructions now reflect the availability of binary package repositories for many Linux distributions, as well as the “Free 25” Enterprise packages that allow you to try CFEngine Enterprise for free, for up to 25 machines.
  • Vim fans rejoice! A new Appendix, contributed by Neil H. Watson, provides details on how to use Vim to edit CFEngine policy files.
  • This release fixes all known errata, including many typos, omissions, and other numerous miscellaneous things.
I hope you enjoy this update! And if you have any feedback, please feel free to contact me.

CFEngine tip #004: How to bootstrap a CFEngine client

My apologies for the long delay since the last tip! Today we come back with a simple tip: how to bootstrap a CFEngine client.

First, some background

CFEngine is designed to operate in a fully distributed fashion – each CFEngine client (i.e. a machine running cf-agent) can operate fully autonomously, using only the policy files stored locally. For regular operation, cf-agent does not require any type of network connectivity.

In real deployments, of course, maintaining each machine’s policy independently would be impractical, so CFEngine has the concept of a policy hub. In CFEngine Community, the policy hub is simply a file server, a machine from which others can download policy files. This allows having a single point of distribution, so that changes made there can be distributed to a large number of clients. In CFEngine Enterprise, the hub has a more complex role, acting also as an aggregator of data from the clients, for the purposes of reporting and analysis.

All CFEngine clients will copy to their local /var/cfengine/inputs/ directory the contents of /var/cfengine/masterfiles/ in the policy hub, so that is where you should make any changes that you want distributed to all the machines.

Bootstrapping

When a new client is installed, it needs to be told which machine is the policy hub to which it should connect. This is done using the following command:

# cf-agent --bootstrap --policy-server=10.0.2.15

You should, of course, replace 10.0.2.15 with the actual IP address of the hub to which you want to bootstrap. The first machine you install will be the policy hub itself, and it should bootstrap to itself. In this case, you have to use its own external IP address, and not “localhost” or 127.0.0.1.

When you issue this command on the policy hub, you will see a message like this, that indicates the host recognizes itself as a policy hub:

# cf-agent --bootstrap --policy-server=10.0.2.15
** CFEngine BOOTSTRAP probe initiated

   @@@     
   @@@      CFEngine

 @ @@@ @    CFEngine Core 3.3.5
 @ @@@ @   
 @ @@@ @   
 @     @   
   @@@     
   @ @     
   @ @     
   @ @     

Copyright (C) CFEngine AS 2008-2012
See Licensing at http://cfengine.com/3rdpartylicenses

 -> This host is: precise32
 -> Operating System Type is linux
 -> Operating System Release is 3.2.0-23-generic-pae
 -> Architecture = i686
 -> Internal soft-class is linux
 -> No previous policy has been cached on this host
 -> Assuming the policy distribution point at: 10.0.2.15:/var/cfengine/masterfiles
 -> Attempting to initiate promised autonomous services...

 ** This host recognizes itself as a CFEngine Policy Hub, with policy distribution and knowledge base.
 -> The system is now converging. Full initialisation and self-analysis could take up to 30 minutes

R: This host assumes the role of policy distribution host
R:  -> Updated local policy from policy server
R:  -> Started the server
R:  -> Started the scheduler
-> Bootstrap to 10.0.2.15 completed successfully

If you issue it on a client, you will see a similar message, but indicating that it is bootstrapping from a different machine.

Regardless of what you use, after bootstrap you should see /var/cfengine/inputs/ populated with the default set of policy files, and both cf-execd and cf-serverd should be running:

# ps axw | grep [c]f-
16099 ?        Ss     0:00 /var/cfengine/bin/cf-execd
16102 ?        Ss     0:00 /var/cfengine/bin/cf-serverd

Once this has been done, CFEngine will start running every 5 minutes, updating its policy files from the hub, and executing them afterwards.

CFEngine tip #003: The distinction between bodies and bundles

This is an extract from Chapter 3 of "Learning CFEngine 3", where you can also find a much more comprehensive description of bodies and bundles in the CFEngine 3 policy language.

The distinction between bundles and bodies can be confusing at first. Remembering these points may help:

  • Bodies are named groups of attributes, whereas bundles are collections of promises. Promises are the units that actually do something in CFEngine (for example, run a command or add a line to a file), whereas attributes specify characteristics of how things are done (for example, whether to run the command in a shell, or where in the file to add the line).

  • The value of an attribute can be a basic data type (string, integer, list, etc.), it can be the name of a body, or it can be the name of a bundle.

  • The type of an attribute’s value is fixed, and determined by the attribute itself (for example, the value of the depth_search attribute in a files: promise is always a body, and the value of an edit_line attribute is always a bundle).

  • For bodies and bundles, their type is always the name of the attribute to which they correspond. For example, bodies to be used with the depth_search attribute are always declared as “body depth_search xyz”, where xyz is an arbitrary name of your choosing. The same goes for bundles: bundles to be used with the edit_line attribute are always declared as “bundle edit_line xyz".

    There are only four types of “top level” bundles that are not used as arguments to attributes: agent, server, knowledge and monitor.

  • The promise types (sections) that can appear in a bundle are determined by the bundle type. For example, commands: promises can only appear in bundles of type agent. 

CFEngine tip #002: How to pass arguments to bundles using arrays

(This tip is based on a section from Chapter 5 of Learning CFEngine 3.)

Many system configuration tasks require groups of name-value pairs as arguments. For example:

  • Editing configuration files in which parameters and their values need to be stored (ssh configuration files, Windows-style INI files, etc.)
  • Setting user parameters. In this case, sets of name-value pairs (home directory, full name, shell, etc.) are associated with a single user, identified by name.

Having sets of related values in a single array has a number of advantages, since they can be manipulated by a single set of promises just by varying the indices used to access them. To make use of this array, you have to pass it as an argument to a bundle. One of the most useful functions in this technique is getindices(), which returns a list containing the indices of the given array, and can be used to produce an enumeration of the elements over which to iterate. The complementary function to get just the values is getvalues(). For example, consider this bundle:

{% gist 2605453 arguments1.cf %}

To pass arrays as arguments we must pass a string with the name of the array, and then dereference it inside the function. The argument we are passing to set_config_values() is “configfiles.sshd”, which refers to the sshd array defined in the configfiles() bundle. The dereferencing happens in the set_config_values() bundle:

{% gist 2605453 set_config_values.cf %}

This bundle receives the name of the array as the “v” parameter, so we dereference the array and its values by using $(v) wherever we would normally use the array name. For example, to loop over the array elements using the indices stored in the $(index) list, we use $($(v)[$(index)]) instead of $(configfiles.sshd[$(index)]).

To group name/value sets into named groups, we can use two-dimensional arrays, as in this example:

{% gist 2605453 users_use.cf %}

In this case the dereferencing can get a little more complicated. For example, let us look at some of the code inside the create_users() bundle:

{% gist 2605453 sys_create_users.cf %}

This bundle is being called from the methods: section of the manage_users() bundle, with the string "manage_users.users" as the value of $(info). We use getindices() directly on this value to get a list of the first-level indices of the array (the user names), which we store in @(user). Then we use implicit looping over @(user) to cycle through all those values, and we use the following construction to access individual elements of each user’s data: $($(info)[$(user)][field]). This expands to $(manage_users.users[$(user)][field]), on which implicit looping is applied through the $(user) variable. Remember that parenthesis (or curly braces, they mean the same) are required around the whole expression, so that CFEngine recognizes it properly as a variable reference.

While the syntax looks complicated, this data structure allows great flexibility in passing around and using data structures to be used in configuration operations.

Get a free copy of "Learning CFEngine 3" in exchange for a review

Are you a technical blogger? O'Reilly's "Blogger Review Program" allows bloggers to get free ebooks and videos, in exchange for writing reviews of them.

I'd love to see more reviews of "Learning CFEngine 3". If you have a technical blog, and would be interested in reviewing the book, I encourage you to visit the program's website to find out more.

Plus, I'm sure you will enjoy the book, and I would love to read your review!