puppet apply #7 – classification via hiera

Hiera introduction

Hiera is used to store data. Roles and profiles are about code reusability, and separating data and code is probably the other main pillar of Puppet best practise.

Hiera layers the data, allowing you to selectively mask values with others based on properties of the puppet client.  The trick is to pick useful properties and sequence them in an appropriate order;  starting with the most generic layer, and getting more and more specific.

Hiera stores its data, usually, as YAML in the data/ subdirectory of control repo. (Other storage formats are available; and if you find yourself on a ‘mature’ site, you will find a hieradata subdirectory.)

  • The base, most generic, layer is data/common.yaml. Here you specify data and values for the majority case.
  • The next layer might be the role. This allows you to assign different or additional values for database servers, web servers, elasticsearch servers, and so on.
  • Another layer might be the hostname. For those irritating corner cases.

Using the hostname tier is kind of frowned upon.  If it’s getting heavy use, that suggests you have your roles poorly defined. If server ‘a’ and server ‘b’ really need loads of different configuration, are they really doing the same job?

How to use hiera (well) is a topic in its own right.

More info on using hiera

Puppet has documentation exploring how to set up and use hiera, such as this page, which was written for the older hiera v3. It includes examples for different properties of systems you might want to include in your hierarchy. That page links to the more recent docs as well.

Using hiera for classification

Hiera can store simple values for your profiles. It can store data structures; lists (arrays) and hashes. It can also merge these data structures across the hierarchy instead of masking. This makes it useful for determining what profiles a given server needs.

In an earlier post, I quoted puppet guidance on what role definitions should contain

  • The only thing roles should do is declare profile classes with include. Don’t declare any component classes or normal resources in a role.Optionally, roles can use conditional logic to decide which profiles to use.

The first sentence basically means a role class should look like this:

# site/role/manifests/webserver.pp
class role::webserver {
  include profile::baseline
  include profile::filesystems
  include profile::apache
}

The final sentence says .. you can put some logic in there.

I will suggest the following approach:

  • Put the logic somewhere else.
  • All that’s left is include statements, so it’s basically a list. So, it’s data, not code.

It’s not my idea, it’s documented by Puppet, such as for the older hiera v3.

It is very common to have a ‘base’ or ‘baseline’ profile.  Puppet include a dummy one in their example control repo.  It’s used to apply the same set of common changes (with logic as needed) across all your systems.

For example: log forwarding, hardening, installation of agents, a standard message of the day, etc. Generally, the baseline profile turns into another list of ‘include’ statements.

So, we end up with two lists: one that is applied to all systems (profile::baseline) and lists that are applied to servers based on roles (eg: role::test1)

Within hiera, we can have places for that

  • Common.yaml for the ‘base’ data.
  • a yaml file for each role.
  • List the classes in each place, merge them together, and hey presto: base plus plus.

Revised control repo

git checkout puppetapply05

A lot of changes here. Most of them should be incremental changes from stuff you’ve seen already.

hiera.yaml

  • Puppet’s example control repo has two levels:  common.yaml and trusted.certname
  • I’ve changed this to support role, hostname in place of trusted.certname, and also add puppetapply.yaml at the top, which will be covered in the next post. I’ve replaced trusted.certname so it’s entirely transparent in the example control repo what that layer in the hierarchy relates to. It saves a long explanation.

manifests/site.pp

  • Now asks hiera for classes to include. It uses the puppet 4.x ‘lookup’ mechanism documented here. The 3.x hiera docs linked above refer to hiera_include which is deprecated.

data/common.yaml

  • The ‘classes’ array in this file is the baseline.
  • The lookup_options data structure is a way of defining the behaviour of hiera within hiera.  It replaces the lookup parameter specified in the 4.x lookup documentation linked above (the part of the code in blue below.)
lookup('classes', {merge => unique}).include

data/role/role::test1.yaml

  • The replaces the balance of site/role/manifests/role::test1.pp
  • The classes array here adds in the scope dumping class.
  • There is no mechanism to assign roles to servers yet, so this is just done to illustrate the baseline/role split discussed above: role::test1 is a test server role, so include a profile useful for developing puppet code.

site/profile/manifests/motd/*

  • These classes have been updated with a ‘motd_behav’ data switch. I’ll use this to illustrate driving profiles using data in a future post.
  • common.yaml sets this switch to ‘ignore’ which means the code is inactive by default, and will leave the motd files as it finds them.
  • The switch can also be sent to ‘absent’ which means it’ll delete the motd files.
  • A new profile, profile::motd::role has been added ready for post #9

site/profile/manifests/puppet/factsd.pp

  • Added ready for post #9.

If you now run puppet apply, it won’t do a lot.

  • motd is included in baseline (common.yaml) but is deactivated in common.yaml via the motd_behav switch
  • clientscope is included in the test1 role, but there’s no way for puppet to know what role a given system has been assigned. It’ll be added to control-repo by the end of the series, but it’s incomplete at present.

Next

This post was heavy in theory. The next post gets back to the command line and uses hiera to manipulate control repo.

puppet apply #8 – puppet apply and hiera

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s