There’s lots of content already out there about puppet CI testing; this is what I initially came up with using GitLab CI, and with various other improvements such blogged about previously in posts tagged gitlab-runner such as:

• Separate rake files. A lot of examples I found had very long complicated rake files to define different ‘subroutines’ that could then be called. For the beginner, a three line rake file that does one job is easier to understand and maintain.
• Less litter in root of the git repo.

 

Running under a shell executor initially, this is my CI setup.

# ci/definition.yml
before_script:
  - /bin/bundle install --gemfile=ci/Gemfile

stages:
  - syntax
  - lint
  - git

lint:
  stage: lint
  script:
    - BUNDLE_GEMFILE=ci/Gemfile /bin/bundle exec rake --rakefile ci/rakefile_lint lint

syntax-manifests:
  stage: syntax
  script:
    - BUNDLE_GEMFILE=ci/Gemfile /bin/bundle exec rake --rakefile ci/rakefile_syntax syntax:manifests

syntax-templates:
  stage: syntax
  script:
    - BUNDLE_GEMFILE=ci/Gemfile /bin/bundle exec rake --rakefile ci/rakefile_syntax syntax:templates

syntax-hiera:
  stage: syntax
  script:
    - BUNDLE_GEMFILE=ci/Gemfile /bin/bundle exec rake --rakefile ci/rakefile_syntax syntax:hiera

commit-msgs:
  stage: git
  script:
    - /usr/bin/git log --oneline --no-merges HEAD ^origin/development | ci/checkcommits.rb

The final check is a fairly simple script that takes a list of commits generated by git and then does pattern matches against them to apply some standards.

The git command won’t generate any commit messages when merging to development or other later protected branches .. that’s something I need to look at.

The nice way to handle it is to wrap some sort of condition around that job, so it doesn’t run at all for protect branches.

Gemfile ..

# ci/Gemfile
source "https://rubygems.org"

gem "puppet-syntax"
gem "puppet-lint"
gem "puppet", '4.10.12'

Rakefile #1

# ci/rakefile_lint
require 'puppet-lint/tasks/puppet-lint'

PuppetLint.configuration.show_ignored = true

Rakefile #2

# ci/rakefile_syntax
require 'puppet-syntax/tasks/puppet-syntax'

PuppetSyntax.fail_on_deprecation_notices = false
PuppetSyntax.check_hiera_keys = true

PuppetSyntax

I’ve broken out checking of hiera, manifests, and templates into three separate tests. Mostly to make my CI a little more complicated.

Puppet syntax has a parameter for disabling hiera checks.

PuppetSyntax.check_hiera_keys = false

If you’re explicit about what to check, this parameter does nothing, eg: syntax:manifests will never check hiera.  This means all the syntax checks could use the same rakefile.

I’ve got it set to true, probably unnecessarily, to make sure it does the checks under syntax:hiera.

 

Stages

https://docs.gitlab.com/ee/ci/yaml/README.html#stages

The checks run in three stages.

• All three syntax checks need to pass.
• If they pass, the lint check runs. I figure that there’s no point checking faulty code for style.
• The final stage will run if that passes, and checks git commits, which I figure sits logically after the success, only, of the other stages.

 

Gemfile.lock

Every GitLab job runs off a fresh fetch of the git repository; which cleans out ci/Gemfile.lock

As a result, the bundle install always has something to do in every job.

Since I started looking at this, I’ve been looking for some steer on whether or not to put it in git, and decided to wait and see.  Alex Harvey’s comment here ..

to keep up to date with upstream in all of these tools (recommended), then git-ignore it

.. confirms that I’ll leave it as is.  It slows things down a bit, but the next step is to use containers anyway, and the obvious thing to do is to populate the gems in the container image.

Sources

https://wikitech.wikimedia.org/wiki/Puppet_coding/testing shows output of rake -T which indicates that syntax checking can be broken down

bundle exec rake -T
[..]
rake lint                  # Run puppet-lint
[..]
rake syntax                # Syntax check Puppet manifests and templates
rake syntax:hiera          # Syntax check Hiera config files
rake syntax:manifests      # Syntax check Puppet manifests
rake syntax:templates      # Syntax check Puppet templates
[..]

That page also documents some debugging options (mostly for spec tests – aimed at modules)