What’s the problem?

Most of the useful bits of this post are, I imagine, relevant to Rakefiles and Gemfiles in general, not just GitLab CI.

In another post about GitLab CI, I noted that .gitlab-ci.yml can be moved from the repository root. Great. Keep Rootian Tidy.

For what I want to do, there’s at least three other files that will end up there: Rakefile, Gemfile, and Gemfile.lock

The first two are config that I, the developer, supply, the latter is created by bundle install. I’d also quite like to have separate Rakefiles for each step in the CI.

So, this is the one where I stumble around in a poorly lit room called ‘Ruby’ .. that is, because I didn’t bring the torch of experience.

One of a number of posts on GitLab CI, see the gitlab-runner tag.

What’s the solution?

Straight up, here’s the fixes. Commentary later in the post.

This example results in the repository getting linted in a pipeline.

You can

• Give the CI yaml file a name you prefer (tell GitLab)
• Give the Rakefile(s) different names.
• Probably call the Gemfile something else .. but I didn’t easily find a way to rename Gemfile.lock, and doing so might cause confusion.

$ cat ci/definition.yml
before_script:
  - /bin/bundle install --gemfile=ci/Gemfile

puppet-lint:
  stage: test
  script:
    - BUNDLE_GEMFILE=ci/Gemfile /bin/bundle exec rake --rakefile=ci/rakefile_lint lint

$ cat ci/Gemfile
source "https://rubygems.org"

gem "puppet-lint"

$ tail -1 .gitignore
ci/Gemfile.lock

• There are other options for how to handle Gemfile.lock; commentary below, and read the bundle-install man page.

$ cat ci/rakefile_lint 
require 'puppet-lint/tasks/puppet-lint'

Workings and commentary

This all needs to work on the Centos 7 VM I stood up to run GitLab runner.

To get going with Ruby, I added the following packages:

• ruby
• rubygem-bundler
• rubygem-rake

bundle install

This came from the man page.

$ man bundle-install
[..]
OPTIONS
       --gemfile=
              The location of the Gemfile(5) that bundler  should  use.  This
              defaults to a gemfile in the current working directory. In gen‐
              eral, bundler will assume that the location of  the  Gemfile(5)
              is  also  the  project root, and will look for the Gemfile.lock
              and vendor/cache relative to it.

I then wanted to understand what other locations got used.  I started with Gemfile in the root.

$ /bin/bundle install
Fetching gem metadata from https://rubygems.org/............
Resolving dependencies...
Installing rake 12.3.2
Installing puppet-lint 2.3.6
Using bundler 1.7.8
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

Gemfile.lock gets created along side. (See .gitignore below.)
Where do the other things go; because if they go alongside the Gemfile, that’s more mess. Turns out:

$ ls ~/bin
puppet-lint  rake
$ ls ~/.gem/ruby/gems/
puppet-lint-2.3.6  rake-12.3.2

Conclusion: move Gemfile, and pass that in the command line.
Gemfile.lock then also gets created in ci/

Gemfile.lock options

From the GitLab task output.

Reinitialized existing Git repository in /home/gitlab-runner/builds/XCHrAuVS/0/puppet/control-repo/.git/
Fetching changes...
From http://gitlab.example.com/puppet/control-repo
   d65bf96..2796865  gitlabrunner2 -> origin/gitlabrunner2
Checking out 27968654 as gitlabrunner2...
Removing ci/Gemfile.lock
Skipping Git submodules setup

Other than performance, I’m not really sure what the potential side effects of this might be, but I’m going to leave it being recreated for every CI run.

Maybe I get fresh gems all the time, which means that if something changes and breaks my CI, I’ll know straight away.

I could version control the gems:

       --local
              Do not attempt to connect to rubygems.org, instead using  just  the
              gems  already  present  in Rubygems´ cache or in vendor/cache. Note
              that  if  a  more  appropriate  platform-specific  gem  exists   on
              rubygems.org, it will not be found.

Checking it into version control is definitely an option; from the man page:

       1.  A Gemfile.lock is required.

           To ensure that the same versions of the gems you  developed  with  and
           tested with are also used in deployments, a Gemfile.lock is required.

           This  is mainly to ensure that you remember to check your Gemfile.lock
           into version control.

But, if I do these things, I’ll be running the same gem versions for ever more.  I’m going to leave it being deleted and see what happens. I will read the rest of the man page at some point ..

.gitignore

$ git status
# HEAD detached at 3a28598
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#	Gemfile.lock

The previous section shows that the GitLab runner cleans out the repo and deletes this file.

Putting it in .gitignore makes this really easy:  I didn’t try it without.

bundle exec

Lint and / or other tools I wanted to run are designed to go searching for files to check.

Changing directory to ‘ci/’ first would break this, as they’d find nothing. So it had to be a mechanism that redirected bundle’s attention, but kept the working directory the same.

Hat tip: stack overflow.

Buried in the last answer was the clue I needed – the environment variable to specify an alternative location.

ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)

So, this completed the move of the Gemfile

BUNDLE_GEMFILE=ci/Gemfile /bin/bundle exec rake lint

Rakefile

Hat tip: Stuart Ellis – that was a case of googling until I found it !

By accident, I found two ways to specify this. As specified on Stuart’s site:

$ BUNDLE_GEMFILE=ci/Gemfile /bin/bundle exec rake --rakefile ci/rakefile_lint lint
manifests/site.pp - WARNING: string containing only a variable on line 71
manifests/site.pp - WARNING: string containing only a variable on line 80
manifests/site.pp - WARNING: variable not enclosed in {} on line 63
manifests/site.pp - WARNING: variable not enclosed in {} on line 72

.. or alternatively with an ‘=’

$ BUNDLE_GEMFILE=ci/Gemfile /bin/bundle exec rake --rakefile=ci/rakefile_lint lint
manifests/site.pp - WARNING: string containing only a variable on line 71
manifests/site.pp - WARNING: string containing only a variable on line 80
manifests/site.pp - WARNING: variable not enclosed in {} on line 63
manifests/site.pp - WARNING: variable not enclosed in {} on line 72