The file got corrupted is usually a shortcut for .. something changed it. I’ve never really seen files get corrupted without good reason .. like a hardware fault or unexpected powerdown. And the corruption wouldn’t have any sort of random consistency to it.

Update 2019-12-22: Another Puppet related file got damaged. I suspect this hard disk is going home!

This one is weird.

MacBook Pro laptop running Centos7.  Wifi doesn’t ‘work’ – the chipset needs firmware and other incantations; I don’t really have a use for it .. it looked hard .. so I’ve not put in the effort. I’m travelling: machine is hard wired to my sister’s Sky router with a long bit of ethernet cable.

I was trying to do a bit of Puppet development when this happened:

# ./puppetapply.sh  --noop
+ /opt/puppetlabs/bin/puppet apply --test --environment=local --noop /etc/puppetlabs/code/environments/local/manifests
/opt/puppetlabs/puppet/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in 
`require': /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/environments.rb:6: 
syntax error, unexpected end-of-input, expecting ']' (SyntaxError)
      environmentpath = Puppet[G
                                 ^

Which is odd, because it was working fine on the train up here yesterday. Why would a ruby file buried in the puppet agent distribution (now) be failing to run.

I took a copy of the file, and then downloaded and reinstalled the puppet agent package. (Hindsight: I didn’t check the datestamp on the file. D’oh.)

I used strings to clean up the copied and reinstalled files. The corruption is partly binary, and diff won’t print the changes on the originals.

5,6c5,9
<       environmentpath = Puppet[:environmentpath]
<       super("Could not find a directory environment named '#{environment_name}' anywhere in the path: #{environmentpath}. Does the directory exist?", original)
---
>       environmentpath = Puppet[G
> super("Could n
> EE-BrightBox-6eggdp
> 0H`l
>  the path: #{environmentpath}. Does the directory exist?", original)

Two lines have got corrupted, and a string inserted which looks to be a router SSID; There’s a EE Brightbox somewhere in the neighbourhood, just in range, sometimes. The unique component of the one around here is different; but the number of characters is the same as the string I’ve found in the file.

So, I can believe that there’s a router out there somewhere with that name, but how on earth did its name get inside my laptop’s operating system at all, and how did it then manage to get injected into this file?

I can pinpoint roughly when it happened

# journalctl --unit=puppet.service
[snip]
Sep 12 20:51:39 systemd[1]: Stopping Puppet agent...
Sep 12 20:51:39 systemd[1]: Stopped Puppet agent.
-- Reboot --
Sep 13 15:21:22 systemd[1]: Started Puppet agent.
Sep 13 15:21:27 puppet-agent[1208]: Starting Puppet client version 4.10.12
-- Reboot --
Sep 13 15:32:30 systemd[1]: Started Puppet agent.
Sep 13 15:32:30 puppet[1209]: /opt/puppetlabs/puppet/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/environments.rb:6: syntax error, unexpected end-of-input, expecting ']' (SyntaxError)
Sep 13 15:32:30 puppet[1209]: environmentpath = Puppet[G
Sep 13 15:32:30 puppet[1209]: ^

The machine was found to be powered off today as well. It does sometimes overheat, so I put it down to that.

Sep 13 15:28:52 systemd-udevd[557]: worker [577] terminated by signal 9 (Killed)
-- Reboot --
Sep 13 15:31:53 systemd-journal[106]: Runtime journal is using 8.0M (max allowed 386.6M, trying to leave 580.0M free of 3.7G

I’ve run rpm -Va on on the machine. MD5 changes (indicated with ‘5’) are what I’d be looking for, along with maybe file size (S). The mtime (T) update is probably optional, as there’s no reason for real file corruption to go through proper channels. Interpretation of flags from fedora docs.

S.5....T.  c /etc/puppetlabs/puppet/puppet.conf

A bunch of changes, but nothing with sticks out as odd, or unexpected. Just that one file?

 

Update 2019-12-22

It was ok two days ago! Odd that it’s bits of puppet that keep getting broken. I guess there’s a physical part of the disk with a fault, and puppet happens to be written there.

-- Reboot --
Started Puppet agent.
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/provider/user/useradd.rb:97: [BUG] Segmentation fault at 0x00000000000000
ruby 2.1.9p490 (2016-03-30 revision 54437) [x86_64-linux]

Check with rpm again ..

#  rpm -Va
[..]
..5......    /opt/puppetlabs/puppet/lib/libfacter.so.3.6.10
[..]
# /bin/yum reinstall puppet-agent
# md5sum /opt/puppetlabs/puppet/lib/libfacter.so.3.6.10 /root/libfacter.so.3.6.10
cc5dff795a911f36793dd07a4c98b97b  /opt/puppetlabs/puppet/lib/libfacter.so.3.6.10
b2e731dc1d9870731509ca4eb776b063  /root/libfacter.so.3.6.10