Following up on my post about troubleshooting commit signing, I wanted to figure out how to create a typical certificate authority (with an intermediate) then sign a signing certificate, and generate a verified commit. Hat tip to the GitLab Development Kit docs for the seed capital.
Create certificates
# create CA root: ca_root.crt
openssl req -new -x509 \
-newkey rsa:4096 -keyout ca_root.key \
-subj "/CN=ca_root" -days 3650 \
-out ca_root.crt \
-addext 'subjectAltName = DNS:ca_root.example.com' \
-addext 'crlDistributionPoints = DNS:example.com,URI:http://example.com/crl.pem'
# create CA intermediate: ca_sign.crt
openssl req -new \
-newkey rsa:4096 -keyout ca_sign.key \
-subj "/CN=ca_sign" \
-out ca_sign.csr
openssl x509 -req \
-in ca_sign.csr -days 1825 \
-out ca_sign.crt \
-CAkey ca_root.key -CA ca_root.crt \
-CAcreateserial \
-extfile <(
echo 'basicConstraints = critical, CA:true, pathlen:0'; \
echo 'subjectAltName = DNS:ca_sign.example.com'; \
echo 'subjectKeyIdentifier = hash'; \
echo 'authorityKeyIdentifier = keyid'; \
echo 'crlDistributionPoints = DNS:example.com,URI:http://example.com/crl.pem' )
# create code signing certificate: codesign.crt
# change email:test2@example.com to the correct email address!
openssl req -new \
-newkey rsa:4096 -keyout codesign.key \
-subj "/CN=codesign" \
-out codesign.csr
openssl x509 -req \
-in codesign.csr -days 365 \
-out codesign.crt \
-CA ca_sign.crt -CAkey ca_sign.key \
-CAcreateserial \
-extfile <(
echo 'subjectAltName = email:test2@example.com'; \
echo 'keyUsage = critical,digitalSignature'; \
echo 'subjectKeyIdentifier = hash'; \
echo 'authorityKeyIdentifier = keyid'; \
echo 'crlDistributionPoints=DNS:example.com,URI:http://example.com/crl.pem')
- Provide a password whenever prompted. When testing, it’s easier to just using the same one. This will mean you’ll need to provide the password:
- Whenever you try to use the private key for the root or the intermediate. Since you’re going to need to trust the root temporarily, you should protected your own HTTPS security by ensuring that you know what certificates are signed by this CA.
- Whenever you try to use the code signing private key, including when you load it GPG for codesigning.
- GPG will prompt to lock it up again, provide the same password.
- These commands were tested with OpenSSL 1.1.1f (Ubuntu 20.04). I also did some testing with OpenSSL 3.0.7 (AlmaLinux 9) and it seems like there may be some quality of life improvements and better default behaviour. I could issue the whole chain with openssl req. But I didn’t fully test it.
- addext with openssl req seems to work a little differently from extfile with openssl x509. Specifying keyUsage, subjectKeyIdentifier, and authorityKeyIdentifier resulted in these being duplicated in the root certificate (it really does add them, not just update them) with two different values for keyUsage, since I tried to specify pathlen:1 .. which was strange.
- Some of the settings provided are to ensure all of GitLab’s signature verification checks pass. My starting point is the testing instructions in the GDK docs.
- To view the certs:
openssl x509 -text -noout -in ca_root.crt openssl x509 -text -noout -in ca_sign.crt openssl x509 -text -noout -in codesign.crt
Server setup
- GitLab needs to trust the root to verify commits made with the signing certificate. Copy the root certificate to:
/etc/gitlab/trusted-certs
and run:
gitlab-ctl reconfigure gitlab-ctl restart
- I wanted to make sure that Sidekiq was trusting the new certificate, and it hadn’t got restarted after adding it. The restart might not be essential.
Client setup
- Check these are available, install if missing:
which pinentry which gpgsm
- Export the signing key and certificate; import everything to gpg
openssl pkcs12 -export -inkey codesign.key -in codesign.crt -name test -out codesigning.p12 gpgsm --import ca_root.crt gpgsm --import ca_sign.crt gpgsm --import codesigning.p12
- By importing in this order, you don’t get warnings about establishing trust.
- Get gpg to trust the root
- Ubuntu 20.04 ships gpgsm 2.2.19 – rather than showing the fingerprint as sha fpr it shows it as fingerprint.
- If you imported the root first, the following command will trust the root.
# check the root is first # check also if your version uses 'sha fpr:' or 'fingerprint:' gpgsm --list-keys # trust the root gpgsm --list-keys | grep 'fingerprint' | head -1 | \ awk -F 'fingerprint: ' '{ print $2 " S relax" }' >> ~/.gnupg/trustlist.txt # for testing, skip this. the certificate revocation details in the certs are fake .. echo "disable-crl-checks" >> ~/.gnupg/gpgsm.conf
- For testing, don’t change your ~/.gitconfig. I suggest instead configuring just the repository you’re testing on, that is –local:
# this will only work inside a git repo. git config --local gpg.program gpgsm git config --local gpg.format x509 git config --global commit.gpgsign true
- From the gpgsm –list-keys output above, identify the ID of the code signing certificate, for example:
ID: 0xD50D2350 S/N: 5F5D8F794AFEDADFFFFC0A4F14A39EC00A429545 Issuer: /CN=ca_sign Subject: /CN=codesign aka: mycommits@example.com - Configure Git to use that ID
git config --local user.signingkey 0xD50D2350 # view config git config --list --show-origin
- Reload. Do this after changing the keyring or any configuration.
gpgconf --reload gpg-agent
Commit!
- commit.gpgsign should ensure Git tries to sign commits, but it can also be specified in the paramters:
-S[<keyid>], --gpg-sign[=<keyid>] GPG-sign commits. The keyid argument is optional and defaults to the committer identity; if specified, it must be stuck to the option without a space. - Test a commit. Using GIT_TRACE and commands from my earlier post to aid with troubleshooting.
GIT_TRACE=1 git commit -S -a -m 'x509 test commit' # check there's a signature GIT_TRACE=1 git log --show-signature # troubleshoot if signatures aren't being generated echo foo | gpgsm --status-fd=2 -bsau 0xD50D2350 git push -u origin HEAD
- Check if it is flagged as ‘verified’ in the UI!
Background: CAcreateserial
In my testing, I noticed that the root was issued with a nice long random serial number. It’s easy to do this because openssl req -x509 behaves differently from openssl x509 -req.
I tried to get the same behaviour for the other certificates, by removing:
-set_serial 1
And then ran into:
ca_root.srl: No such file or directory
140549622408512:error:06067099:digital envelope routines:EVP_PKEY_copy_parameters:different parameters:../crypto/evp/p_lib.c:93:
140549622408512:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:69:fopen('ca_root.srl','r')
140549622408512:error:2006D080:BIO routines:BIO_new_file:no such file:../crypto/bio/bss_file.c:76:
CAcreateserial fixes this, from man openssl-x509:
X509(1SSL)
-CAcreateserial
With this option the CA serial number file is created if it does not exist: it will contain the
serial number "02" and the certificate being signed will have the 1 as its serial number. If the
-CA option is specified and the serial number file does not exist a random number is generated;
this is the recommended practice.
Using this creates a .srl file for the root and for the intermediate, and achieves my goal of a random serial number seed for every certificate.
Background: pathlen:0
This is specified in the CSR signing of the intermediate.
What this does is it prevents the intermediate from issuing CA certificates:
X509v3 Basic Constraints: critical CA:TRUE
I wasn’t bothered about this for testing, except that once I had a test chain loaded into GPG I ran into issues that was preventing code signing:
$ gpgsm --list-keys
/home/ben/.gnupg/pubring.kbx
----------------------------
ID: 0x025B0134
S/N: 01
Issuer: /CN=ca_root
Subject: /CN=ca_sign
aka: (dns-name ca_sign.example.com)
validity: 2024-02-07 16:03:37 through 2029-02-05 16:03:37
key type: 4096 bit RSA
chain length: unlimited
fingerprint: 6B:44:30:4D:2F:C0:D6:EF:1E:2D:E8:66:15:20:15:BD:02:5B:01:34
ID: 0x06A78E1E
S/N: 1E2CC30790B9820109B447EF3C45903170AB1D0D
Issuer: /CN=ca_root
Subject: /CN=ca_root
aka: (dns-name ca_root.example.com)
validity: 2024-02-07 15:47:26 through 2034-02-04 15:47:26
key type: 4096 bit RSA
chain length: [error: Duplicated value]
fingerprint: CA:DA:CE:C7:3D:2B:9F:03:EB:CD:C2:ED:B4:CB:FD:F4:06:A7:8E:1E
$ echo foo | gpgsm --status-fd=2 -bsau 0x95FAC778
gpgsm: dirmngr cache-only key lookup failed: Not found
gpgsm: issuer certificate {9DC109E5FB89965E4E278268D2CF78B7E00588C1} not found using authorityKeyIdentifier
gpgsm: dirmngr cache-only key lookup failed: Not found
gpgsm: error getting authorityKeyIdentifier: Duplicated value
gpgsm: can't sign using '0x95FAC778': Duplicated value
thewatertower.org 
Leave a comment