Everything you wanted to know about GPG – but were scared to ask

Each year, around the New Year Holiday, I get to re-learn GPG in all its glory. I’ve used GPG for many years and have marveled at how well it works (when it does), yet how hard it is to get setup right. Each year, I re-read my notes from the previous year and renew my keys for one more year.

So here is a summary of my notes – maybe it’ll help you understand GPG just a little bit better.

What is GPG?

GPG is an open source implementation of the OpenPGP protocol. It is available on Windows, Linux, Mac, and Android. On Windows, I have found Gpg4win to be a fine product (donations requested). On Linux and Android, it is likely a simple matter of installing gnupg with your package manager of choice. On Android, I use termux so it is as simple as

pkg install gnupg

On Linux it is likely one of

sudo apt-get install gnupg

sudo yum install gnupg

On the MAC I use brew, so it is just

brew install gnupg

How PKI works

We now see how a simple PKI implementation works. PKI is an acronym for Public Key Infrastructure.

In a PKI system, a user creates a keypair which consists of a public and private key, and then shares the public key widely. The user protects the private key very securely. Private keys are often protected with an additional “passphrase”. This is shown at left (see Figure 1).

Reversible operations

The essence of PKI is that an operation performed on a bytestream using the public key is deterministic, fast, and only reversible with the private key. This is shown below. It is generally the case that there is nothing specific that distinguishes the private key from the public key – beyond a choice at keypair creation time. This reversibility is shown next (See Figure 2).

Figure 2. The reversibility of operations with public and private keys.

On the upper line, an input bytestream is encrypted using the public key to produce some cipher text. That cipher text can then be decrypted using the private key. On the lower line, the same input bytestream is encrypted using the private key to produce cipher text. That cipher text can be decrypted using the public key. Unlike symmetric key cryptography where the operations “encryption” and “decryption” are opposites, in asymmetric key cryptography the operations achieve a reversal but not by performing the operations in reverse.

Signing with PKI

The two operations one performs are signing and encryption. First, here’s signing, see Figure 3 below.

Figure 3. Signing and Verification with PKI

In signing, Alice computes a cryptographic hash of an input bytestream. Alice then takes that cryptographic hash, some optional metadata about the bytestream, and maybe additional information like the date and time and encrypts it using her private key. The recipient of the hash (Bob) has the corresponding public key that Alice has distributed. Bob takes the hash and decrypts it using Alice’s public key. This produces the cryptographic hash, and any metadata that was included in the signature. Bob can also compute the cryptographic hash on the same input bytestream and verify that computation. If the cryptographic hashes match, it indicates to Bob that the signature was in fact generated by Alice.

Encryption in PKI

Encryption is very similar, and shown next (See Figure 4 below).

Figure 4. Encryption and Decryption using PKI.

In Figure 4, Alice wants to encrypt a document for Bob. To do this, Alice encrypts the input bytestream using Bob’s public key, and transmits that ciphertext to Bob. Since Bob is the only person who has the corresponding private key, Bob can decrypt the ciphertext and regenerate the input bytestream.

Signing and Encryption in PKI

Putting all of this together, we illustrate (in Figure 5) how encryption and signing are done together.

Figure 5. Encryption and Signing together with PKI.

Alice wishes to send some bytestream securely to Bob. For this, Alice computes a signature (computes cryptographic hash of the bytestream and encrypts using her private key) and encrypts the bytestream using Bob’s public key. The ciphertext and the signature are communicated to Bob. Bob can verify the signature and decrypt the data.

Importantly, if anyone intercepts the communication, they are powerless to do anything. Not having Bob’s private key, they can’t decrypt the ciphertext. They can decrypt the signature (as they could also have Alice’s public key). However all they’ll have is a cryptographic hash of the input ciphertext.

How GPG works

GPG is an implementation of OpenPGP, a framework for encrypting, decrypting, and signing messages, and for storing and exchanging public keys. It is a Public Key Infrastructure (PKI) system with some novel twists.

GPG keypairs

A GPG Key is a little bit more complicated than a simple PKI key shown above. Figure 6 below shows the three kinds of GPG keys you will see referenced later.

The three keypairs shown above are Alice’s keypairs. First (top left) is Alice’s “top-secret” keypair. This is the one that Alice guards most carefully, it is rarely ever used, and something that is stored in a vault or some such very safe place. It is further protected with a passphrase.

This top-secret keypair contains three PKI keypairs. These are the master keypair, the signing keypair and the encryption keypair. Each has a private and a public key. The signing and encryption keypairs are signed using the private key of the master keypair. The master keypair is used only to sign and certify the other two keypairs.

If you remove the private key from the master keypair, you get a keypair that is called the “laptop” keypair, and this is the one that Alice would use daily. It is also protected by a passphrase, and good practice is to have a different passphrase than the master keypair.

Finally, the three public keys from the three keypairs are called the “GPG Public Key” and this is the one that Alice shares widely. The public keys here are signed using the private key in the master keypair. Anyone (say Bob) who receives this public keypair can verify that signature (using the public key from the master keypair).

Signing in GPG

With that in place, let us look at signing and encryption in GPG.

Figure 7. Signing and Verification in GPG.

Alice signs a bytestream using her signing private key. Bob receives this signature and can verify it using the signing public key. Since the signing public key is signed (by Alice) using her master private key, Bob can verify the signing public key is authentic using the master public key.

Encryption in GPG

Alice wishes to encrypt a file for Bob. She has Bob’s GPG Public key that contains a public encryption key. She encrypts the bytestream using Bob’s public encryption key and sends the ciphertext to Bob. Bob (and only Bob) can decrypt it using his private encryption key.

Figure 8. Encryption and Decryption using GPG

Putting it all together with GPG

Finally, let’s put this all together and show how this works in GPG. See Figure 9 below.

Alice wants to send a message to Bob. For this, she has Bob’s public GPG keypair. First, she generates a session key for use with some symmetric cipher technique. She encrypts that symmetric key (the session key) using Bob’s public encryption key. Using that session key, she encrypts the bytestream and generates ciphertext. She signs the bytestream and generates a signature. She transmits the encrypted session key, the ciphertext and the signature to Bob over a (potentially) insecure channel.

Bob receives the three items above and decrypts the session key using his encryption private key. With the session key, he decrypts the ciphertext. Finally he computes and verifies the signature.

So there you have it, that’s GPG.

GPG peculiarities

Why does GPG use subkeys?

The GPG Keypair shown above consists of three different keypairs. The encryption and signing keys are called subkeys. These keys have no use by themselves (divorced from the master keypair).

In GPG, the master key is used to certify the subkeys. The public keys are shared widely (such as on key servers). The master key is equivalent to the owner’s “identity”. It is setup once, and hopefully never changed. On the other hand, from time to time, a person may rotate their signing and encryption keys. Over time, different documents could be signed and encrypted using different subkeys. However, all of these keys are certified by the same master keypair.

Why a “top-secret” and a “daily” key?

As above, the master keypair is the thing that protect’s the owner’s “identity”. The private key in the master keypair is used only to certify the subkeys. Therefore, it is not used on a day to day basis. Having a “top-secret” key with a different passphrase than the “daily” or “laptop” key is therefore a good practice.

Code and Command samples

Here are some code and command samples of common GPG operations.

Making a RAMDISK

It is never a good idea to store your master private key on persistent storage. I always work on the master private key on a secure machine that is air-gapped. The master private key is stored only on a ramdisk. On a MAC, shell scripts have this preamble.

#!/usr/bin/env bash

diskutil erasevolume HFS+ 'gpg-ephemeral-disk' `hdiutil attach -nomount ram://32768`

pushd /Volumes/gpg-ephemeral-disk
export GNUPGHOME=/Volumes/gpg-ephemeral-disk/gpg
mkdir ${GNUPGHOME}

chmod 700 ${GNUPGHOME}

The first line makes a ramdisk and the rest of the lines setup a temporary GPG environment that stores all data on this ramdisk.

Why a shell-script? Most of these operations are done infrequently and having shell scripts is a good way to “document” it for myself.

Making a keypair

I make my keypair using a shell-script like this one.

#!/usr/bin/env bash

cat > ./keygen.txt <<EOF
%echo Generating a basic OpenPGP key
Key-Type: RSA
Key-Length: 4096
Key-Usage: sign, cert
Name-Real: "Amrith Kumar - Test tester@tester"
Name-Comment: Not for production use
Name-Email: tester@tester
Expire-Date: 0
%commit
%echo done
EOF

gpg --batch --generate-key ./keygen.txt

That generates the master keypair as an RSA keypair with a key length of 4kb (the maximum). This key is used only for signing and certification. It is set to never expire.

Now, I can add the subkeys to this keypair.

#!/usr/bin/env bash

keyid=`gpg --list-secret-keys --keyid-format 0xlong --with-colons | grep 'sec:u:4096' -A 1 | grep fpr | sed 's/fpr//' | sed 's/://g'`

gpg --quick-add-key ${keyid} rsa4096 sign 20240101T000000
gpg --quick-add-key ${keyid} rsa4096 encrypt 20240101T000000

That generates the two subkeys, one for signing and one for encryption. It sets both of them to expire on January 1st, 2024 (and this is the reason why I get to relearn all of this stuff around the New Year holiday).

Another way of making the master keypair is to use python-gnupg.

#!/usr/bin/env python3

import gnupg

gpg = gnupg.GPG(gnupghome='/Volumes/gpg-ephemeral-disk/gpg')

# gpg.verbose = True

# WARNING: This generates a master-key with no passphrase.
# In practice you will put a passphrase on it later.

new_key = gpg.gen_key_input(key_type='RSA', key_length=4096,
                            name_real='Amrith Kumar (test key)',
                            name_email='tester@tester',
                            name_comment='Not for production use',
                            expire_date=0, no_protection=True,
                            key_usage='sign, cert')

key = gpg.gen_key(new_key)

encrkey = gpg.add_subkey(key.fingerprint, algorithm='rsa4096',
                         usage='encrypt', expire='20240101T012345')

signkey = gpg.add_subkey(key.fingerprint, algorithm='rsa4096',
                         usage='sign', expire='20240101T012345')

Renewing the subkeys each year

Each year, you have to move the expiry date on the subkeys forward (a year). Here’s what I do. You need to do this using the master keypair

#!/usr/bin/env bash

signkeyid=`gpg --list-keys --keyid-format 0xlong --with-colons | grep 'sub:u:4096' -A 1 | grep ':s:' -A 1 | grep fpr | sed 's/fpr//' | sed 's/://g'`

encrkeyid=`gpg --list-keys --keyid-format 0xlong --with-colons | grep 'sub:u:4096' -A 1 | grep ':e:' -A 1 | grep fpr | sed 's/fpr//' | sed 's/://g'`

gpg --quick-set-expire ${keyid} 20260101T012345 ${signkeyid}
gpg --quick-set-expire ${keyid} 20260101T012345 ${encrkeyid}

Another way to get the key fingerprints is this

#!/usr/bin/env bash

gpg --list-keys --with-fingerprint --with-subkey-fingerprint | grep '^sub' -A 1| grep '\[S\]' -A 1 | tail -n 1 | sed 's/ //g'

gpg --list-keys --with-fingerprint --with-subkey-fingerprint | grep '^sub' -A 1| grep '\[E\]' -A 1 | tail -n 1 | sed 's/ //g'

gpg --list-keys --with-fingerprint --with-subkey-fingerprint | grep '^pub' -A 1| egrep '\[SC\]|\[CS\]' -A 1 | tail -n 1 | sed 's/ //g'

Making the “daily” or “laptop” keypair

This is how you transform the master keypair into the daily keypair.

gpg --armor --export-secret-keys tester@tester > tester.top-secret-key.gpg

gpg --armor --export-secret-subkeys tester@tester > tester.laptop.gpg

gpg --delete-secret-key tester@tester

Now save the laptop and top-secret keypairs somewhere safe, destroy the ramdisk, and restart the machine you were using.

Setup on Daily use machine

Import the daily use keypair on the daily use machine.

gpg --import tester.laptop.gpg

Conclusion

I’ve skipped over some of the more mundane things like changing the passphrase, encryption, signing, verification and decryption. There are numerous links online that illustrate those 🙂