Key Takeaways
- Correct encryption/decryption is difficult. Please don’t attempt it on your own unless you have a valid reason.
- When utilizing AES, start with CBC mode using the
Aes.Create()method, and if possible, upgrade to GCM mode with theAesGcmclass. - Nonce and IV must be unique within the same encryption key. However, they can be safely transferred along with the ciphertext without extra encryption.
- Secret keys, IVs, and nonces should always be generated using cryptographic randomization.
- Design applications to support key rotation.
Before Starting
Let’s have some agreement on how I use terms in this blog. The target audience of this article is application developers who don’t necessarily have expertise in security. Therefore, specific terms may be unfamiliar to them:
- A cipher is an algorithm for performing encryption or decryption. Two well-known ciphers are AES and RSA.
- The term plaintext refers to the original information that we want to encrypt. Although we call it plaintext, it may be anything that can be represented as an array of bytes: a string, an image, or a document…
- The ciphertext is the encryption process’s output. Despite being called “text,” it is actually an array of bytes.
- By industry standard, I am referring to solutions that adhere to the recommendations of reputable security organizations and standards like the National Institute of Standards and Technology (NIST) or Federal Information Processing Standards (FIPS).
A Naive Implementation
For simplicity, let’s say we need to implement this interface to handle encryption and decryption in a .NET application:
public interface IEncryptionService
{
string Encrypt(string plaintext);
string Decrypt(string cyphertext);
}
The interface is very straightforward and self-explanatory. In the demo code in this post, we focus solely on string encryption, assuming that the original string is UTF-8 encoded. The first version of the concrete class implementation is described below:
public class NaiveEncryptionService : IEncryptionService
{
private readonly byte[] encryptionKey;
private readonly byte[] iv;
public NaiveEncryptionService(byte[] encryptionKey, byte[] iv)
{
this.encryptionKey = encryptionKey;
this.iv = iv;
}
public string Encrypt(string plaintext)
{
byte[] cyphertextBytes;
using var aes = Aes.Create();
var encryptor = aes.CreateEncryptor(encryptionKey, iv);
using (var memoryStream = new MemoryStream())
{
using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
{
using (var streamWriter = new StreamWriter(cryptoStream))
{
streamWriter.Write(plaintext);
}
}
cyphertextBytes = memoryStream.ToArray();
return Convert.ToBase64String(cyphertextBytes);
}
}
public string Decrypt(string cyphertext)
{
var cyphertextBytes = Convert.FromBase64String(cyphertext);
using var aes = Aes.Create();
var decryptor = aes.CreateDecryptor(encryptionKey, iv);
using (var memoryStream = new MemoryStream(cyphertextBytes))
{
using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
{
using (var streamReader = new StreamReader(cryptoStream))
{
return streamReader.ReadToEnd();
}
}
}
}
}
We treat the encryption key and IV as secrets and load them into the application configuration from environment variables. But wait, what is IV? Do we only need a secret key for encryption with AES?
The answer lies in the .NET implementation’s default mode of AES. Although AES is a standard algorithm, it can operate in various modes. In .NET, the default mode is Cipher Block Chaining (CBC). The image below depicts how this mode works.
Before encryption, each block is XORed with the ciphertext of the previous block’s encryption (if you recall, ciphertext simply refers to the name of the encrypted output data). The first block also requires something to be XORed with, too. And that’s the initial vector (IV).
Now that we understand why the IV is required, can you spot what’s wrong with the code snippet above?
We reuse IV in all of our encryption. This weakens and may destroy the AES; thus, it should be avoided. Look at Figure 1; you’ll realize that with the same IV and encryption key, if two plaintexts have the same prefix blocks, their ciphertexts also share the same prefix blocks. We need to prevent this and have improved our implementation by using a random IV.
A Better Version with Random IV Values
Here is the second version of our implementation:
public class RandomIvEncryptionService : IEncryptionService
{
private readonly byte[] encryptionKey;
public RandomIvEncryptionService(byte[] encryptionKey)
{
this.encryptionKey = encryptionKey;
}
public string Encrypt(string plaintext)
{
byte[] cyphertextBytes;
using var aes = Aes.Create();
var encryptor = aes.CreateEncryptor(encryptionKey, aes.IV);
using (var memoryStream = new MemoryStream())
{
using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
{
using (var streamWriter = new StreamWriter(cryptoStream))
{
streamWriter.Write(plaintext);
}
}
cyphertextBytes = memoryStream.ToArray();
return new AesCbcCiphertext(aes.IV, cyphertextBytes).ToString();
}
}
public string Decrypt(string ciphertext)
{
var cbcCiphertext = AesCbcCiphertext.FromBase64String(ciphertext);
using var aes = Aes.Create();
var decryptor = aes.CreateDecryptor(encryptionKey, cbcCiphertext.Iv);
using (var memoryStream = new MemoryStream(cbcCiphertext.CiphertextBytes))
{
using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
{
using (var streamReader = new StreamReader(cryptoStream))
{
return streamReader.ReadToEnd();
}
}
}
}
}
Compared with the previous implementation, we can see some updates:
- The IV is no longer set by us. Now, we rely on the .NET
Aesclass to generate a new random IV whenever we create a new instance fromAes.Create(). - Because the IV is generated randomly, we concatenate it with the AES output as byte arrays before performing the base64 encoding step. It’s safe to expose the IV along with the final ciphertext and transfer them together. This led to a new class called
AesCbcCiphertext, which abstracted the serialization/deserialization process.
public class AesCbcCiphertext
{
public byte[] Iv { get; }
public byte[] CiphertextBytes { get; }
public static AesCbcCiphertext FromBase64String(string data)
{
var dataBytes = Convert.FromBase64String(data);
return new AesCbcCiphertext(
dataBytes.Take(16).ToArray(),
dataBytes.Skip(16).ToArray()
);
}
public AesCbcCiphertext(byte[] iv, byte[] ciphertextBytes)
{
Iv = iv;
CiphertextBytes = ciphertextBytes;
}
public override string ToString()
{
return Convert.ToBase64String(Iv.Concat(CiphertextBytes).ToArray());
}
}
The AesCbcCiphertext class has two public properties: the generated IV bytes and the AES output bytes. We concatenate them before doing base64 encoding in the overridden ToString() method. Additionally, there is a static method to parse a base64 string, extracting the first 16 bytes (128 bits) as the IV and the rest as the AES ciphertext.
At this point, we are using AES, which is an NIST-approved cipher. The usage of IV and CBC mode satisfies the NIST SP 800–38A recommendation. Everything seems to be “industry standard”. However, it’s important to note that CBC mode does have some inherent vulnerabilities and potential for attacks, as mentioned by Microsoft and CloudFlare.
Not only Confidentiality but also Integrity
In the previous examples, our focus was primarily on ensuring information confidentiality. The CBC mode of AES protects the plaintext information from attackers. However, we also want to ensure that the ciphertext we’re decrypting originates from a sender who possesses the secret key rather than an attacker. This is called data authenticity or data integrity.
Even though we can check for data integrity on the plaintext (before the encryption) or on the ciphertext (after the encryption), in this post, we only focus on the encrypt-then-sign paradigm (a.k.a protecting the ciphertext authenticity). There are two common approaches:
- Use a hash function with a secret key on the output of the encryption cipher. We must maintain two secret keys: one for the encryption cipher and another for verifying the hash. For example, we can apply SHA256 with a secret key on the AES CBC cipher output, sometimes referred to as
AES-256-CBC-HMAC-SHA-256. - Use an Authenticated Encryption cipher (AE) that supports both confidentiality and authenticity using only one secret key. Here, we use AES-256 in GCM (Galois/Counter Mode) mode. Fortunately, .NET provides a managed class for GCM, as shown in the snippet below.
public class AuthenticatedEncryptionService : IEncryptionService
{
private readonly byte[] encryptionKey;
public AuthenticatedEncryptionService(byte[] encryptionKey)
{
this.encryptionKey = encryptionKey;
}
public string Encrypt(string plaintext)
{
using var aes = new AesGcm(encryptionKey);
var nonce = new byte[AesGcm.NonceByteSizes.MaxSize];
RandomNumberGenerator.Fill(nonce);
var plaintextBytes = Encoding.UTF8.GetBytes(plaintext);
var ciphertextBytes = new byte[plaintextBytes.Length];
var tag = new byte[AesGcm.TagByteSizes.MaxSize];
aes.Encrypt(nonce, plaintextBytes, ciphertextBytes, tag);
return new AesGcmCiphertext(nonce, tag, ciphertextBytes).ToString();
}
public string Decrypt(string ciphertext)
{
var gcmCiphertext = AesGcmCiphertext.FromBase64String(cipherText);
using var aes = new AesCcm(encryptionKey);
var plaintextBytes = new byte[gcmCiphertext.CiphertextBytes.Length];
aes.Decrypt(gcmCiphertext.Nonce, gcmCiphertext.CiphertextBytes, gcmCiphertext.Tag, plaintextBytes);
return Encoding.UTF8.GetString(plaintextBytes);
}
}
We need another data bag class, AesGcmCiphertext, instead of the AesCbcCiphertext class from the previous sample. Let’s look at the new class:
public class AesGcmCiphertext
{
public byte[] Nonce { get; }
public byte[] Tag { get; }
public byte[] CiphertextBytes { get; }
public static AesGcmCiphertext FromBase64String(string data)
{
var dataBytes = Convert.FromBase64String(data);
return new AesGcmCiphertext(
dataBytes.Take(AesGcm.NonceByteSizes.MaxSize).ToArray(),
dataBytes[^AesGcm.TagByteSizes.MaxSize..],
dataBytes[AesGcm.NonceByteSizes.MaxSize..^AesGcm.TagByteSizes.MaxSize]
);
}
public AesGcmCiphertext(byte[] nonce, byte[] tag, byte[] ciphertextBytes)
{
Nonce = nonce;
Tag = tag;
CiphertextBytes = ciphertextBytes;
}
public override string ToString()
{
return Convert.ToBase64String(Nonce.Concat(CiphertextBytes).Concat(Tag).ToArray());
}
}
Two significant differences are:
- The
IVproperty is replaced byNonce. From the application developer’s point of view, we can think ofNonceas anIVwith a shorter length (96 bits instead of 128 bits). Under the hood, theNonceis concatenated to a 32-bit integer to create the final IV. That 32-bit integer is maintained internally by the cipher. - The new
Tagproperty:Tagallows us to verify the ciphertext without decrypting it. BothNonceandTagmust be transmitted alongside the ciphertext. LikeIV,NonceandTagdo not need to be kept secret.
Unfortunately, GCM mode imposes stricter requirements than CBC mode. While a duplicated IV can harm your AES encryption in CBC mode, violating the uniqueness of the nonce in GCM mode is much more disastrous!
This requirement [uniqueness of nonce within the usage of the same key] is almost as important as the secrecy of the key. — NIST SP 800–38D, section 8.
The tweet below is an excellent example of what an attacker can get when we reuse a nonce with the same key.
Why using CTR mode with the same nonce is a bad idea pic.twitter.com/iZEqx6tFzB
— Ange (@angealbertini) January 21, 2014
So, have we reached the top-notch implementation of AES usage yet?
Key Rotation
Key rotation is a crucial requirement that application developers mostly ignore. It’s inevitable because:
- The bit-length of IV (or nonce) is finite, which means that after a certain number of runs, we will encounter repeated IV values for the same key. And, remember from the previous section, we need to avoid this.
- Compliance standards often mandate the rotation of encryption keys. PropertyGuru, for example, requires the key to be rotated every year.
The most straightforward application that supports key rotations will be designed to:
- Accept a list of encryption keys instead of a single one. Each encryption key will be associated with a unique ID, allowing us to choose the appropriate key for each ciphertext.
- Keep the ID of the key used in the encryption phase encoded along with the rest of the ciphertext. This way, the decryptor will know which key to use for decryption.
- Either (I) keep a history of all keys, or (II) keep only the most recent N keys when a key rotation occurs. If the (II) approach is implemented, the application must upgrade the encrypted data to use the new key before discarding any encryption keys. Otherwise, the data will be lost forever.
The following class is an oversimplified version of an implementation of the encryption service that satisfies the above requirements. As you can see, the class itself uses the previous AuthenticatedEncryptionService class to encrypt/decrypt data with AES GCM.
public class KeyRotationAwareEncryptionService : IEncryptionService
{
private readonly IEncryptionKeyProvider encryptionKeyProvider;
public KeyRotationAwareEncryptionService(IEncryptionKeyProvider encryptionKeyProvider)
{
this.encryptionKeyProvider = encryptionKeyProvider;
}
public string Encrypt(string plainText)
{
var encryptionKey = encryptionKeyProvider.GetCurrentEncryptionKey();
var encryptionService = new AuthenticatedEncryptionService(encryptionKey.Data);
var base64EncryptedText = encryptionService.Encrypt(plainText);
return new VersionedAesGcmCiphertext(encryptionKey.Id, base64EncryptedText).ToString();
}
public string Decrypt(string cipherText)
{
var versionedCiphertext = VersionedAesGcmCiphertext.FromString(cipherText);
var encryptionKey = encryptionKeyProvider.GetEncryptionKeyById(versionedCiphertext.KeyId);
var encryptionService = new AuthenticatedEncryptionService(encryptionKey.Data);
return encryptionService.Decrypt(versionedCiphertext.Ciphertext);
}
public string UpgradeCiphertextWithCurrentEncryptionKey(string cipherText)
{
return Encrypt(Decrypt(cipherText));
}
}
Instead of accepting a single encryption key, this class delegates the key management to another class that implements IEncryptionKeyProvider.
public interface IEncryptionKeyProvider
{
public EncryptionKey GetCurrentEncryptionKey();
public EncryptionKey GetEncryptionKeyById(string keyId);
public void RotateKey(byte[] newKeyData);
}
public class EncryptionKeyProvider : IEncryptionKeyProvider
{
private readonly IList<byte[]> encryptionKeys;
public EncryptionKeyProvider(IList<byte[]> encryptionKeys)
{
this.encryptionKeys = encryptionKeys;
}
public EncryptionKey GetCurrentEncryptionKey()
{
return new EncryptionKey($"v{encryptionKeys.Count - 1}", encryptionKeys.Last());
}
public EncryptionKey GetEncryptionKeyById(string keyId)
{
var keyIndex = int.Parse(keyId[1..]);
return new EncryptionKey(keyId, encryptionKeys[keyIndex]);
}
public void RotateKey(byte[] newKeyData)
{
encryptionKeys.Add(newKeyData);
}
}
The above EncryptionKeyProvider maintains a list of encryption keys internally. We can rotate the key by appending the new key to the end of the list. To retrieve the data of a specific key, we query by the key version, which is essentially the index of that key in the list. The encryption service always utilizes the most recently added key, but it can decrypt using any previously used key. As a result, we must maintain the key version or ID information in the final ciphertext. The new data bag is implemented as follows:
public class VersionedAesGcmCiphertext
{
public string KeyId { get; }
public string Ciphertext { get; }
public VersionedAesGcmCiphertext(string keyId, string ciphertext)
{
KeyId = keyId;
Ciphertext = ciphertext;
}
public static VersionedAesGcmCiphertext FromString(string versionedCiphertext)
{
var parts = versionedCiphertext.Split('$');
return new VersionedAesGcmCiphertext(parts[0], parts[1]);
}
public override string ToString()
{
return $"{KeyId}${Ciphertext}";
}
}
We can prepend the key version to the ciphertext of our AES GCM encryption service, separated by the $ character. Not prepending the key version to the ciphertext before encoding with base64, as we do with tags and nonces, is purely for readability purposes. By examining the string representation of the final ciphertext, we can easily determine if it needs to be upgraded with the new key.
Let’s close this article with an example of how to use this implementation. First, create the encryption service with the initial encryption key and encrypt this plaintext: Một thông điệp bí mật (a Vietnamese string with UTF-8 characters).
var keyProvider = new EncryptionKeyProvider(new List<byte[]>());
var encryptionService = new KeyRotationAwareEncryptionService(keyProvider);
var plainText = "Một thông điệp bí mật";
// insert the first encryption key
keyProvider.RotateKey(Encoding.UTF8.GetBytes("00000000001111111111222222222233"));
// encrypt the plaintext with the first key
var cipherText = encryptionService.Encrypt(plainText);
// the ciphertext is similar to
// v0$i6EFWA/Lk8rDxf+xs/cDcGC7UQtPu16wjlT4DwyoothhfsFK09ZBd9zmu7zrx2Hn5fZTJh6054G/SQ==
Now, rotate the key with a new one and encrypt the same plaintext again. We will have another ciphertext with a different prefix (v1$ instead of v0$)
// rotate the current key with the new one
keyProvider.RotateKey(Encoding.UTF8.GetBytes("00000000001111111111222222222244"));
// encrypt the plaintext with the first key
var cipherText = encryptionService.Encrypt(plainText);
// the ciphertext is similar to
// v1$MK6dRecb2wdI2DDjhhyrKjocJOgtGx3OZWWhk3uUFvPsAhZ/qW1Yfe32lx+PkbSgZd02+6hpEfIJBg==
As long as we retain both keys in the key provider, we can decrypt both ciphertexts using the same encryptionService.Decrypt() method. Our implementation can determine which key to utilize by parsing the prefix and querying the key provider for the corresponding key data.
Lastly, we can also upgrade the old ciphertext to utilize the latest key without performing decryption followed by manual encryption:
var cipherText = encryptionService.UpgradeCiphertextWithCurrentEncryptionKey(plainText);
Conclusion
In this article, we embarked on a journey from a naive implementation of AES encryption to an enhanced implementation that ensures data confidentiality and integrity. We also introduced a simple key rotation mechanism. Unless you have specific reasons, I recommend against implementing all of that stuff by yourself, but instead use a client-side library like AWS Encryption SDK or a server-side encryption service like the Transit engine of HashiCorp Vault. Finally, if you have security experts in your company, do consult them after following some random articles on the internet (like this one 😄).
Happy encrypting!