Securing credentials

A harness signs in to its model with a credential: an Anthropic or OpenAI API key, or the OAuth token of a Claude or ChatGPT subscription (claude setup-token, the codex login). Whoever holds one can call the model on your account and spend its quota. Treat every one of them as a password to your billing account.

Where a credential livesCopy Anchor Link

Your own credentialsCopy Anchor Link

When the application runs on your own key or token, keep it in a secret manager and have your platform hand it to the process at runtime, as an environment variable or a mounted file: the value then never appears in the code, the image or the deployment configuration.

• One credential per application and environment, so that one leak, or one revocation, touches one of them. Give it the smallest scope and the lowest spending limit the provider allows.
• Never in an image or a template: everything in an image is readable by whoever pulls it, and from every sandbox started from it.
• Never in a log: neither the value, nor the headers or the environment that carry it.
• Revoke and replace it as soon as you suspect it leaked, from the provider's console, then look at its usage there.

Storing credentialsCopy Anchor Link

An application that lets its users bring their own credential (their API key, or the token of their subscription) has to store it, and send it on later, in clear, to the model's API. That rules out a hash, which is one-way and fits a secret you only ever check, such as a password. A credential you have to use again needs reversible encryption:

• An authenticated algorithm, such as AES-256-GCM: it encrypts, and its authentication tag reveals a ciphertext that was tampered with.
• The encryption key kept apart from the data, in a secret manager or a key management service (KMS). With a KMS, encrypt each credential with a data key of its own and store that data key wrapped by the KMS key, so that the KMS key never leaves it (envelope encryption).
• A fresh nonce for every encryption: with GCM, reusing one under the same key breaks the encryption.
• The context as associated data: authenticated without being encrypted, it ties the ciphertext to its record, its user and its provider, so that a ciphertext copied elsewhere no longer decrypts.
• The id of the key stored with the ciphertext, so that the key can be rotated.

import { createCipheriv, createDecipheriv, randomBytes } from 'node:crypto';

/** What a stored credential belongs to. It must be the same to decrypt as it was to encrypt. */
export type CredentialContext = { id: string; userId: string; provider: string };

/** The fields to store, alongside the context. The nonce and the tag are not secret. */
export type EncryptedCredential = {
  keyId: string;
  ciphertext: Buffer;
  nonce: Buffer;
  authTag: Buffer;
};

const ALGORITHM = 'aes-256-gcm';
const NONCE_LENGTH = 12;
const TAG_LENGTH = 16;

/** A 32-byte key, injected at runtime by a secret manager. Never stored with the data. */
function loadKey(variable: string): Buffer {
  const key = Buffer.from(process.env[variable] ?? '', 'base64');
  if (key.length !== 32) throw new Error(`${variable} must hold a base64-encoded 32-byte key.`);
  return key;
}

// The active key encrypts. During a rotation, the previous ones stay here to decrypt only.
const ACTIVE_KEY_ID = 'credentials-v1';
const KEYS = new Map([[ACTIVE_KEY_ID, loadKey('CREDENTIALS_KEY_V1')]]);

function keyFor(keyId: string): Buffer {
  const key = KEYS.get(keyId);
  if (!key) throw new Error('Unknown encryption key.');
  return key;
}

/** The associated data. JSON of an array is unambiguous, where a concatenation is not. */
function associatedData(context: CredentialContext, keyId: string): Buffer {
  return Buffer.from(
    JSON.stringify(['credential:v1', keyId, context.id, context.userId, context.provider]),
  );
}

export function encryptCredential(
  credential: string,
  context: CredentialContext,
): EncryptedCredential {
  const nonce = randomBytes(NONCE_LENGTH); // never reused under the same key
  const cipher = createCipheriv(ALGORITHM, keyFor(ACTIVE_KEY_ID), nonce, {
    authTagLength: TAG_LENGTH,
  });
  cipher.setAAD(associatedData(context, ACTIVE_KEY_ID));
  const ciphertext = Buffer.concat([cipher.update(credential, 'utf8'), cipher.final()]);
  return { keyId: ACTIVE_KEY_ID, ciphertext, nonce, authTag: cipher.getAuthTag() };
}

export function decryptCredential(
  encrypted: EncryptedCredential,
  context: CredentialContext,
): string {
  if (encrypted.nonce.length !== NONCE_LENGTH || encrypted.authTag.length !== TAG_LENGTH) {
    throw new Error('Invalid encrypted credential.');
  }
  const decipher = createDecipheriv(ALGORITHM, keyFor(encrypted.keyId), encrypted.nonce, {
    authTagLength: TAG_LENGTH,
  });
  decipher.setAAD(associatedData(context, encrypted.keyId));
  decipher.setAuthTag(encrypted.authTag);
  // final() throws when the ciphertext, the tag, the key or the context differs.
  return Buffer.concat([decipher.update(encrypted.ciphertext), decipher.final()]).toString('utf8');
}

/** Encrypts again, under the active key, a credential an older key encrypted. */
export function reencryptCredential(
  encrypted: EncryptedCredential,
  context: CredentialContext,
): EncryptedCredential {
  if (encrypted.keyId === ACTIVE_KEY_ID) return encrypted;
  return encryptCredential(decryptCredential(encrypted, context), context);
}

• Store the four fields alongside the context, wherever your credentials live: as binary values, or as base64 strings where the store has no binary type. The errors say nothing of the credential: log them as they are.
• The context does not change. To move a credential to another user or provider, decrypt it with the old context and encrypt it again with the new one. As it holds the record's id, that id must be known before the record is first saved: generate it in the application, as a UUID.
• A nonce is never reused under the same key. A random 96-bit nonce stays safe for about 2³² encryptions under one key (NIST SP 800-38D): rotate the key well before. A uniqueness check on (keyId, nonce), where your store offers one, adds a guard: should it ever reject a write, encrypt again, with a new nonce.
• To rotate the key, add the new one to KEYS, make it the active key, run reencryptCredential() over every stored credential, then remove the old one. With envelope encryption, the KMS rotates its own key, and loadKey() gives way to unwrapping the data key with it.

What encryption protects, and what it does notCopy Anchor Link

Encryption protects the credentials against the theft of the stored data alone: a copy, a backup or a replica of it, without the key, is of no use. It does not protect them against a compromise of the application, which holds the right to decrypt them: whoever controls it can decrypt them too. Narrow what such a compromise exposes:

• Decrypt a credential only when a turn needs it, keep it in memory only, and never write it back anywhere in clear.
• Give the right to decrypt to the one service that calls the model, and nothing else: with a KMS, to its identity alone.
• Audit the decryptions: a KMS logs every one of them, and an unusual volume shows a compromise.
• Let your users revoke a credential they gave you, and delete it for good when they do.

Further readingCopy Anchor Link

• The OWASP Cryptographic Storage Cheat Sheet, on choosing algorithms, managing keys and storing secrets.
• How the credentials travel and stay out of the sandbox, in ai-sdk-sandbox-sbx and ai-sdk-sandbox-cloud-run.