Key Vault Best Practices
Secure secrets, keys, and certificates with Azure Key Vault access policies and rotation.
Prerequisites
- Azure subscription with Key Vault permissions
- Understanding of Azure RBAC
- Basic understanding of cryptographic keys and certificates
Azure Key Vault Best Practices
Azure Key Vault is a cloud service for securely storing and accessing secrets, encryption keys, and certificates. It eliminates the need to store sensitive data in application code, configuration files, or environment variables. This is one of the most common sources of credential leaks and security breaches. Properly implementing Key Vault is arguably the single most impactful security improvement you can make in your Azure environment.
This guide covers every aspect of Key Vault implementation: architecture decisions, access control models, secret management patterns, encryption key operations, certificate lifecycle management, rotation strategies, network security, monitoring, disaster recovery, and operational best practices. Whether you are securing your first application or designing a Key Vault strategy for an enterprise with hundreds of services, this guide provides the patterns and code examples you need.
Key Vault Architecture Decisions
Before creating your first Key Vault, several architectural decisions will shape your security posture, operational model, and blast radius for years. Getting these right from the start avoids painful migrations later.
Vault Topology: How Many Vaults?
Microsoft recommends using separate Key Vaults per application, per environment. This isolation strategy limits the blast radius of a compromised identity to a single application's secrets in a single environment.
- Per application: If one vault is compromised, only that application's secrets are exposed. Other applications remain unaffected.
- Per environment: Prevents accidental cross-environment access. Development code cannot read production secrets, even if a developer has broad permissions in non-production.
- Shared vault for common secrets: Certificates and keys shared across multiple services (such as a wildcard TLS certificate or a shared API gateway key) can live in a dedicated shared vault with strict access controls and separate audit logging.
| Pattern | Example | Pros | Cons |
|---|---|---|---|
| One vault per app per env | myapp-prod-kv, myapp-dev-kv | Best isolation, simplest RBAC, clearest audit trail | Many vaults to manage, higher IaC complexity |
| Shared per environment | prod-shared-kv, dev-shared-kv | Fewer vaults, centralized management | Broader blast radius, complex fine-grained RBAC |
| Shared per team per env | team-a-prod-kv, team-b-prod-kv | Balanced isolation and manageability | Team boundaries can shift, creating orphaned vaults |
| Single vault (anti-pattern) | company-kv | Simplest management | Worst isolation, hardest to secure, throttling risk |
Vault Naming Convention
Key Vault names must be globally unique across all of Azure (3–24 characters, alphanumeric and hyphens only). Use a consistent naming pattern like {app}-{env}-kv or {org}-{app}-{env}-kv. Shorter prefixes are better since you only have 24 characters. Some organizations include a random suffix to prevent name enumeration: {app}-{env}-{random4}-kv.
Standard vs Premium Tier
Key Vault comes in two tiers. The primary difference is HSM (Hardware Security Module) support for cryptographic keys.
| Feature | Standard | Premium |
|---|---|---|
| Software-protected keys | Yes | Yes |
| HSM-protected keys | No | Yes (FIPS 140-2 Level 2) |
| Secrets and certificates | Yes | Yes |
| Price per 10K operations | $0.03 (secrets, keys) | $0.03 (secrets) / $1.00 (HSM keys) |
| Managed HSM | No | Available separately (FIPS 140-2 Level 3) |
| Recommendation | Most workloads | Compliance requiring HSM (PCI-DSS, HIPAA, CMMC) |
Managed HSM for Highest Assurance
For workloads requiring FIPS 140-2 Level 3 compliance, Azure offers Managed HSM as a separate product. It provides single-tenant, dedicated HSM pools with customer-controlled security domains. Managed HSM costs significantly more (~$3.20/hour per HSM unit) but meets the strictest regulatory requirements. Standard Key Vault Premium tier (FIPS 140-2 Level 2) is sufficient for most compliance frameworks.
Creating a Production Key Vault
# Create a Key Vault with all recommended security settings
az keyvault create \
--name myapp-prod-kv \
--resource-group myRG \
--location eastus2 \
--sku standard \
--enable-rbac-authorization true \
--enable-purge-protection true \
--retention-days 90 \
--default-action Deny \
--bypass AzureServices
# Verify the configuration
az keyvault show --name myapp-prod-kv --query '{
name: name,
rbacAuthorization: properties.enableRbacAuthorization,
purgeProtection: properties.enablePurgeProtection,
softDelete: properties.enableSoftDelete,
softDeleteRetention: properties.softDeleteRetentionInDays,
networkDefaultAction: properties.networkAcls.defaultAction
}'// modules/keyvault.bicep
@description('Application name used in vault naming')
param appName string
@description('Environment name (dev, staging, prod)')
@allowed(['dev', 'staging', 'prod'])
param environment string
@description('Azure region')
param location string = resourceGroup().location
@description('Subnet ID for Private Endpoint (required for prod)')
param privateEndpointSubnetId string = ''
@description('Private DNS Zone ID for vault.azure.net')
param privateDnsZoneId string = ''
var vaultName = '${appName}-${environment}-kv'
var isProd = environment == 'prod'
resource vault 'Microsoft.KeyVault/vaults@2023-07-01' = {
name: vaultName
location: location
properties: {
sku: {
family: 'A'
name: 'standard'
}
tenantId: subscription().tenantId
enableRbacAuthorization: true
enablePurgeProtection: true
enableSoftDelete: true
softDeleteRetentionInDays: isProd ? 90 : 7
networkAcls: {
defaultAction: isProd ? 'Deny' : 'Allow'
bypass: 'AzureServices'
ipRules: []
virtualNetworkRules: []
}
}
tags: {
application: appName
environment: environment
managedBy: 'bicep'
}
}
// Private Endpoint for production
resource privateEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (isProd && !empty(privateEndpointSubnetId)) {
name: '${vaultName}-pe'
location: location
properties: {
subnet: {
id: privateEndpointSubnetId
}
privateLinkServiceConnections: [
{
name: '${vaultName}-pe-connection'
properties: {
privateLinkServiceId: vault.id
groupIds: ['vault']
}
}
]
}
}
resource privateDnsZoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-11-01' = if (isProd && !empty(privateDnsZoneId)) {
parent: privateEndpoint
name: 'default'
properties: {
privateDnsZoneConfigs: [
{
name: 'vault'
properties: {
privateDnsZoneId: privateDnsZoneId
}
}
]
}
}
// Diagnostic settings for audit logging
resource diagnostics 'Microsoft.Insights/diagnosticSettings@2021-05-01-preview' = {
name: '${vaultName}-diagnostics'
scope: vault
properties: {
workspaceId: '' // Replace with Log Analytics workspace ID
logs: [
{
categoryGroup: 'audit'
enabled: true
retentionPolicy: {
enabled: isProd
days: isProd ? 365 : 30
}
}
{
categoryGroup: 'allLogs'
enabled: true
retentionPolicy: {
enabled: isProd
days: isProd ? 90 : 7
}
}
]
metrics: [
{
category: 'AllMetrics'
enabled: true
}
]
}
}
output vaultName string = vault.name
output vaultUri string = vault.properties.vaultUri
output vaultId string = vault.idAccess Control: RBAC vs Access Policies
Key Vault supports two access control models: Azure RBAC (recommended) and vault access policies (legacy). All new vaults should use Azure RBAC exclusively. Azure RBAC provides granular permissions through built-in roles, integrates with Conditional Access policies, supports Azure PIM for just-in-time access, and provides a consistent authorization model across all Azure resources.
Key Vault RBAC Roles
Azure provides six built-in roles that follow the principle of least privilege, separating read and write permissions across the three object types (secrets, keys, certificates).
| Role | Secrets | Keys | Certificates | Use Case |
|---|---|---|---|---|
| Key Vault Administrator | Full | Full | Full | Platform team, vault lifecycle management |
| Key Vault Secrets Officer | Full (CRUD) | None | None | CI/CD pipelines that write secrets |
| Key Vault Secrets User | Read only | None | None | Applications reading secrets at runtime |
| Key Vault Crypto Officer | None | Full (CRUD + operations) | None | Key management automation, rotation |
| Key Vault Crypto User | None | Use only (encrypt/decrypt/sign/verify) | None | Applications performing cryptographic operations |
| Key Vault Certificates Officer | None | None | Full (CRUD + issue) | Certificate lifecycle management, renewal automation |
| Key Vault Reader | List (no values) | List (no values) | List (no values) | Audit, inventory, monitoring (cannot read secret values) |
# Grant the app's managed identity read-only access to secrets
az role assignment create \
--assignee <managed-identity-principal-id> \
--role "Key Vault Secrets User" \
--scope /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.KeyVault/vaults/myapp-prod-kv
# Grant the CI/CD pipeline write access to secrets only
az role assignment create \
--assignee <cicd-service-principal-id> \
--role "Key Vault Secrets Officer" \
--scope /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.KeyVault/vaults/myapp-prod-kv
# Grant developers read-only access in non-production
az role assignment create \
--assignee-object-id <dev-group-object-id> \
--assignee-principal-type Group \
--role "Key Vault Secrets User" \
--scope /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.KeyVault/vaults/myapp-dev-kv
# Grant the platform team full admin access
az role assignment create \
--assignee-object-id <platform-group-object-id> \
--assignee-principal-type Group \
--role "Key Vault Administrator" \
--scope /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.KeyVault/vaults/myapp-prod-kv
# Grant certificate automation access to certificates only
az role assignment create \
--assignee <cert-automation-principal-id> \
--role "Key Vault Certificates Officer" \
--scope /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.KeyVault/vaults/myapp-prod-kv
# Verify all role assignments on the vault
az role assignment list \
--scope /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.KeyVault/vaults/myapp-prod-kv \
--output tableNever Grant Key Vault Administrator to Applications
Applications should only receive the minimum required role. A web app that reads database connection strings needs Key Vault Secrets User, not Administrator. Over-privileged service identities are one of the most common Key Vault security misconfigurations and a frequent finding in security audits. If a compromised application has Administrator access, an attacker can exfiltrate all secrets, keys, and certificates, and even delete the vault if purge protection is not enabled.
Migrating from Access Policies to RBAC
If you have existing vaults using access policies, you can migrate to RBAC without downtime. The process involves creating equivalent RBAC role assignments for all existing access policy entries, then switching the authorization model.
# Step 1: List current access policies to understand existing permissions
az keyvault show --name myapp-prod-kv \
--query 'properties.accessPolicies[].{
objectId: objectId,
secretPerms: permissions.secrets,
keyPerms: permissions.keys,
certPerms: permissions.certificates
}' --output table
# Step 2: Create equivalent RBAC role assignments
# (Map each access policy to the appropriate RBAC role)
# Example: objectId with secrets [get, list] -> Key Vault Secrets User
az role assignment create \
--assignee-object-id <object-id-from-access-policy> \
--role "Key Vault Secrets User" \
--scope $(az keyvault show --name myapp-prod-kv --query id -o tsv)
# Step 3: Switch authorization model to RBAC
# WARNING: Access policies are removed immediately. Ensure all RBAC assignments
# are in place before running this command.
az keyvault update \
--name myapp-prod-kv \
--enable-rbac-authorization true
# Step 4: Verify access still works
az keyvault secret list --vault-name myapp-prod-kv --output tableSecrets Management Patterns
Secrets are the most commonly used Key Vault object type. They store arbitrary string values up to 25 KB: connection strings, API keys, passwords, tokens, and configuration values. The goal is to ensure secrets never appear in code, configuration files, environment variable definitions, or deployment scripts.
App Service Key Vault References (Zero-Code Integration)
The simplest integration pattern uses Key Vault references in App Service application settings. Your application reads secrets as ordinary environment variables with no SDK dependency and no code changes. The App Service platform resolves Key Vault references at startup and on configuration refresh.
# Enable system-assigned managed identity on App Service
az webapp identity assign --name my-web-app --resource-group myRG
# Grant the identity read access to the vault
az role assignment create \
--assignee $(az webapp identity show --name my-web-app -g myRG --query principalId -o tsv) \
--role "Key Vault Secrets User" \
--scope $(az keyvault show --name myapp-prod-kv --query id -o tsv)
# Set app settings with Key Vault references
# The app reads these as normal environment variables, no SDK needed
az webapp config appsettings set --name my-web-app --resource-group myRG \
--settings \
"DatabaseConnection=@Microsoft.KeyVault(VaultName=myapp-prod-kv;SecretName=db-connection-string)" \
"ApiKey=@Microsoft.KeyVault(VaultName=myapp-prod-kv;SecretName=external-api-key)" \
"RedisPassword=@Microsoft.KeyVault(VaultName=myapp-prod-kv;SecretName=redis-password)"
# To reference a specific version (pin to a version)
# "DbPassword=@Microsoft.KeyVault(VaultName=myapp-prod-kv;SecretName=db-password;SecretVersion=abc123def456)"
# To reference using the secret URI directly
# "DbPassword=@Microsoft.KeyVault(SecretUri=https://myapp-prod-kv.vault.azure.net/secrets/db-password)"Key Vault References Resolve Automatically
When a secret is rotated in Key Vault, App Service Key Vault references (without a pinned version) automatically pick up the new value within 24 hours or when the app restarts. To force an immediate refresh, restart the app or touch any application setting. For near-real-time rotation, use the secret URI format without a version and configure your app to call the Key Vault SDK directly with a short cache TTL.
Reading Secrets in Application Code
When you need more control over secret retrieval, such as caching, versioning, or conditional access, use the Azure SDK. Always authenticate withDefaultAzureCredential, which automatically uses managed identity in Azure, Azure CLI credentials during local development, and environment variables in CI/CD pipelines.
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
from functools import lru_cache
import time
import logging
logger = logging.getLogger(__name__)
class SecretManager:
"""Manages Key Vault secrets with caching and error handling."""
def __init__(self, vault_url: str, cache_ttl: int = 300):
self.credential = DefaultAzureCredential()
self.client = SecretClient(
vault_url=vault_url,
credential=self.credential
)
self.cache_ttl = cache_ttl
self._cache = {}
self._cache_timestamps = {}
def get_secret(self, name: str) -> str:
"""Get a secret with TTL-based caching."""
now = time.time()
# Check cache
if name in self._cache:
age = now - self._cache_timestamps[name]
if age < self.cache_ttl:
return self._cache[name]
logger.info(f"Cache expired for secret '{name}' (age={age:.0f}s)")
# Fetch from Key Vault
try:
secret = self.client.get_secret(name)
self._cache[name] = secret.value
self._cache_timestamps[name] = now
logger.info(f"Fetched secret '{name}' from Key Vault")
return secret.value
except Exception as e:
# Return cached value if available (stale but better than failure)
if name in self._cache:
logger.warning(
f"Failed to refresh secret '{name}', using cached value: {e}"
)
return self._cache[name]
raise
def invalidate(self, name: str = None):
"""Invalidate cached secret(s)."""
if name:
self._cache.pop(name, None)
self._cache_timestamps.pop(name, None)
else:
self._cache.clear()
self._cache_timestamps.clear()
# Usage
secrets = SecretManager(
vault_url="https://myapp-prod-kv.vault.azure.net",
cache_ttl=300 # 5-minute cache
)
db_connection = secrets.get_secret("db-connection-string")
api_key = secrets.get_secret("external-api-key")// Program.cs: Configure Key Vault as a configuration source
using Azure.Identity;
using Azure.Extensions.AspNetCore.Configuration.Secrets;
var builder = WebApplication.CreateBuilder(args);
// Add Key Vault as a configuration source
// Secrets are accessible via IConfiguration["SecretName"]
var vaultUri = new Uri(builder.Configuration["KeyVault:VaultUri"]!);
builder.Configuration.AddAzureKeyVault(
vaultUri,
new DefaultAzureCredential(),
new AzureKeyVaultConfigurationOptions
{
// Reload secrets every 5 minutes
ReloadInterval = TimeSpan.FromMinutes(5),
// Optional: filter which secrets to load
Manager = new PrefixKeyVaultSecretManager("MyApp")
}
);
// Register services
builder.Services.AddSingleton<IConfiguration>(builder.Configuration);
var app = builder.Build();
// Usage in a controller or service
app.MapGet("/api/test", (IConfiguration config) =>
{
// Key Vault secrets are available as configuration values
var dbConnection = config["db-connection-string"];
return Results.Ok("Connected");
});
app.Run();
// Custom secret manager that filters by prefix
public class PrefixKeyVaultSecretManager : KeyVaultSecretManager
{
private readonly string _prefix;
public PrefixKeyVaultSecretManager(string prefix)
{
_prefix = prefix + "--";
}
public override bool Load(SecretProperties properties)
{
// Only load secrets that start with the app prefix
return properties.Name.StartsWith(_prefix);
}
public override string GetKey(KeyVaultSecret secret)
{
// Remove prefix and convert -- to : for configuration hierarchy
return secret.Name[_prefix.Length..]
.Replace("--", ConfigurationPath.KeyDelimiter);
}
}Secrets in Kubernetes (AKS)
For AKS workloads, the CSI Secrets Store driver with the Azure Key Vault provider mounts secrets as files in the pod or syncs them to Kubernetes Secrets. Combined with Workload Identity, this provides a fully managed, identity-based secret access path with no credentials stored in the cluster.
# SecretProviderClass: defines which secrets to mount from Key Vault
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: myapp-secrets
namespace: production
spec:
provider: azure
parameters:
usePodIdentity: "false"
useVMManagedIdentity: "false"
clientID: "<managed-identity-client-id>" # Workload Identity client ID
keyvaultName: "myapp-prod-kv"
tenantId: "<tenant-id>"
objects: |
array:
- |
objectName: db-connection-string
objectType: secret
- |
objectName: redis-password
objectType: secret
- |
objectName: tls-certificate
objectType: secret
objectAlias: tls.crt
- |
objectName: encryption-key
objectType: key
objectVersion: ""
# Sync to Kubernetes Secret (optional, for env vars)
secretObjects:
- secretName: myapp-kv-secrets
type: Opaque
data:
- objectName: db-connection-string
key: DB_CONNECTION
- objectName: redis-password
key: REDIS_PASSWORD
---
# Deployment using the SecretProviderClass
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-api
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: my-api
template:
metadata:
labels:
app: my-api
azure.workload.identity/use: "true"
spec:
serviceAccountName: my-api-sa
containers:
- name: my-api
image: myacr.azurecr.io/my-api:v1.2.3
# Option 1: Read secrets from mounted volume (file-based)
volumeMounts:
- name: secrets-store
mountPath: "/mnt/secrets"
readOnly: true
# Option 2: Read secrets as environment variables (synced K8s Secret)
env:
- name: DB_CONNECTION
valueFrom:
secretKeyRef:
name: myapp-kv-secrets
key: DB_CONNECTION
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: myapp-kv-secrets
key: REDIS_PASSWORD
volumes:
- name: secrets-store
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: myapp-secretsEncryption Keys
Key Vault supports RSA and Elliptic Curve (EC) keys for encryption, decryption, signing, and verification. Keys can be software-protected (Standard tier) or HSM-protected (Premium tier). The most common use cases are customer-managed keys (CMK) for Azure service encryption and application-level envelope encryption.
Customer-Managed Keys (CMK)
Many Azure services encrypt data at rest by default with Microsoft-managed keys. Customer-managed keys (CMK) let you control the encryption key, providing an additional layer of control and the ability to revoke access by disabling the key.
| Azure Service | CMK Support | Key Type | Notes |
|---|---|---|---|
| Azure Storage | Yes | RSA 2048, 3072, 4096 | Per-account or per-scope. Auto-rotation supported. |
| Azure SQL Database | Yes (TDE) | RSA 2048, 3072 | Transparent Data Encryption with CMK. |
| Cosmos DB | Yes | RSA 2048, 3072, 4096 | Must be set at account creation. |
| Azure Disk Encryption | Yes | RSA 2048, 3072, 4096 | Server-Side Encryption (SSE) with CMK. |
| Azure Kubernetes Service | Yes | RSA 2048 | Encrypts etcd secrets at rest. |
| Azure Cognitive Services | Yes | RSA 2048, 3072, 4096 | Encrypts stored data and models. |
# Create an RSA key in Key Vault
az keyvault key create \
--vault-name myapp-prod-kv \
--name storage-encryption-key \
--kty RSA \
--size 3072 \
--ops encrypt decrypt wrapKey unwrapKey \
--protection software
# Enable auto-rotation for the key
az keyvault key rotation-policy update \
--vault-name myapp-prod-kv \
--name storage-encryption-key \
--value '{
"lifetimeActions": [
{
"trigger": { "timeBeforeExpiry": "P30D" },
"action": { "type": "Rotate" }
},
{
"trigger": { "timeBeforeExpiry": "P15D" },
"action": { "type": "Notify" }
}
],
"attributes": {
"expiryTime": "P1Y"
}
}'
# Grant the storage account's managed identity key permissions
az role assignment create \
--assignee $(az storage account show --name mystorageacct -g myRG --query identity.principalId -o tsv) \
--role "Key Vault Crypto Service Encryption User" \
--scope $(az keyvault show --name myapp-prod-kv --query id -o tsv)
# Configure the storage account to use the CMK
az storage account update \
--name mystorageacct \
--resource-group myRG \
--encryption-key-vault $(az keyvault show --name myapp-prod-kv --query properties.vaultUri -o tsv) \
--encryption-key-name storage-encryption-key \
--encryption-key-source Microsoft.Keyvault \
--key-vault-user-identity-id "" # Uses system-assigned identityApplication-Level Envelope Encryption
For application-level encryption (encrypting specific fields, files, or messages), use envelope encryption: generate a data encryption key (DEK) locally, encrypt the data with the DEK, then wrap (encrypt) the DEK with a Key Vault key (the key encryption key, or KEK). Store the wrapped DEK alongside the encrypted data. To decrypt, unwrap the DEK using Key Vault, then decrypt the data locally. This pattern keeps bulk data encryption fast (local symmetric encryption) while protecting the DEK with Key Vault's access controls and audit logging.
from azure.identity import DefaultAzureCredential
from azure.keyvault.keys import KeyClient
from azure.keyvault.keys.crypto import CryptographyClient, EncryptionAlgorithm
from cryptography.fernet import Fernet
import base64
import json
credential = DefaultAzureCredential()
key_client = KeyClient(
vault_url="https://myapp-prod-kv.vault.azure.net",
credential=credential
)
# Get the Key Vault key (KEK)
key = key_client.get_key("data-encryption-kek")
crypto_client = CryptographyClient(key, credential=credential)
def encrypt_data(plaintext: bytes) -> dict:
"""Encrypt data using envelope encryption."""
# Step 1: Generate a local data encryption key (DEK)
dek = Fernet.generate_key()
fernet = Fernet(dek)
# Step 2: Encrypt data locally with the DEK (fast, symmetric)
ciphertext = fernet.encrypt(plaintext)
# Step 3: Wrap (encrypt) the DEK with the Key Vault key (KEK)
wrapped_dek = crypto_client.wrap_key(
algorithm="RSA-OAEP-256",
key=dek
)
# Step 4: Return the encrypted data + wrapped DEK
return {
"ciphertext": base64.b64encode(ciphertext).decode(),
"wrapped_dek": base64.b64encode(wrapped_dek.encrypted_key).decode(),
"kek_id": key.id,
"algorithm": "RSA-OAEP-256"
}
def decrypt_data(envelope: dict) -> bytes:
"""Decrypt envelope-encrypted data."""
# Step 1: Unwrap the DEK using Key Vault
wrapped_dek = base64.b64decode(envelope["wrapped_dek"])
unwrapped = crypto_client.unwrap_key(
algorithm=envelope["algorithm"],
encrypted_key=wrapped_dek
)
# Step 2: Decrypt data locally with the DEK
fernet = Fernet(unwrapped.key)
ciphertext = base64.b64decode(envelope["ciphertext"])
return fernet.decrypt(ciphertext)
# Usage
encrypted = encrypt_data(b"sensitive patient data")
decrypted = decrypt_data(encrypted) # b"sensitive patient data"Key Vault Key Operations Are Rate-Limited
Key Vault allows 2,000 RSA 2048-bit operations or 1,000 RSA 3072/4096-bit operations per 10 seconds per vault. For high-throughput encryption, always use envelope encryption (wrap/unwrap the DEK, not encrypt individual records). This limits Key Vault calls to key lifecycle events rather than per-record operations.
Certificate Management
Key Vault provides full certificate lifecycle management: creation, import, renewal, and deployment. It can issue certificates from integrated Certificate Authorities (DigiCert, GlobalSign) or self-signed certificates for internal use. Certificates stored in Key Vault can be automatically deployed to App Service, Application Gateway, and other Azure services.
Certificate Operations
# Create a self-signed certificate (for development/internal use)
az keyvault certificate create \
--vault-name myapp-prod-kv \
--name internal-api-cert \
--policy '{
"issuerParameters": { "name": "Self" },
"keyProperties": {
"exportable": true,
"keySize": 2048,
"keyType": "RSA",
"reuseKey": false
},
"secretProperties": {
"contentType": "application/x-pkcs12"
},
"x509CertificateProperties": {
"subject": "CN=api.internal.contoso.com",
"subjectAlternativeNames": {
"dnsNames": [
"api.internal.contoso.com",
"*.api.internal.contoso.com"
]
},
"validityInMonths": 12,
"keyUsage": ["digitalSignature", "keyEncipherment"]
},
"lifetimeActions": [
{
"trigger": { "daysBeforeExpiry": 30 },
"action": { "actionType": "AutoRenew" }
}
]
}'
# Import an existing PFX certificate
az keyvault certificate import \
--vault-name myapp-prod-kv \
--name wildcard-cert \
--file ./wildcard-contoso.pfx \
--password "PfxPassword123"
# Create a certificate with DigiCert CA integration
az keyvault certificate create \
--vault-name myapp-prod-kv \
--name public-api-cert \
--policy '{
"issuerParameters": {
"name": "DigiCertIssuer",
"certificateType": "OV-SSL"
},
"keyProperties": {
"exportable": true,
"keySize": 2048,
"keyType": "RSA"
},
"x509CertificateProperties": {
"subject": "CN=api.contoso.com",
"validityInMonths": 12
},
"lifetimeActions": [
{
"trigger": { "daysBeforeExpiry": 60 },
"action": { "actionType": "AutoRenew" }
},
{
"trigger": { "daysBeforeExpiry": 30 },
"action": { "actionType": "EmailContacts" }
}
]
}'
# List all certificates with expiry dates
az keyvault certificate list \
--vault-name myapp-prod-kv \
--query '[].{name:name, expires:attributes.expires, enabled:attributes.enabled}' \
--output table
# Deploy certificate to App Service
az webapp config ssl import \
--resource-group myRG \
--name my-web-app \
--key-vault myapp-prod-kv \
--key-vault-certificate-name wildcard-certSecret Rotation Strategies
Secrets should be rotated regularly to limit the window of exposure if credentials are compromised. Azure Key Vault supports multiple rotation patterns, from fully automated to event-driven custom rotation.
Built-In Auto-Rotation
Key Vault natively supports auto-rotation for storage account keys. For other secret types, you configure rotation policies that trigger events when secrets approach expiry, which you handle with custom automation.
Event-Driven Rotation Pattern
The recommended pattern for custom secrets uses Key Vault events with Event Grid to trigger an Azure Function. When a secret approaches its expiry date, Key Vault emits a SecretNearExpiry event. The Function rotates the credential in the downstream system and stores the new value in Key Vault.
# Set a secret with an expiration date
az keyvault secret set \
--vault-name myapp-prod-kv \
--name db-password \
--value "$(openssl rand -base64 32)" \
--expires "2026-06-01T00:00:00Z" \
--tags purpose=database rotation=automatic
# Configure a rotation policy
az keyvault secret rotation-policy update \
--vault-name myapp-prod-kv \
--name db-password \
--value '{
"lifetimeActions": [
{
"trigger": { "timeBeforeExpiry": "P30D" },
"action": { "type": "Notify" }
}
],
"attributes": {
"expiryTime": "P90D"
}
}'
# Subscribe to near-expiry events
az eventgrid event-subscription create \
--name secret-rotation-handler \
--source-resource-id /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.KeyVault/vaults/myapp-prod-kv \
--included-event-types \
Microsoft.KeyVault.SecretNearExpiry \
Microsoft.KeyVault.SecretExpired \
--endpoint /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.Web/sites/rotation-function/functions/RotateSecret \
--endpoint-type azurefunction
# Subscribe to certificate events
az eventgrid event-subscription create \
--name cert-expiry-alert \
--source-resource-id /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.KeyVault/vaults/myapp-prod-kv \
--included-event-types \
Microsoft.KeyVault.CertificateNearExpiry \
Microsoft.KeyVault.CertificateExpired \
--endpoint /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.Logic/workflows/cert-renewal-workflow \
--endpoint-type webhookimport azure.functions as func
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
import pyodbc
import secrets
import string
import logging
import json
logger = logging.getLogger(__name__)
def generate_password(length: int = 32) -> str:
"""Generate a cryptographically secure password."""
alphabet = string.ascii_letters + string.digits + "!@#$%&*"
# Ensure at least one of each required character type
password = [
secrets.choice(string.ascii_uppercase),
secrets.choice(string.ascii_lowercase),
secrets.choice(string.digits),
secrets.choice("!@#$%&*"),
]
password += [secrets.choice(alphabet) for _ in range(length - 4)]
secrets.SystemRandom().shuffle(password)
return ''.join(password)
def main(event: func.EventGridEvent) -> None:
"""Rotate a database password when Key Vault fires SecretNearExpiry."""
event_data = event.get_json()
secret_name = event_data.get("ObjectName", "")
vault_name = event_data.get("VaultName", "")
logger.info(f"Rotation triggered for secret '{secret_name}' "
f"in vault '{vault_name}'")
if secret_name != "db-password":
logger.info(f"Skipping non-database secret: {secret_name}")
return
credential = DefaultAzureCredential()
client = SecretClient(
vault_url=f"https://{vault_name}.vault.azure.net",
credential=credential
)
# Step 1: Generate a new password
new_password = generate_password()
# Step 2: Get current connection info from Key Vault
server_secret = client.get_secret("db-server-name")
current_password = client.get_secret("db-password")
# Step 3: Connect to SQL and change the password
conn_str = (
f"DRIVER={{ODBC Driver 18 for SQL Server}};"
f"SERVER={server_secret.value};"
f"DATABASE=master;"
f"UID=app_user;"
f"PWD={current_password.value}"
)
with pyodbc.connect(conn_str) as conn:
conn.autocommit = True
conn.execute(
f"ALTER LOGIN app_user WITH PASSWORD = '{new_password}'"
)
logger.info("SQL password changed successfully")
# Step 4: Store the new password in Key Vault
client.set_secret(
"db-password",
new_password,
tags={"rotatedBy": "azure-function", "rotationType": "automatic"}
)
logger.info("New password stored in Key Vault")
# Step 5: Verify the new password works
verify_conn_str = conn_str.replace(current_password.value, new_password)
with pyodbc.connect(verify_conn_str) as conn:
conn.execute("SELECT 1")
logger.info("Password rotation verified successfully")Dual-Secret Rotation for Zero Downtime
When rotating secrets for services that cache credentials (like connection pools), use a dual-secret pattern: create the new secret as a new version, wait for all consumers to pick up the new version (based on your cache TTL), then disable the old version. For database passwords, consider maintaining two user accounts (app_user_1 and app_user_2) and alternating which one is active. This ensures the old password remains valid during the transition window.
Network Security
Network-level controls are your first line of defense for Key Vault. A properly secured vault is accessible only from authorized networks and has all access attempts logged.
Private Endpoints
For production workloads, disable public access to Key Vault entirely and use Private Endpoints. This ensures the vault is only reachable from within your VNet, eliminating internet-based attack vectors.
# Disable public access to the vault
az keyvault update \
--name myapp-prod-kv \
--public-network-access Disabled
# Create a Private Endpoint for the vault
az network private-endpoint create \
--name myapp-prod-kv-pe \
--resource-group myRG \
--vnet-name myVNet \
--subnet private-endpoints-subnet \
--private-connection-resource-id $(az keyvault show --name myapp-prod-kv --query id -o tsv) \
--group-id vault \
--connection-name myapp-prod-kv-pe-connection
# Create Private DNS Zone for vault.azure.net
az network private-dns zone create \
--resource-group myRG \
--name privatelink.vaultcore.azure.net
# Link the DNS zone to the VNet
az network private-dns link vnet create \
--resource-group myRG \
--zone-name privatelink.vaultcore.azure.net \
--name myVNet-vault-link \
--virtual-network myVNet \
--registration-enabled false
# Create DNS zone group for automatic DNS record management
az network private-endpoint dns-zone-group create \
--resource-group myRG \
--endpoint-name myapp-prod-kv-pe \
--name default \
--private-dns-zone privatelink.vaultcore.azure.net \
--zone-name vaultcore
# Verify DNS resolution (should return private IP)
nslookup myapp-prod-kv.vault.azure.net
# Allow trusted Azure services to bypass network rules
# (needed for Key Vault references in App Service, CMK for storage, etc.)
az keyvault update \
--name myapp-prod-kv \
--bypass AzureServicesTrusted Services Bypass
When you set the default network action to Deny, some Azure services lose access to the vault. The --bypass AzureServices flag allows trusted first-party services (Azure Storage for CMK, Azure SQL for TDE, App Service for Key Vault references, Azure Backup, etc.) to access the vault even with public access disabled. Always enable this bypass unless your compliance requirements prohibit it.
Monitoring, Auditing, and Alerting
Every Key Vault operation should be logged and monitored. Diagnostic logs capture who accessed which secret, key, or certificate, when, from what IP address, and whether the operation succeeded or failed. This audit trail is essential for security incident response and compliance.
Enabling Diagnostic Logging
# Enable diagnostic logging to Log Analytics
az monitor diagnostic-settings create \
--name kv-diagnostics \
--resource $(az keyvault show --name myapp-prod-kv --query id -o tsv) \
--workspace /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.OperationalInsights/workspaces/myLogAnalytics \
--logs '[
{"categoryGroup": "audit", "enabled": true, "retentionPolicy": {"enabled": true, "days": 365}},
{"categoryGroup": "allLogs", "enabled": true, "retentionPolicy": {"enabled": true, "days": 90}}
]' \
--metrics '[{"category": "AllMetrics", "enabled": true}]'
# Alert on unauthorized access attempts (HTTP 403)
az monitor metrics alert create \
--name kv-unauthorized-access \
--resource-group myRG \
--scopes $(az keyvault show --name myapp-prod-kv --query id -o tsv) \
--condition "count ServiceApiResult where StatusCode includes 403 > 5" \
--window-size 5m \
--evaluation-frequency 1m \
--severity 2 \
--action /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.Insights/actionGroups/security-team
# Alert on high request volume (potential credential stuffing)
az monitor metrics alert create \
--name kv-high-request-volume \
--resource-group myRG \
--scopes $(az keyvault show --name myapp-prod-kv --query id -o tsv) \
--condition "count ServiceApiResult > 1000" \
--window-size 10m \
--evaluation-frequency 5m \
--severity 3 \
--action /subscriptions/<sub-id>/resourceGroups/myRG/providers/Microsoft.Insights/actionGroups/security-team// Find all failed access attempts in the last 24 hours
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.KEYVAULT"
| where ResultSignature == "Forbidden" or httpStatusCode_d == 403
| project
TimeGenerated,
OperationName,
CallerIPAddress,
identity_claim_oid_g,
requestUri_s,
ResultSignature,
ResultDescription
| order by TimeGenerated desc
// Track secret access patterns (who reads which secrets)
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.KEYVAULT"
| where OperationName == "SecretGet"
| summarize
AccessCount = count(),
LastAccess = max(TimeGenerated),
UniqueCallers = dcount(identity_claim_oid_g)
by SecretName = tostring(split(requestUri_s, "/")[4])
| order by AccessCount desc
// Detect unusual access patterns (access outside business hours)
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.KEYVAULT"
| where OperationName in ("SecretGet", "KeySign", "KeyDecrypt")
| extend HourOfDay = datetime_part("hour", TimeGenerated)
| where HourOfDay < 6 or HourOfDay > 22 // Outside 6 AM - 10 PM
| project TimeGenerated, OperationName, CallerIPAddress,
identity_claim_oid_g, requestUri_s
| order by TimeGenerated desc
// Find secrets that have not been accessed in 90 days (candidates for cleanup)
let ActiveSecrets = AzureDiagnostics
| where ResourceProvider == "MICROSOFT.KEYVAULT"
| where OperationName == "SecretGet"
| where TimeGenerated > ago(90d)
| distinct SecretName = tostring(split(requestUri_s, "/")[4]);
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.KEYVAULT"
| where OperationName == "SecretSet"
| extend SecretName = tostring(split(requestUri_s, "/")[4])
| where SecretName !in (ActiveSecrets)
| summarize LastSet = max(TimeGenerated) by SecretName
| order by LastSet asc
// Monitor approaching secret expirations
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.KEYVAULT"
| where OperationName == "SecretList" or OperationName == "SecretGet"
| extend ExpiresOn = todatetime(exp_s)
| where isnotempty(ExpiresOn)
| where ExpiresOn between (now() .. now() + 30d)
| project SecretName = tostring(split(requestUri_s, "/")[4]),
ExpiresOn, DaysUntilExpiry = datetime_diff('day', ExpiresOn, now())
| order by DaysUntilExpiry ascDisaster Recovery and Backup
Key Vault provides built-in resilience through soft delete and purge protection, but a comprehensive DR strategy requires understanding recovery scenarios and implementing backup procedures for critical secrets.
Built-In Protection Features
- Soft delete (enabled by default): Deleted vaults and objects are recoverable for the retention period (7–90 days, default 90). This protects against accidental deletion by administrators.
- Purge protection: When enabled, prevents permanent deletion of vaults and objects even by administrators. The only way to permanently remove objects is to wait for the retention period to expire. This is required for CMK scenarios with Azure Storage and Azure SQL.
- Geo-replication: Key Vault automatically replicates data to a paired region. During a regional outage, the vault fails over to the secondary region in read-only mode within minutes. Once the primary region recovers, the vault returns to read-write mode.
Backup and Restore
Key Vault supports backup and restore of individual secrets, keys, and certificates. Backups are encrypted and can only be restored to a vault in the same Azure subscription and geography. This is useful for migrating between vaults or as additional protection beyond soft delete.
# Backup individual secrets
az keyvault secret backup \
--vault-name myapp-prod-kv \
--name db-connection-string \
--file ./backups/db-connection-string.backup
az keyvault secret backup \
--vault-name myapp-prod-kv \
--name external-api-key \
--file ./backups/external-api-key.backup
# Backup a key
az keyvault key backup \
--vault-name myapp-prod-kv \
--name storage-encryption-key \
--file ./backups/storage-encryption-key.backup
# Backup a certificate
az keyvault certificate backup \
--vault-name myapp-prod-kv \
--name wildcard-cert \
--file ./backups/wildcard-cert.backup
# Restore a secret to a different vault (same subscription and geo)
az keyvault secret restore \
--vault-name myapp-dr-kv \
--file ./backups/db-connection-string.backup
# Script: Backup all secrets in a vault
for secret_name in $(az keyvault secret list --vault-name myapp-prod-kv --query '[].name' -o tsv); do
echo "Backing up secret: $secret_name"
az keyvault secret backup \
--vault-name myapp-prod-kv \
--name "$secret_name" \
--file "./backups/secrets/${secret_name}.backup"
done
# Recover a soft-deleted vault
az keyvault recover --name myapp-prod-kv
# Recover a soft-deleted secret
az keyvault secret recover --vault-name myapp-prod-kv --name db-password
# List soft-deleted vaults in the subscription
az keyvault list-deleted --output tableBackup Limitations
Key Vault backups can only be restored to a vault in the same Azure subscription and the same Azure geography (e.g., a backup from East US can be restored to West US, but not to West Europe). Store backup files securely. They contain encrypted secret material that is tied to the Azure subscription's infrastructure keys. For cross-geography DR, maintain a separate vault in the target geography and sync secrets through your CI/CD pipeline or a custom synchronization process.
Key Vault Throttling and Performance
Key Vault has rate limits that can affect application performance if not properly managed. Understanding these limits and implementing appropriate caching strategies is essential for production workloads.
| Operation Type | Limit (Standard Tier) | Limit (Premium Tier) |
|---|---|---|
| GET operations (secrets, keys, certs) | 4,000 per 10 seconds per vault | 4,000 per 10 seconds per vault |
| PUT/PATCH/DELETE operations | 2,000 per 10 seconds per vault | 2,000 per 10 seconds per vault |
| RSA 2048-bit key operations | 2,000 per 10 seconds per vault | 2,000 per 10 seconds per vault (HSM) |
| RSA 3072/4096-bit key operations | 500 per 10 seconds per vault | 1,000 per 10 seconds per vault (HSM) |
| EC P-256 key operations | 2,000 per 10 seconds per vault | 2,000 per 10 seconds per vault (HSM) |
| Total transactions per vault | 4,000 per 10 seconds | 4,000 per 10 seconds |
Scaling Beyond Throttle Limits
If your application exceeds Key Vault rate limits, consider these strategies: (1) implement secret caching with a reasonable TTL (5–15 minutes), (2) spread secrets across multiple vaults if you have many applications, (3) use envelope encryption instead of direct key operations for high-throughput scenarios, and (4) for AKS, use the CSI Secrets Store driver which caches secrets at the node level, reducing the number of Key Vault calls.
Azure Policy for Key Vault Governance
Azure Policy enforces organizational standards across all Key Vaults. Use built-in policies and custom policy definitions to ensure every vault meets your security requirements.
# Require purge protection on all Key Vaults
az policy assignment create \
--name require-kv-purge-protection \
--display-name "Key Vault: Require purge protection" \
--policy "/providers/Microsoft.Authorization/policyDefinitions/0b60c0b2-2dc2-4e1c-b5c9-abbed971de53" \
--scope /subscriptions/<sub-id> \
--enforcement-mode Default
# Require RBAC authorization (not access policies)
az policy assignment create \
--name require-kv-rbac \
--display-name "Key Vault: Require RBAC authorization" \
--policy "/providers/Microsoft.Authorization/policyDefinitions/12d4fa5e-1f9f-4c21-97a9-b99b3c6611b5" \
--scope /subscriptions/<sub-id> \
--enforcement-mode Default
# Require Private Endpoints for Key Vault
az policy assignment create \
--name require-kv-private-endpoint \
--display-name "Key Vault: Require Private Endpoint" \
--policy "/providers/Microsoft.Authorization/policyDefinitions/a6abeaec-4d90-4a02-805f-6b26c4d3fbe9" \
--scope /subscriptions/<sub-id> \
--enforcement-mode Default
# Require minimum soft-delete retention (90 days)
az policy assignment create \
--name require-kv-soft-delete-90d \
--display-name "Key Vault: Minimum 90-day soft-delete retention" \
--policy "/providers/Microsoft.Authorization/policyDefinitions/fbb52984-0eca-4a72-8c20-51e1d524da85" \
--scope /subscriptions/<sub-id> \
--params '{"minimumRetentionDays": {"value": 90}}' \
--enforcement-mode Default
# Require secrets to have expiration dates set
az policy assignment create \
--name require-kv-secret-expiry \
--display-name "Key Vault: Secrets must have expiration date" \
--policy "/providers/Microsoft.Authorization/policyDefinitions/98728c90-32c7-4049-8429-847dc0f4fe37" \
--scope /subscriptions/<sub-id> \
--enforcement-mode Default
# Audit Key Vaults without diagnostic logging
az policy assignment create \
--name audit-kv-diagnostics \
--display-name "Key Vault: Audit missing diagnostic settings" \
--policy "/providers/Microsoft.Authorization/policyDefinitions/cf820ca0-f99e-4f3e-84fb-66e913812d21" \
--scope /subscriptions/<sub-id> \
--enforcement-mode DefaultBest Practices Summary
After working with hundreds of Key Vault implementations, these practices consistently separate well-secured environments from those with gaps.
Architecture
- Use one vault per application per environment. The vault isolation limits blast radius and simplifies access control.
- Enable purge protection and set retention to 90 days. This prevents permanent data loss even from administrative errors or compromised accounts.
- Use Azure RBAC (not access policies) for all new vaults. Migrate existing vaults from access policies to RBAC during your next security review.
Access Control
- Always use managed identities for application access. Never store Key Vault credentials (client secrets, certificates) in application code or configuration.
- Apply least-privilege roles. Applications get Secrets User or Crypto User, never Administrator.
- Use PIM (Privileged Identity Management) for human access to production vaults. Require just-in-time activation with approval for Key Vault Administrator.
Secrets
- Set expiration dates on all secrets. Use Azure Policy to enforce this across the organization.
- Implement rotation for all secrets, either automated (preferred) or with near-expiry alerts and runbooks for manual rotation.
- Cache secrets in application memory with a reasonable TTL. Avoid reading from Key Vault on every request.
Network and Monitoring
- Disable public access and use Private Endpoints for production vaults. Allow trusted Azure services bypass for CMK and Key Vault references.
- Enable diagnostic logging to Log Analytics. Retain audit logs for at least one year for compliance.
- Alert on 403 (unauthorized) responses and unusual access patterns. Investigate every alert; these may indicate credential compromise or misconfiguration.
Key Vault Security Checklist
Run this checklist for every production vault: (1) RBAC authorization enabled, (2) purge protection enabled with 90-day retention, (3) public access disabled with Private Endpoint, (4) diagnostic logging to Log Analytics, (5) all secrets have expiration dates, (6) rotation policies or near-expiry alerts configured, (7) no application identities with Administrator role, (8) Azure Policy enforcing standards. If any item is unchecked, prioritize fixing it.
Key Takeaways
- 1Use separate Key Vaults per environment (dev, staging, production) and per application.
- 2Prefer Azure RBAC over legacy access policies for Key Vault access control.
- 3Enable soft-delete and purge protection to prevent accidental or malicious key loss.
- 4Use managed identities for Key Vault access and never store Key Vault credentials in code.
- 5Set up automatic secret rotation using Event Grid and Azure Functions.
- 6Monitor Key Vault access with diagnostic logs and set alerts for suspicious operations.
Frequently Asked Questions
What is the difference between Key Vault secrets, keys, and certificates?
Should I use access policies or Azure RBAC for Key Vault?
How do I rotate secrets in Key Vault?
What is soft-delete and purge protection?
How should I reference Key Vault secrets in applications?
Written by CloudToolStack Team
Cloud engineers and architects with hands-on experience across AWS, Azure, and GCP. We write guides based on real-world production patterns, not just documentation rewrites.
Disclaimer: This guide is for educational purposes. Cloud services change frequently; always refer to official documentation for the latest information. AWS, Azure, and GCP are trademarks of their respective owners.