Verifying GCP Workload Identity Federation (WIF) from CI/CD
Why long-lived service account JSON keys are a major liability and how to test and verify GCP Workload Identity Federation token exchange cleanly.
In modern enterprise DevOps, storing static service account JSON keys in GitLab CI variables or GitHub Actions secrets is an anti-pattern. If a key is leaked, copied into job logs, or stored indefinitely without rotation, it represents a severe security vulnerability.
Google Cloud's Workload Identity Federation (WIF) eliminates static credentials entirely. It allows external workloads (GitLab, GitHub, Kubernetes clusters) to authenticate to GCP by exchanging an ephemeral OIDC (OpenID Connect) token for short-lived Google Cloud access tokens.
However, when configuring WIF for the first time, debugging a failed token exchange can be frustrating. Pipelines fail with opaque 403 Forbidden or invalid_grant errors.
Here is a straightforward runbook for validating your WIF setup cleanly using step-by-step CLI verification.
1. How the WIF Flow Works
Before debugging, understand the three-step token exchange:
[CI Runner]
│ 1. Generates short-lived OIDC JWT (e.g., CI_JOB_JWT_V2 in GitLab or ACTIONS_ID_TOKEN_REQUEST in GitHub)
▼
[GCP Security Token Service (STS)]
│ 2. Validates JWT signature, issuer, and claims against Workload Identity Pool Provider
▼
[GCP Service Account Impersonation]
│ 3. Exchanges STS token for short-lived GCP Service Account OAuth2 access token
▼
[GCP Resources (GKE, Cloud Storage, Artifact Registry)]
If any single link in this chain has an attribute mismatch, the exchange fails.
2. Inspecting the OIDC Token First
The most common reason for failure is an attribute condition or attribute mapping mismatch (e.g., expecting assertion.sub to match a specific repository path or branch).
Inside your CI pipeline, output the raw decoded payload of your runner's OIDC token.
In GitLab CI:
test_oidc_token:
stage: test
id_tokens:
MY_OIDC_TOKEN:
aud: "https://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/MY_POOL/providers/MY_PROVIDER"
script:
# Print JWT payload (middle section between dots)
- echo "$MY_OIDC_TOKEN" | cut -d'.' -f2 | base64 -d 2>/dev/null || true
In GitHub Actions:
jobs:
inspect-token:
runs-on: ubuntu-latest
permissions:
id-token: write
steps:
- name: Inspect Claims
run: |
echo "Audience and issuer claims should match GCP pool provider config."
Inspect the decoded JSON:
- Check
iss(Issuer URL): Does it match the issuer configured on your GCP Pool Provider? - Check
aud(Audience): Does it match the full resource name of your provider? - Check
suborproject_path: Are repository names, usernames, or branch references matching the exact case and spelling configured in your IAM bindings?
3. Manually Testing Token Exchange via cURL
You do not need to push 20 dummy commits to test your pipeline. You can capture the OIDC token from your runner and perform the two-step exchange via curl to identify the exact error response.
Step 1: Exchange OIDC Token with GCP STS
OIDC_TOKEN="<your_captured_jwt_token>"
PROVIDER_RESOURCE="//iam.googleapis.com/projects/<PROJECT_NUM>/locations/global/workloadIdentityPools/<POOL_NAME>/providers/<PROVIDER_NAME>"
# Request federated STS token
STS_RESPONSE=$(curl -s -X POST https://sts.googleapis.com/v1/token \
-H "Content-Type: application/json" \
-d "{
\"audience\": \"$PROVIDER_RESOURCE\",
\"grantType\": \"urn:ietf:params:oauth:grant-type:token-exchange\",
\"requestedTokenType\": \"urn:ietf:params:oauth:token-type:access_token\",
\"scope\": \"https://www.googleapis.com/auth/cloud-platform\",
\"subjectTokenType\": \"urn:ietf:params:oauth:token-type:id_token\",
\"subjectToken\": \"$OIDC_TOKEN\"
}")
echo "$STS_RESPONSE"
If this step fails:
400 Bad Request / invalid_grant: The issuer URL, audience, or token expiration is rejected by the provider.403 Forbidden: Your attribute condition on the provider evaluated tofalse(e.g., the branch or ref was disallowed).
Step 2: Exchange STS Token for Service Account Access Token
If Step 1 succeeds, extract the access_token from $STS_RESPONSE and impersonate the target Google Service Account:
FEDERATED_TOKEN=$(echo "$STS_RESPONSE" | jq -r '.access_token')
SA_EMAIL="ci-deployer@your-gcp-project.iam.gserviceaccount.com"
SA_RESPONSE=$(curl -s -X POST \
"https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${SA_EMAIL}:generateAccessToken" \
-H "Authorization: Bearer $FEDERATED_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"scope": ["https://www.googleapis.com/auth/cloud-platform"],
"lifetime": "3600s"
}')
echo "$SA_RESPONSE"
If this step fails:
403 Permission Denied: The target Service Account is missing theroles/iam.workloadIdentityUserbinding. The member binding must match your federated identity:textprincipalSet://iam.googleapis.com/projects/<PROJECT_NUM>/locations/global/workloadIdentityPools/<POOL_NAME>/attribute.repository/<REPO_NAME>
4. Testing GCP Access with the Token
Once Step 2 returns an access token (ya29...), test an actual GCP API call:
ACCESS_TOKEN=$(echo "$SA_RESPONSE" | jq -r '.accessToken')
# Test listing storage buckets or GKE clusters
curl -s -H "Authorization: Bearer $ACCESS_TOKEN" \
"https://storage.googleapis.com/storage/v1/b?project=your-gcp-project"
If this returns your bucket list, your federation pipeline is 100% verified.
Summary
WIF issues are almost never random; they stem from:
- Mismatched
aud/ provider resource names. - Inaccurate attribute condition filters (regex or string matching).
- Missing
roles/iam.workloadIdentityUserpermission on the destination Service Account.
Deconstructing the exchange into raw cURL steps isolates whether the failure is at the STS provider layer or the Service Account impersonation layer in seconds.
Technical Discussion (0)
Ask a question or reply to a thread below. Your email is never displayed publicly.
No comments yet. Start the technical discussion above.