A pinned client trusts one public key, not a whole CA, so the only rotation that can break it is a key change nobody planned for. SPKI certificate pinning against a GCP Private CA leaf gives you that control, because you issue the certificate, you hold the key and you decide when the key moves. Ship the backup pin before that day and the rotation is a non-event. Skip it and every pinned client fails closed at once.
We built the whole loop on real infrastructure: a DevOps-tier CA Service pool, leaves issued from CSRs, pins computed with openssl, a pinned Python client and curl --pinnedpubkey, then the outage after an unplanned key change and the backup pin that avoids it. Every command was run in November 2026 against CA Service in europe-west1 from Ubuntu 26.04 with OpenSSL 3.5.
Choose What to Pin
Pin the SPKI hash of a leaf key you issue yourself, and always carry at least one backup pin. The SPKI pin is the base64 SHA-256 of the certificate’s DER-encoded SubjectPublicKeyInfo, the format RFC 7469 defined for HTTP header pinning (which browsers have since dropped) and the one curl, OkHttp and most mobile pinning libraries still use. Because it hashes only the public key, a renewed certificate on the same key keeps the same pin, so routine renewals never touch your clients.
The three things people call pinning behave very differently when the certificate changes, and the table shows why we pin the leaf key rather than the certificate or the root.
| Pin target | Same-key renewal | New-key rotation | Verdict |
|---|---|---|---|
| Whole certificate (DER hash) | Breaks | Breaks | Too brittle for any renewal cadence |
| Leaf SPKI (this guide) | Holds | Breaks unless pre-pinned | Use it, with a backup pin |
| Root or intermediate SPKI | Holds | Holds | Only proves the issuer, not the server key |
Pin only when you ship both ends of the connection, such as your mobile app talking to your API or your own service-to-service client. Never pin for third-party clients, because you cannot push them a new pin set before a rotation, and they will break on your schedule instead of theirs. The same logic rules out pinning a Google-managed wildcard from DNS-authorized Certificate Manager certs, since Google rotates those keys on its own timetable and a pin on them fails without warning. A private CA is where pinning makes sense, because every key change goes through you.
Set Up the Lab Hosts
You need two places to work: a workstation with an authenticated gcloud for the CA Service calls, and a Linux server that holds the private keys, runs nginx and runs the pinned client. In our lab the workstation is admin01 and the server is web01, an e2-small Ubuntu 26.04 VM in europe-west1-b. The private keys never leave web01, so only CSRs and signed certificates travel between the two, which is the same split you want in production.
Install nginx and the Python cryptography package on web01. The client needs cryptography because the standard ssl module hands you the peer certificate as bytes but has no call that extracts its public key.
sudo apt-get update
sudo apt-get install -y nginx python3-cryptography
The client and server share one VM here, so point the service name at loopback and make a working directory. On a real client, DNS does this job.
echo "127.0.0.1 pay.internal.example.com" | sudo tee -a /etc/hosts
mkdir -p ~/pki && cd ~/pki
These are the versions the rest of the guide was captured with on web01.

Any OpenSSL 3.x build and a current cryptography wheel compute identical hashes, because the pin depends only on the key bytes and SHA-256. On the workstation, gcloud version reported Google Cloud SDK 591.0.0. If yours is much older, update it before running the privateca commands, since flag names in that group have changed over the years and an old SDK fails with confusing argument errors.
Create a DevOps-Tier CA Pool and Root CA
A DevOps-tier pool with one EC P-256 root is enough for pinning, because the client checks the leaf key and the root only has to anchor the chain. Enable CA Service in the project first, otherwise every privateca call fails with a permission error that hides the real cause.
gcloud services enable privateca.googleapis.com
Create the pool, then the root CA inside it. The --max-chain-length=0 flag lets the root sign leaf certificates but no subordinate CAs, and --auto-enable saves a separate enable call.
gcloud privateca pools create pay-devops-pool --location=europe-west1 --tier=devops --labels=env=lab
gcloud privateca roots create pay-root-ca --pool=pay-devops-pool --location=europe-west1 --subject="CN=Pay Internal Root CA, O=Example Pay Ltd" --key-algorithm=ec-p256-sha256 --max-chain-length=0 --validity=P5Y --auto-enable --labels=env=lab
Then export the root certificate. This file is the trust anchor every client loads, so it ships with the client the same way the pins do.
gcloud privateca roots describe pay-root-ca --pool=pay-devops-pool --location=europe-west1 --format="value(pemCaCertificates)" > root-ca.pem
openssl x509 -in root-ca.pem -noout -subject -enddate
The root creation sits on a row of progress dots for a while before it reports done and enables the CA. The subject and the five-year expiry confirm you exported the right certificate.

Pick the tier on purpose. Google’s CA Service tier table lists DevOps as having no support for listing, describing or revoking certificates, while Enterprise has all three. For short-lived service certificates that trade is fine, because the pin and a 30-day lifetime are your revocation story anyway. For a regulated workload that needs a revocation trail, use the Enterprise setup from deploying GCP Private CA for financial service certs instead.
Generate the Primary and Backup Keys
Create two keys on day one: the primary that goes live now and the backup you will rotate to later. The backup key is what makes the later rotation safe, because its pin can ship to clients months before any certificate exists for it. Generate both on web01 and lock their permissions.
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out pay-a.key
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out pay-b.key
chmod 600 pay-a.key pay-b.key
Build a CSR for each key with the service name in the SAN. Clients check the SAN against the hostname they dialled, and the CN alone no longer counts, so a CSR without the SAN gives you a certificate that fails before the pin is even checked.
openssl req -new -key pay-a.key -out pay-a.csr -subj "/CN=pay.internal.example.com/O=Example Pay Ltd" -addext "subjectAltName=DNS:pay.internal.example.com"
openssl req -new -key pay-b.key -out pay-b.csr -subj "/CN=pay.internal.example.com/O=Example Pay Ltd" -addext "subjectAltName=DNS:pay.internal.example.com"
Store the backup key the way you store anything that can impersonate the service, in Secret Manager, an HSM-backed KMS key or an offline vault, never as a second copy next to the live key. If the backup leaks along with the primary, the backup pin protects nothing, because an attacker holding either key passes every client check. Once its pin is recorded, the backup private key should exist in exactly one place until rotation day, and the CSR is all the server needs to request its certificate.
Get a Leaf Certificate from the Pool
Issue the primary certificate from its CSR, so CA Service signs the key you already hold and never sees it. Copy the CSRs to the workstation first.
scp web01:pki/pay-a.csr web01:pki/pay-b.csr .
Now issue a 30-day certificate for the primary key and list the pool. We left the labels flag in on purpose, to show what the DevOps tier does with it.
gcloud privateca certificates create --issuer-pool=pay-devops-pool --issuer-location=europe-west1 --csr=pay-a.csr --cert-output-file=pay-a.pem --validity=P30D --labels=env=lab
gcloud privateca certificates list --issuer-pool=pay-devops-pool --location=europe-west1
gcloud prints a warning that the labels are dropped, and the list call returns nothing even though a certificate was just issued.

Both come from the DevOps tier not exposing certificate lifecycle, so CA Service gives you no way to list or describe what you issued. Keep your own inventory of serials, keys and expiry dates, because nothing in CA Service will tell you which leaf is about to expire. The output file holds two certificates, the leaf followed by the root, which is exactly what nginx expects for ssl_certificate.
Copy the bundle and the root back to the server.
scp pay-a.pem root-ca.pem web01:pki/
Every later certificate in this guide uses the same certificates create command with a different CSR and output file, minus the labels flag, so we won’t repeat it in full each time.
Generate the SPKI Pin with OpenSSL
The pin is four OpenSSL stages: pull the public key, convert it to DER, hash it with SHA-256 and base64 the raw digest. Run it against the issued certificate on web01.
openssl x509 -in pay-a.pem -pubkey -noout | openssl pkey -pubin -outform DER | openssl dgst -sha256 -binary | openssl enc -base64
You can get the same value without any certificate at all, straight from the key or its CSR. That is the step that makes a backup pin possible, because pay-b has no certificate yet.
openssl pkey -in pay-b.key -pubout -outform DER | openssl dgst -sha256 -binary | openssl enc -base64
openssl req -in pay-b.csr -pubkey -noout | openssl pkey -pubin -outform DER | openssl dgst -sha256 -binary | openssl enc -base64
Each key produces one 44-character pin, whichever artifact you hash.

In our run the certificate and key for pay-a both gave i0csBYis...4S8=, and the key and CSR for pay-b both gave AWnaOIbD...3JU=. If two of those disagree for the same key, you hashed the wrong encoding, usually PEM text instead of DER bytes or the whole certificate instead of the public key. A pin computed the wrong way fails on every connection, so check it against a live server before it ships to any client.
Configure Nginx to Serve the Certificate
Nginx stands in for your API here, serving the CA Service leaf and returning a one-line body so the client has something to read. Create the TLS directory and open the site file.
sudo mkdir -p /etc/nginx/tls
sudo vim /etc/nginx/sites-available/pay.conf
Paste this server block. The certificate and key paths stay fixed through every rotation, so swapping keys later is a file copy and a reload rather than a config change.
server {
listen 443 ssl;
server_name pay.internal.example.com;
ssl_certificate /etc/nginx/tls/pay.pem;
ssl_certificate_key /etc/nginx/tls/pay.key;
ssl_protocols TLSv1.2 TLSv1.3;
location / {
return 200 "pay api ok\n";
}
}
Enable the site, install the primary certificate and key, and reload. The private key goes in with mode 600 so only root can read it.
sudo ln -s /etc/nginx/sites-available/pay.conf /etc/nginx/sites-enabled/pay.conf
sudo install -m 644 pay-a.pem /etc/nginx/tls/pay.pem
sudo install -m 600 pay-a.key /etc/nginx/tls/pay.key
sudo nginx -t
sudo systemctl reload nginx
Confirm the chain validates against the exported root before adding any pins, so a pin failure later can’t be confused with a trust failure.
curl -s --cacert root-ca.pem https://pay.internal.example.com/
A plain pay api ok means nginx is serving the CA Service leaf and the chain checks out. A certificate verify error at this point is a chain or SAN problem, not a pinning one, so fix it here before you go further. Debugging trust and pinning at the same time is how a ten-minute job turns into an afternoon.
Add the Pin to a Python Client
The client runs the normal TLS checks first, then hashes the server’s public key and refuses the connection if the hash is not in its pin set. Keeping the pins in a file rather than in the code means a pin update is a config push, which matters on the day you rotate. Create the client on web01.
vim pinned_client.py
This is the exact file we used for every run below.
import base64, hashlib, socket, ssl, sys
from cryptography import x509
from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat
HOST = "pay.internal.example.com"
PORT = 443
ROOT_CA = "root-ca.pem"
def load_pins(path):
with open(path) as f:
return {line.split("#")[0].strip() for line in f if line.split("#")[0].strip()}
def spki_pin(cert_der):
cert = x509.load_der_x509_certificate(cert_der)
spki = cert.public_key().public_bytes(Encoding.DER, PublicFormat.SubjectPublicKeyInfo)
return base64.b64encode(hashlib.sha256(spki).digest()).decode()
def connect(pins):
ctx = ssl.create_default_context(cafile=ROOT_CA)
with socket.create_connection((HOST, PORT), timeout=5) as raw:
with ctx.wrap_socket(raw, server_hostname=HOST) as tls:
observed = spki_pin(tls.getpeercert(binary_form=True))
if observed not in pins:
raise ssl.SSLError(f"SPKI pin mismatch: server key {observed} is not in the pin set")
tls.sendall(f"GET / HTTP/1.1\r\nHost: {HOST}\r\nConnection: close\r\n\r\n".encode())
body = tls.recv(4096).decode().split("\r\n\r\n", 1)[1].strip()
print(f"pin OK ({observed}) {tls.version()} -> {body}")
if __name__ == "__main__":
connect(load_pins(sys.argv[1] if len(sys.argv) > 1 else "pins.txt"))
Three checks stack here. create_default_context(cafile=...) trusts only the CA Service root, and it turns on hostname checking and CERT_REQUIRED by default. The pin check then runs after the handshake and before any request bytes go out, so a wrong key never sees application data. Start the pin set with the primary key only, since that is the state most teams ship and the one that breaks later.
echo "i0csBYisJdUQ9MVk+S4ZIDVHo1ApIf2TrACPmEqM4S8= # pay-a, live" > pins.txt
python3 pinned_client.py pins.txt
curl enforces the same pin with --pinnedpubkey, which takes sha256// followed by the base64 hash. It is the quickest way to check a pin from a shell or a health probe.
curl -s --cacert root-ca.pem --pinnedpubkey "sha256//i0csBYisJdUQ9MVk+S4ZIDVHo1ApIf2TrACPmEqM4S8=" https://pay.internal.example.com/
Both pass, and the openssl s_client pipeline in the middle of the capture shows the key nginx is actually serving hashes to the same value.

Keep that live-server check as a script, because the rotation steps below poll it and you will want it in your health checks too. Create served-pin.sh.
vim served-pin.sh
It prints the SPKI pin of whatever key an endpoint is serving right now.
#!/bin/bash
# prints the SPKI pin of the key a TLS endpoint is serving right now
host=${1:?usage: served-pin.sh host [port]}
openssl s_client -connect "$host:${2:-443}" -servername "$host" </dev/null 2>/dev/null \
| openssl x509 -pubkey -noout \
| openssl pkey -pubin -outform DER \
| openssl dgst -sha256 -binary \
| openssl enc -base64
Make it executable with chmod +x served-pin.sh. Pointed at a server you don’t control, it also shows what key a vendor endpoint presents, and that answer should come before any decision to pin a vendor at all.
Keep the Same Key on Renewal
Renewing from the same CSR gives a new certificate with a new serial and new dates but the same public key, so no pinned client notices. Issue the renewal from pay-a.csr on the workstation.
gcloud privateca certificates create --issuer-pool=pay-devops-pool --issuer-location=europe-west1 --csr=pay-a.csr --cert-output-file=pay-a-renewed.pem --validity=P30D
After copying pay-a-renewed.pem to web01, compare serial and pin on both certificates, then swap the renewal in and run the client with the unchanged pin file.
for f in pay-a.pem pay-a-renewed.pem; do echo "$f serial=$(openssl x509 -in $f -noout -serial | cut -d= -f2) pin=$(openssl x509 -in $f -pubkey -noout | openssl pkey -pubin -outform DER | openssl dgst -sha256 -binary | openssl enc -base64)"; done
sudo install -m 644 pay-a-renewed.pem /etc/nginx/tls/pay.pem && sudo systemctl reload nginx
python3 pinned_client.py pins.txt
The serials differ, 81D2F955... against 0FCAECB1..., and the pins match.

The client passes without any change on its side, which is why same-key renewal suits the routine 30 or 90 day cycle. Don’t make it permanent, though. A key that never changes is a key with unlimited time to leak, so plan a scheduled key change at least yearly and run it through the backup pin flow below.
Switch to an Unpinned Key
Rotating to a key the clients have never seen breaks every pinned client at once, even though the new certificate is perfectly valid. To reproduce the outage, generate a third key that is not in any pin set.
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out pay-c.key && chmod 600 pay-c.key
openssl req -new -key pay-c.key -out pay-c.csr -subj "/CN=pay.internal.example.com/O=Example Pay Ltd" -addext "subjectAltName=DNS:pay.internal.example.com"
Issue pay-c.pem from that CSR with the same certificates create command and copy it back. Put it live, then test with a plain curl, the pinned client and a pinned curl.
sudo install -m 644 pay-c.pem /etc/nginx/tls/pay.pem && sudo install -m 600 pay-c.key /etc/nginx/tls/pay.key && sudo systemctl reload nginx
curl -s --cacert root-ca.pem https://pay.internal.example.com/
python3 pinned_client.py pins.txt
curl -sS --cacert root-ca.pem --pinnedpubkey "sha256//i0csBYisJdUQ9MVk+S4ZIDVHo1ApIf2TrACPmEqM4S8=" https://pay.internal.example.com/; echo "exit=$?"
Unpinned curl still gets pay api ok, because the chain is valid. Both pinned clients refuse the connection.

That split is what makes pin outages hard to diagnose. Browsers, monitoring probes and anything unpinned report the service healthy while every pinned app fails, so the dashboard stays green during the incident. Watch the pinned client’s error rate during a rotation, not the load balancer’s, and give at least one synthetic probe the same pin set your apps carry.
Error: “SPKI pin mismatch: server key … is not in the pin set”
The Python client raises ssl.SSLError with the observed pin, here jv32WyU0...Zm8=, which belongs to pay-c. Never answer it by disabling the check, because that turns the outage into a silent downgrade. Either put the old key back on the server, which restores service in seconds, or push a pin set that contains the new key. Rolling back is the right call when the clients are mobile apps that take days to update.
Error: “curl: (90) SSL: public key does not match pinned public key”
Exit code 90 is curl’s CURLE_SSL_PINNEDPUBKEYNOTMATCH, and per the libcurl documentation it aborts before sending or receiving any data. In a health check script, treat 90 as its own alert, separate from the 60 you get for an untrusted chain, because the two need different people. A 90 means somebody changed a key, while a 60 means the trust store or the chain is wrong.
Move to the Backup Pin
Ship the backup pin before the key changes, and the rotation passes on the first connection. Put the primary key back on the server, then give the client a pin set holding the live key and the backup key whose pin you computed from pay-b.key.
sudo install -m 644 pay-a-renewed.pem /etc/nginx/tls/pay.pem && sudo install -m 600 pay-a.key /etc/nginx/tls/pay.key && sudo systemctl reload nginx
vim pins.txt
The pin file now carries both keys, with a comment saying which is which.
i0csBYisJdUQ9MVk+S4ZIDVHo1ApIf2TrACPmEqM4S8= # pay-a, live
AWnaOIbDewE0neJu374NUzo4zGlFKkTmPsFyk1T03JU= # pay-b, backup (key only, no cert yet)
Issue pay-b.pem from pay-b.csr on the workstation and copy it over. Then swap the key in and poll until nginx really serves it before testing.
sudo install -m 644 pay-b.pem /etc/nginx/tls/pay.pem && sudo install -m 600 pay-b.key /etc/nginx/tls/pay.key && sudo systemctl reload nginx
until [ "$(./served-pin.sh pay.internal.example.com)" = "AWnaOIbDewE0neJu374NUzo4zGlFKkTmPsFyk1T03JU=" ]; do sleep 1; done; echo "nginx now serves pay-b"
python3 pinned_client.py pins.txt
The client passes on the old key before the swap and on the backup key after it, with no client change in between.

The polling loop is there because of what went wrong on our first attempt. Without it, the client ran straight after systemctl reload nginx returned and still reported the old pay-a pin, and a second run moments later saw pay-b. The reload returns once nginx has been signalled, not once the new workers own every connection, so a rotation script that verifies immediately can pass against the old key and report a rotation that never happened.
Curl accepts a pin set too, with the hashes separated by semicolons. Use this form in probes during the overlap window, since either key is acceptable until the old one is retired.
curl -s --cacert root-ca.pem --pinnedpubkey "sha256//i0csBYisJdUQ9MVk+S4ZIDVHo1ApIf2TrACPmEqM4S8=;sha256//AWnaOIbDewE0neJu374NUzo4zGlFKkTmPsFyk1T03JU=" https://pay.internal.example.com/
Once the rotation has soaked, retire pay-a from the pin set and add a fresh backup. Generate the next key now, while nothing is on fire, and take its pin from the key.
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out pay-d.key && chmod 600 pay-d.key
openssl pkey -in pay-d.key -pubout -outform DER | openssl dgst -sha256 -binary | openssl enc -base64
The new pin set is the live pay-b plus the pay-d backup, and the client still passes against pay-b.
AWnaOIbDewE0neJu374NUzo4zGlFKkTmPsFyk1T03JU= # pay-b, live
QGWNhGijBtyuYT+U0F7UshlYupf/N0ZJxH3uqBbJPHE= # pay-d, backup
That is the steady state to aim for: two pins, one live and one waiting, with the backup’s private key stored away from the server. If you ever find a client running with a single pin, treat it as an outage that hasn’t happened yet, because the next key change will take it down.
Run the Rotation in Production
A production key rotation is the lab sequence with a wait for client adoption in the middle, and the order matters more than the tooling.
- Generate the next key and CSR, compute its pin from the key
- Ship a client release or config push that adds the pin
- Wait until client telemetry shows the new pin set across the fleet
- Issue the certificate from CA Service and deploy it
- Poll the served pin, then watch pinned-client errors
- Drop the old pin and add the next backup
Step 3 is the one teams cut short. The calendar says nothing about how many installs still run last quarter’s pin set, so only telemetry can tell you it is safe to deploy. The zero-incident cert rotation demo on GCP runs a live traffic generator through a rotation, and the Cloud Monitoring runbook for cert rotations covers the alerting you want watching it.
Pinning narrows one risk and leaves others alone. It does not help against these three.
- A leaked leaf private key, which still matches the pin
- A compromised client device or patched app binary
- A bad pin pushed by your own release pipeline
The first is why the leaf key rotates on a schedule, the second is out of scope for TLS entirely, and the third is why the pin file goes through review like code. CAA records from the Cloud DNS DNSSEC and CAA setup cover the public side, while the Terraform and ArgoCD cert inventory keeps new services from quietly issuing outside the pool. For the curl side of your probes, the curl cheat sheet lists the TLS flags worth keeping next to --pinnedpubkey.
Delete the CA Pool and Lab Resources
The teardown runs in the order our lab used: disable the root CA, delete it with --skip-grace-period, then delete the pool, which succeeded on the first try once the CA was gone. The --ignore-active-certificates flag is needed because the issued leaves are still inside their 30 days.
gcloud privateca roots disable pay-root-ca --pool=pay-devops-pool --location=europe-west1 --quiet
gcloud privateca roots delete pay-root-ca --pool=pay-devops-pool --location=europe-west1 --skip-grace-period --ignore-active-certificates --quiet
gcloud privateca pools delete pay-devops-pool --location=europe-west1 --quiet
gcloud warns that the CA can not be undeleted, and it means it.

With the root gone, nothing can issue another leaf that chains to root-ca.pem, so every client that trusts it is stranded at the next renewal. Never run this against a pool that still backs production traffic. Finish by deleting the key material and the VM, keys first, because they are the only part of this lab that can impersonate the service.
rm -f ~/pki/pay-*.key
gcloud compute instances delete web01 --zone=europe-west1-b --quiet
gcloud privateca pools list --location=europe-west1
An empty pool list confirms nothing is left billing in the region. On a real service you would keep the pool and run the rotation section on a schedule, with the backup pin always one step ahead of the key in production.