Below is an aggregation of solutions to some issues that cert-manager users may face:
- TLS Terminology, including commonly misused terms
- Troubleshooting issuing ACME certificates
- How to change the Cluster Resource Namespace
- How to sync secrets across namespaces
- Failing to create resources due to Webhook
Can I trigger a renewal from cert-manager at will?
This is a feature in cert-manager starting in
v0.16 using the
cmctl CLI. More information can be found on the renew command's page
When do certs get re-issued?
To determine if a certificate needs to be re-issued, cert-manager looks at the the spec of
Certificate resource and latest
CertificateRequests as well as the data in
Secret containing the X.509 certificate.
The issuance process will always get triggered if the:
Certificate's spec, does not exist, is missing private key or certificate data or contains corrupt data
- private key stored in the
Secretdoes not match the private key spec on
- public key of the issued certificate does not match the private key stored in the
- cert-manager issuer annotations on the
Secretdo not match the issuer specified on the
- DNS names, IP addresses, URLS or email addresses on the issued certificate do not match those on the
- certificate needs to be renewed (because it has expired or the renewal time is now or in the past)
- certificate has been marked for renewal manually using
Additionally, if the latest
CertificateRequest for the
Certificate is found, cert-manager will also re-issue if:
- the common name on the CSR found on the
CertificateRequestdoes not match that on the
- the subject fields on the CSR found on the
CertificateRequestdo not match the subject fields of the
- the duration on the
CertificateRequestdoes not match the duration on the
isCAfield value on the
Certificatespec does not match that on the
- the DNS names, IP addresses, URLS or email addresses on the
CertificateRequestspec do not match those on the
- key usages on the
CertificateRequestspec do not match those on the
Note that for certain fields re-issuance on change gets triggered only if there
CertificateRequest that cert-manager can use to determine whether
Certificate's spec has changed since the previous issuance. This is because
some issuers may not respect the requested values for these fields, so we cannot
rely on the values in the issued X.509 certificates. One such field is
.spec.duration- change to this field will only trigger re-issuance if there is
CertificateRequest to compare with. In case where you need to re-issue, but
re-issuance does not get triggered automatically due to there being no
CertificateRequest (i.e after backup and restore), you can use
cmctl renew to trigger it manually.
Why isn't my root certificate in my issued Secret's
Occasionally, people work with systems which have made a flawed choice regarding TLS chains. The TLS spec has the following section for the "Server Certificate" section of the TLS handshake:
This is a sequence (chain) of certificates. The sender's certificate MUST come first in the list. Each following certificate MUST directly certify the one preceding it. Because certificate validation requires that root keys be distributed independently, the self-signed certificate that specifies the root certificate authority MAY be omitted from the chain, under the assumption that the remote end must already possess it in order to validate it in any case.
In a standard, secure and correctly configured TLS environment, adding a root certificate to the chain is almost always unnecessary and wasteful.
There are two ways that a certificate can be trusted:
- explicitly, by including it in a trust store.
- through a signature, by following the certificate's chain back up to an explicitly trusted certificate.
Crucially, root certificates are by definition self-signed and they cannot be validated through a signature.
As such, if we have a client trying to validate the certificate chain sent by the server, the client must already have the root before the connection is started. If the client already has the root, there was no point in it being sent by the server!
The same logic with not sending root certificates applies for servers trying to validate client certificates; the same justification is given in the TLS RFC.
How can I see all the historic events related to a certificate object?
cert-manager publishes all events to the Kubernetes events mechanism, you can get the events for your specific resources using
kubectl describe <resource> <name>.
Due to the nature of the Kubernetes event mechanism these will be purged after a while. If you're using a dedicated logging system it might be able or is already also storing Kubernetes events.
What happens if issuance fails? Will it be retried?
cert-manager will retry a failed issuance except for a few rare edge cases where manual intervention is needed.
If an issuance fails because of a temporary error, it will be retried again with a short exponential backoff (currently 5 seconds to 5 minutes). A temporary error is one that does not result in a failed
If the issuance fails with an error that resulted in a failed
CertificateRequest, it will be retried with a longer binary exponential backoff (1 hour to 32 hours) to avoid overwhelming external services.
You can always trigger immediate renewal using the
cmctl renew command
Is ECC (elliptic-curve cryptography) supported?
cert-manager supports ECDSA key pairs! You can set your certificate to use ECDSA in the
privateKey part of your Certificate resource.
apiVersion: cert-manager.io/v1kind: Certificatemetadata:name: ecdsaspec:secretName: ecdsa-certisCA: falseprivateKey:algorithm: ECDSAsize: 256dnsNames:- ecdsa.example.comissuerRef:[...]
duration is not defined, what will be the default value?
duration is 90 days. If
renewBefore has not been set,
Certificate will be renewed 2/3 through its actual duration.
Kubernetes has a builtin
CertificateSigningRequest API. Why not use that?
Kubernetes has a Certificate Signing Requests API,
kubectl certificates command which allows you to approve certificate signing requests
and have them signed by the certificate authority (CA) of the Kubernetes cluster.
This API and CLI have occasionally been misused to sign certificates for use by non-control-plane Pods but this is a mistake. For the security of the Kubernetes cluster, it is important to limit access to the Kubernetes certificate authority, and it is important that you do not use that certificate authority to sign certificates which are used outside of the control-plane, because such certificates increase the opportunity for attacks on the Kubernetes API server.
In Kubernetes 1.19 the Certificate Signing Requests API has reached V1 and it can be used more generally by following (or automating) the Request Signing Process.
cert-manager currently has some limited experimental support for this resource.