Release Notes

Final Release `v1.4.0`

Special thanks to the external contributors who contributed to this release:

Deprecated Features and Breaking Changes

Removal of the cert-manager operator package on Red Hat Marketplace

Since cert-manager v0.15 there has been a package for cert-manager on Red Hat Marketplace, but this has now been removed because it was not maintained and was found to be unreliable: #4055 #3732 #436

It is replaced by a new package which is generated via the Community Operators Repository, and which is therefore available on OperatorHub.io, OpenShift Container Platform and OKD.

Please uninstall the existing cert-manager package and re-install by following the OLM Installation Documentation.

Upgrading cert-manager CRDs and stored versions of cert-manager custom resources

We have deprecated the following cert-manager APIs:

cert-manager.io/v1alpha2
cert-manager.io/v1alpha3
cert-manager.io/v1beta1
acme.cert-manager.io/v1alpha2
acme.cert-manager.io/v1alpha3
acme.cert-manager.io/v1beta1

These APIs will be removed in cert-manager 1.6.

⛔️ If you are upgrading cert-manager on a cluster which has previously had cert-manager < v1.0.0, you will need to ensure that all cert-manager custom resources are stored in etcd at v1 version and that cert-manager CRDs do not reference the deprecated APIs by the time you upgrade to v1.6.

This is explained in more detail in the Upgrading existing cert-manager resources page.

This change was made in the cert-manager PR #4021.

Helm chart: `securityContext` defaults to non-root

The Helm chart now follows the current Pod hardening best practices as defined by the Kyverno pod-security restricted policy.

To pass the validation, the controller, webhook, and cainjector Pods are now running as non-root:

apiVersion: v1
kind: Pod
spec:
  securityContext:
    runAsNonRoot: true

⛔️ If you are using custom containers that run as root with the Helm chart, you will need to set this back to false.

Implemented in the cert-manager PR #4036.

CA, Vault and Venafi issuer handling of `ca.crt` and `tls.crt`

The CA, Vault, and Venafi issuer now produce a tls.crt that is de-duplicated, in the correct order (leaf at the top, issuing certificate at the bottom) and verified (i.e. each signature can be verified).

The CA issuer now produces a ca.crt that contains the "most" root CA that cert-manager is aware of. ca.crt may thus not be the actual self-signed root CA, since cert-manager may not be aware of it.

Fixed in the cert-manager PRs #3982, #3983, and #3985.

⛔️ You may need to adjust systems that consume the ca.crt from Secrets managed by cert-manager with the CA issuer.

Vault renewal bug

The renewal behavior has changed when a Certificate has a duration value of more than 90 days and renewBefore has not been set.

Previously, the Certificate was renewed 30 days before expiry; now, the renewal happens 2/3 through the duration.

This change was necessary to fix a bug where users of the Vault issuer would see a clash between the default renewal duration of 30 days and the duration of certificates issued by the Vault PKI.

⛔️ If you were relying on the default renewal happening 30 days before expiry, we would advise setting renewBefore to 30 days (720h) to keep the old behavior.

Fixed in the cert-manager PR #4092.

New Features

Experimental Support for Kubernetes CertificateSigningRequests

It is now possible to use the built-in Kubernetes CertificateSigningRequest resources with cert-manager. The CA Issuer is currently the only supported issuer. The feature is experimental and can be enabled by adding a flag to the cert-manager controller. For example, with Helm:

helm install cert-manager jetstack/cert-manager \
  --set extraArgs="{--feature-gates=ExperimentalCertificateSigningRequestControllers=true}"

Note that you will still need to manually approve the CSR object before cert-manager can sign the CSR.

The documentation is available on the the Kubernetes CSR usage page.

Implemented in cert-manager PR #4064.

Helm chart: webhook externally accessible for bare-metal

In some Kubernetes setups, the apiserver is not able to talk to kube-dns (i.e., when Kubernetes is running on bare-metal with no special resolv.conf).

To work around that, the cert-manager webhook can now be configured to be accessible from outside of the cluster. For example, in values.yaml:

# values.yaml
webhook:
  serviceType: LoadBalancer
  loadBalancerIP: 198.51.100.20
  url:
    host: 198.51.100.20

Implemented in cert-manager PRs #3876, #3932.

Helm chart: Service labels

The cert-manager controller Service now supports custom labels using the top-level field in values.yaml:

# values.yaml
serviceLabels:
  app: armada-api

This may be useful in conjunction with Prometheus' labelmap. For example, with the following sample Prometheus configuration:

# prometheus.yaml
- action: labelmap
  regex: __meta_kubernetes_service_label_(.+)

With the above example, the source label __meta_kubernetes_service_label_app='armada-api' becomes the new label app='armada-api' when metrics related to this Service are scraped.

Implemented in the cert-manager PR #4009.

Akamai DNS01 solver

The Akamai DNS01 solver has been [updated][4007] to use the v2 of the OPEN EdgeGrid Go package.

Bug Fixes

The RFC2136 issuer is now able to handle DNS01 challenges that map to multiple TXT records. This lets you create Let's Encrypt certificates using RFC2136 with multiple DNS names. Fixed in the cert-manager PR #3622.
The comparison function PublicKeysEqual is now correct for public keys. Fixed in PR #3914.
The ACME issuer now works correctly with Certificates that have a long name (52 characters or more). These Certificates would not get renewed due to non-unique Order names being generated. Fixed in the cert-manager PR #3866.
Orders that are used with a misbehaving ACME server should not get stuck anymore. By misbehaving, we mean an ACME server that would validate the authorizations before having set the status of the order to "ready". Fixed in the cert-manager PR #3805.
The internal issuers now set the condition Ready=False with the reason RequestDenied when a CertificateRequest has been Denied. This is to keep the same behavior where a terminal state of a CertificateRequest should have a Ready condition. Fixed in the cert-manager PR #3878.

Other Changes

The cert-manager controller now uses the configmapsleases resource instead of the configmaps one for leader election. The only noticeable difference is that a new Lease object is now being created in the leader election namespace. Implemented in the cert-manager PR #4016.

The keyAlgorithm for the ACME Issuer is now deprecated, and the EAB MAC algorithm is now hard-coded to HS256.
```
apiVersion: cert-manager.io/v1
kind: Issuer
  spec:
    acme:
      externalAccountBinding:
        keyAlgorithm: HS256      # DEPRECATED.
```
Previously, we used to have a fork of golang/crypto which allowed us to set the EAB MAC algorithm. We now use the upstream version of golang/crypto where the EAB MAC algorithm is hard-coded to HS256.

This change were implemented in the cert-manager PRs #3877 and #3936.

If you happen to look at the cert-manager controller logs, you may see this new message about optimistic locking:

I0419 controller.go:158] msg="re-queuing item due to optimistic locking on resource" error="Operation cannot be fulfilled on certificates.cert-manager.io   sauron-adverts-evo-app-tls: the object has been modified; please apply your changes to the latest version and try again"

This message, shown at the info level, replaces the error level message that showed previously:

E0419 controller.go:158] msg="re-queuing item due to error processing" error="Operation cannot be fulfilled on certificates.cert-manager.io sauron-adverts-evo-app-tls: the   object has been modified; please apply your changes to the latest version and try again"

The goal is to prevent users from thinking that the optimistic locking mechanism has to do with their issues, when in reality it mostly isn't and is the normal operation mode for Kubernetes controllers.

Fixed in the cert-manager PR #3794.

The util.UsageContentCommittment (which contained a spelling mistake) was deprecated in favor of util.UsageContentCommitment. The only people impacted by this deprecation are the the people importing the Go package github.com/jetstack/cert-manager/pkg/api/util.

The webhook now panics when it is not able to register the API schemes. Previously, the webhook would silently skip the error and start.

A couple of legacy functions in test/e2e/util package have been removed. These functions can be found in the test/unit/gen package.

The Kubernetes Go dependencies have been updated from v0.19.0 to v0.21.0.

When waiting for DNS propagating, the ACME DNS01 self-check now returns a better message when an unexpected DNS response code is received, such as SERVFAIL.

Before:

Could not find the start of authority

After:

Could not find the SOA record in the DNS tree
for the domain '_acme-challenge.foo.example.com'
using nameservers [8.8.8.8, 8.8.4.4]

In addition to the above, you will get a new message when the DNS returns an unexpected response code:

When querying the SOA record for the domain
'_acme-challenge.foo.example.com' using nameservers
[8.8.8.8, 8.8.4.4], rcode was expected to be 'NOERROR'
or 'NXDOMAIN', but got 'SERVFAIL'

Fixed in the cert-manager PR #3906.

The distroless/static base image was updated to the latest version as of 2021-05-20.

Honorable mentions

Tim Ramlot (@inteon) has done a fantastic job at adding the Istio VirtualService support for HTTP01 challenges in #3724. It took an immense effort to have this PR ready and merged for the 1.4 release.

After a lot of thinking, we have decided that trying to support every custom resource for every proxy could not be done in-tree due to the Go dependency weight that each integration adds. Jake Sanders proposed an out-of-tree approach that will be worked on as part of cert-manager 1.5.