NEW: Get project updates onTwitterandMastodon

Pomerium Ingress

This tutorial covers installing the Pomerium Ingress Controller and securing it with cert-manager. Pomerium is an identity-aware proxy that can also provide a custom ingress controller for your Kubernetes services.

Prerequisites

  1. Install Kubectl and set the context to the cluster you'll be working with.

  2. Install Helm on your local computer. See Installing Helm for the best installation method for your operating system.

  3. Pomerium connects to an identity provider (IdP) to authenticate users. See one of their guides to learn how to set up your IdP of choice to provide oauth2 validation.

  4. This tutorial assumes you have a domain space reserved for this cluster (such as *.cluster.example.com). You will need access to DNS for this domain to assign A and CNAME records as needed.

Install cert-manager

  1. Create a namespace for cert-manager:

    kubectl create namespace cert-manager
  2. Add the jetstack repository and update Helm:

    helm repo add jetstack https://charts.jetstack.io
    helm repo update
  3. Install cert-manager to your cluster:

    helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace \
    --set installCRDs=true
  4. Confirm deployment with kubectl get pods --namespace cert-manager:

    kubectl get pods --namespace cert-manager
    NAME READY STATUS RESTARTS AGE
    cert-manager-5d7f97b46d-8g942 1/1 Running 0 33s
    cert-manager-cainjector-69d885bf55-6x5v2 1/1 Running 0 33s
    cert-manager-webhook-8d7495f4-s5s6p 1/1 Running 0 33s

Configure a Private Certificate Issuer

For secure communication between Pomerium services, create a private certificate issuer. This issuer will reside in the pomerium namespace, which we will use when creating the Ingress Controller. The certificates issued will only be used for communication between Pomerium components.

  1. Define an issuer in the file pomerium-issuer.yaml:

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
    name: pomerium-ca
    namespace: pomerium
    spec:
    selfSigned: {}
    ---
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
    name: pomerium-ca
    namespace: pomerium
    spec:
    isCA: true
    secretName: pomerium-ca
    commonName: pomerium ca
    issuerRef:
    name: pomerium-ca
    kind: Issuer
    ---
    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
    name: pomerium-issuer
    namespace: pomerium
    spec:
    ca:
    secretName: pomerium-ca
  2. Deploy the issuer:

    kubectl apply -f pomerium-issuer.yaml

Configure Let's Encrypt Issuer

For communication between the Ingresses and the internet, we'll want to use certificates signed by a trusted certificate authority like Let's Encrypt. This example creates two Let's Encrypt issuers, one for staging and one for production.

The Let's Encrypt production issuer has strict rate limits. Before your configuration is finalized you may have to recreate services several times, hitting those limits. It's easy to confuse rate limiting with errors in configuration or operation while building your stack.

Because of this, we will start with the Let's Encrypt staging issuer. Once your configuration is all but finalized, we will switch to a production issuer. Both of these issuers are configured to use the HTTP01 challenge provider.

  1. The following YAML defines a staging certificate issuer. You must update the email address to your own. The email field is required by Let's Encrypt and used to notify you of certificate expiration and updates.

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
    name: letsencrypt-staging
    spec:
    acme:
    # The ACME server URL
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: user@example.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
    name: letsencrypt-staging
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
    ingress:
    class: pomerium

    You can download and edit the example and apply it with kubectl apply -f, or edit, and apply the custom resource in one command:

    kubectl create --edit -f https://cert-manager.io/docs/tutorials/acme/example/pomerium-staging-issuer.yaml
  2. Create a production issuer and deploy it. As with the staging issuer, update this example with your own email address:

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
    name: letsencrypt-prod
    spec:
    acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: user@example.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
    name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
    ingress:
    class: pomerium
    kubectl create --edit -f https://cert-manager.io/docs/tutorials/acme/example/pomerium-production-issuer.yaml

You can confirm on the status of the issuers after you create them:

kubectl describe issuer letsencrypt-staging
kubectl describe issuer letsencrypt-prod

You should see the issuer listed with a registered account.

Install The Pomerium Ingress Controller

  1. Set your kubectl context to the Pomerium namespace:

    kubectl config set-context --current --namespace=pomerium
  2. Create certificate configurations for Pomerium. Our example is named pomerium-certificates.yaml:

    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
    name: pomerium-cert
    namespace: pomerium
    spec:
    secretName: pomerium-tls
    issuerRef:
    name: pomerium-issuer
    kind: Issuer
    usages:
    - server auth
    - client auth
    dnsNames:
    - pomerium-proxy.pomerium.svc.cluster.local
    - pomerium-authorize.pomerium.svc.cluster.local
    - pomerium-databroker.pomerium.svc.cluster.local
    - pomerium-authenticate.pomerium.svc.cluster.local
    ---
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
    name: pomerium-redis-cert
    namespace: pomerium
    spec:
    secretName: pomerium-redis-tls
    issuerRef:
    name: pomerium-issuer
    kind: Issuer
    usages:
    - server auth
    - client auth
    dnsNames:
    - pomerium-redis-master.pomerium.svc.cluster.local
    - pomerium-redis-headless.pomerium.svc.cluster.local
    - pomerium-redis-replicas.pomerium.svc.cluster.local

    Replace localhost.pomerium.io with the domain name you'll assign to the Ingress. Keep the subdomain authenticate.

    This example defines 2 certificates:

    • One using the self-signed pomerium-issuer to encrypt traffic between Pomerium's services,
    • One using the self-signed pomerium-issuer to encrypt traffic to and from the Redis service(s) used by Pomerium.

    Additional certificates will be issued as new Ingresses are created.

  3. Apply the certificate configuration, and confirm:

    kubectl apply -f pomerium-certificates.yaml
    kubectl get certificate
    NAME READY SECRET AGE
    pomerium-ca True pomerium-ca 10s
    pomerium-cert True pomerium-tls 10s
    pomerium-redis-cert True pomerium-redis-tls 10s
  4. Create a values file for Helm to use when installing Pomerium. Our example is named pomerium-values.yaml.

    authenticate:
    existingTLSSecret: pomerium-tls
    idp:
    provider: "google"
    clientID: YOUR_CLIENT_ID
    clientSecret: YOUR_SECRET
    serviceAccount: YOUR_SERVICE_ACCOUNT
    ingress:
    annotations:
    cert-manager.io/issuer: letsencrypt-staging
    tls:
    secretName: authenticate.localhost.pomerium.io-tls
    proxy:
    existingTLSSecret: pomerium-tls
    databroker:
    existingTLSSecret: pomerium-tls
    storage:
    clientTLS:
    existingSecretName: pomerium-redis-tls
    existingCASecretKey: ca.crt
    authorize:
    existingTLSSecret: pomerium-tls
    redis:
    enabled: true
    generateTLS: false
    tls:
    certificateSecret: pomerium-redis-tls
    ingressController:
    enabled: true
    config:
    rootDomain: localhost.pomerium.io #Change this to your reserved domain space.
    existingCASecret: pomerium-tls
    generateTLS: false # On by default, disabled when cert-manager or another solution is in place.

    The options required in the authenticate.idp block will vary depending on your identity provider.

    Update config.rootDomain to match your domain space.

  5. Add Pomerium's Helm repo:

    helm repo add pomerium https://helm.pomerium.io
  6. Install Pomerium to the cluster:

    helm upgrade --install pomerium pomerium/pomerium --values ./pomerium-values.yaml
  7. Use kubectl to confirm that the Pomerium Proxy has stood up, and get the external IP needed to route your domain space to the cluster:

    kubectl get svc pomerium-proxy
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    pomerium-proxy LoadBalancer 10.128.117.25 192.0.2.20 443:30006/TCP,9090:30707/TCP 2m37s

Define a Test Service

To test our new Ingress Controller, we will add the kuard app to our cluster and define an Ingress for it.

  1. Define the kuard deployment and associated service:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: kuard
    spec:
    selector:
    matchLabels:
    app: kuard
    replicas: 1
    template:
    metadata:
    labels:
    app: kuard
    spec:
    containers:
    - image: gcr.io/kuar-demo/kuard-amd64:1
    imagePullPolicy: Always
    name: kuard
    ports:
    - containerPort: 8080
    apiVersion: v1
    kind: Service
    metadata:
    name: kuard
    spec:
    ports:
    - port: 80
    targetPort: 8080
    protocol: TCP
    selector:
    app: kuard

    You can download and reference these files locally, or you can reference them from the GitHub source repository for this documentation.

    To install the example service from the tutorial files straight from GitHub:

    kubectl apply -f https://cert-manager.io/docs/tutorials/acme/example/deployment.yaml
    kubectl apply -f https://netlify.cert-manager.io/docs/tutorials/acme/example/service.yaml
  2. Create a new Ingress manifest (example-ingress.yaml) for our test service:

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
    name: kuard
    annotations:
    cert-manager.io/issuer: letsencrypt-staging
    ingress.pomerium.io/policy: '[{"allow":{"and":[{"domain":{"is":"example.com"}}]}}]'
    spec:
    ingressClassName: pomerium
    rules:
    - host: kuard.localhost.pomerium.io
    http:
    paths:
    - path: /
    pathType: Prefix
    backend:
    service:
    name: kuard
    port:
    number: 80
    tls:
    - hosts:
    - kuard.localhost.pomerium.io
    secretName: kuard.localhost.pomerium.io-tls

    Again, change the references to localhost.pomerium.io to match your domain space.

  3. Apply the Ingress manifest to the cluster:

    kubectl apply -f example-ingress.yaml

The Pomerium Ingress Controller will use cert-manager to automatically provision a certificate from the letsencrypt-staging issuer for the route to kuard.localhost.pomerium.io.

Once you've configured all your application services correctly in the cluster, adjust the issuer for your Ingresses (including the Authenticate service) to use letsencrypt-prod.