Pomerium Ingress

This tutorial covers installing the Pomerium Ingress Controller and securing it with cert-manager. Pomerium is an identity-aware proxy that can also provide a custom ingress controller for your Kubernetes services.

Prerequisites

  1. Install Kubectl and set the context to the cluster you’ll be working with.

  2. Install Helm on your local computer. See Installing Helm for the best installation method for your operating system.

  3. Pomerium connects to an identity provider (IdP) to authenticate users. See one of their guides to learn how to set up your IdP of choice to provide oauth2 validation.

  4. This tutorial assumes you have a domain space reserved for this cluster (such as *.cluster.example.com). You will need access to DNS for this domain to assign A and CNAME records as needed.

Install cert-manager

  1. Create a namespace for cert-manager:

    kubectl create namespace cert-manager
    
  2. Add the jetstack repository and update Helm:

    helm repo add jetstack https://charts.jetstack.io
    helm repo update
    
  3. Install cert-manager to your cluster:

    helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace \
    --set installCRDs=true
    
  4. Confirm deployment with kubectl get pods --namespace cert-manager:

    kubectl get pods --namespace cert-manager
    NAME                                       READY   STATUS    RESTARTS   AGE
    cert-manager-5d7f97b46d-8g942              1/1     Running   0          33s
    cert-manager-cainjector-69d885bf55-6x5v2   1/1     Running   0          33s
    cert-manager-webhook-8d7495f4-s5s6p        1/1     Running   0          33s
    

Configure a Private Certificate Issuer

For secure communication between Pomerium services, create a private certificate issuer. This issuer will reside in the pomerium namespace, which we will use when creating the Ingress Controller. The certificates issued will only be used for communication between Pomerium components.

  1. Define an issuer in the file pomerium-issuer.yaml:

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      name: pomerium-ca
      namespace: pomerium
    spec:
      selfSigned: {}
    ---
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      name: pomerium-ca
      namespace: pomerium
    spec:
      isCA: true
      secretName: pomerium-ca
      commonName: pomerium ca
      issuerRef:
        name: pomerium-ca
        kind: Issuer
    ---
    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      name: pomerium-issuer
      namespace: pomerium
    spec:
      ca:
        secretName: pomerium-ca
    
  2. Deploy the issuer:

    kubectl apply -f pomerium-issuer.yaml
    

Configure Let’s Encrypt Issuer

For communication between the Ingresses and the internet, we’ll want to use certificates signed by a trusted certificate authority like Let’s Encrypt. This example creates two Let’s Encrypt issuers, one for staging and one for production.

The Let’s Encrypt production issuer has strict rate limits. Before your configuration is finalized you may have to recreate services several times, hitting those limits. It’s easy to confuse rate limiting with errors in configuration or operation while building your stack.

Because of this, we will start with the Let’s Encrypt staging issuer. Once your configuration is all but finalized, we will switch to a production issuer. Both of these issuers are configured to use the HTTP01 challenge provider.

  1. The following YAML defines a staging certificate issuer. You must update the email address to your own. The email field is required by Let’s Encrypt and used to notify you of certificate expiration and updates.

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      name: letsencrypt-staging
    spec:
      acme:
        # The ACME server URL
        server: https://acme-staging-v02.api.letsencrypt.org/directory
        # Email address used for ACME registration
        email: user@example.com
        # Name of a secret used to store the ACME account private key
        privateKeySecretRef:
          name: letsencrypt-staging
        # Enable the HTTP-01 challenge provider
        solvers:
          - http01:
              ingress:
                class:  pomerium
    

    You can download and edit the example and apply it with kubectl apply -f, or edit, and apply the custom resource in one command:

    kubectl create --edit -f https://cert-manager.io/docs/tutorials/acme/example/pomerium-staging-issuer.yaml
    
  2. Create a production issuer and deploy it. As with the staging issuer, update this example with your own email address:

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      name: letsencrypt-prod
    spec:
      acme:
        # The ACME server URL
        server: https://acme-v02.api.letsencrypt.org/directory
        # Email address used for ACME registration
        email: user@example.com
        # Name of a secret used to store the ACME account private key
        privateKeySecretRef:
          name: letsencrypt-prod
        # Enable the HTTP-01 challenge provider
        solvers:
        - http01:
            ingress:
              class: pomerium
    
    kubectl create --edit -f https://cert-manager.io/docs/tutorials/acme/example/pomerium-production-issuer.yaml
    

You can confirm on the status of the issuers after you create them:

kubectl describe issuer letsencrypt-staging
kubectl describe issuer letsencrypt-prod

You should see the issuer listed with a registered account.

Install The Pomerium Ingress Controller

  1. Set your kubectl context to the Pomerium namespace:

    kubectl config set-context --current --namespace=pomerium
    
  2. Create certificate configurations for Pomerium. Our example is named pomerium-certificates.yaml:

    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      name: pomerium-cert
      namespace: pomerium
    spec:
      secretName: pomerium-tls
      issuerRef:
        name: pomerium-issuer
        kind: Issuer
      usages:
        - server auth
        - client auth
      dnsNames:
        - pomerium-proxy.pomerium.svc.cluster.local
        - pomerium-authorize.pomerium.svc.cluster.local
        - pomerium-databroker.pomerium.svc.cluster.local
        - pomerium-authenticate.pomerium.svc.cluster.local
    ---
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      name: pomerium-redis-cert
      namespace: pomerium
    spec:
      secretName: pomerium-redis-tls
      issuerRef:
        name: pomerium-issuer
        kind: Issuer
      usages:
        - server auth
        - client auth
      dnsNames:
        - pomerium-redis-master.pomerium.svc.cluster.local
        - pomerium-redis-headless.pomerium.svc.cluster.local
        - pomerium-redis-replicas.pomerium.svc.cluster.local
    
    

    Replace localhost.pomerium.io with the domain name you’ll assign to the Ingress. Keep the subdomain authenticate.

    This example defines 2 certificates:

    • One using the self-signed pomerium-issuer to encrypt traffic between Pomerium’s services,
    • One using the self-signed pomerium-issuer to encrypt traffic to and from the Redis service(s) used by Pomerium.

    Additional certificates will be issued as new Ingresses are created.

  3. Apply the certificate configuration, and confirm:

    kubectl apply -f pomerium-certificates.yaml
    
    kubectl get certificate
    NAME                    READY   SECRET                 AGE
    pomerium-ca           True    pomerium-ca              10s
    pomerium-cert           True    pomerium-tls           10s
    pomerium-redis-cert     True    pomerium-redis-tls     10s
    
  4. Create a values file for Helm to use when installing Pomerium. Our example is named pomerium-values.yaml.

    authenticate:
      existingTLSSecret: pomerium-tls
      idp:
        provider: "google"
        clientID: YOUR_CLIENT_ID
        clientSecret: YOUR_SECRET
        serviceAccount: YOUR_SERVICE_ACCOUNT
      ingress:
        annotations:
          cert-manager.io/issuer: letsencrypt-staging
        tls:
          secretName: authenticate.localhost.pomerium.io-tls 
    
    proxy:
      existingTLSSecret: pomerium-tls
    
    databroker:
      existingTLSSecret: pomerium-tls
      storage:
        clientTLS:
          existingSecretName: pomerium-redis-tls
          existingCASecretKey: ca.crt
    
    authorize:
      existingTLSSecret: pomerium-tls
    
    redis:
      enabled: true
      generateTLS: false
      tls:
        certificateSecret: pomerium-redis-tls
    
    ingressController:
      enabled: true
    
    config:
      rootDomain: localhost.pomerium.io #Change this to your reserved domain space.
      existingCASecret: pomerium-tls
      generateTLS: false # On by default, disabled when cert-manager or another solution is in place.
    
    

    The options required in the authenticate.idp block will vary depending on your identity provider.

    Update config.rootDomain to match your domain space.

  5. Add Pomerium’s Helm repo:

    helm repo add pomerium https://helm.pomerium.io
    
  6. Install Pomerium to the cluster:

    helm upgrade --install pomerium pomerium/pomerium --values ./pomerium-values.yaml
    
  7. Use kubectl to confirm that the Pomerium Proxy has stood up, and get the external IP needed to route your domain space to the cluster:

    kubectl get svc pomerium-proxy
    NAME             TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                        AGE
    pomerium-proxy   LoadBalancer   10.128.117.25    192.0.2.20       443:30006/TCP,9090:30707/TCP   2m37s
    

Define a Test Service

To test our new Ingress Controller, we will add the kuard app to our cluster and define an Ingress for it.

  1. Define the kuard deployment and associated service:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: kuard
    spec:
      selector:
        matchLabels:
          app: kuard
      replicas: 1
      template:
        metadata:
          labels:
            app: kuard
        spec:
          containers:
          - image: gcr.io/kuar-demo/kuard-amd64:1
            imagePullPolicy: Always
            name: kuard
            ports:
            - containerPort: 8080
    
    
    apiVersion: v1
    kind: Service
    metadata:
      name: kuard
    spec:
      ports:
      - port: 80
        targetPort: 8080
        protocol: TCP
      selector:
        app: kuard
    
    

    You can download and reference these files locally, or you can reference them from the GitHub source repository for this documentation.

    To install the example service from the tutorial files straight from GitHub:

    kubectl apply -f https://cert-manager.io/docs/tutorials/acme/example/deployment.yaml
    kubectl apply -f https://netlify.cert-manager.io/docs/tutorials/acme/example/service.yaml
    
  2. Create a new Ingress manifest (example-ingress.yaml) for our test service:

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: kuard
      annotations:
        cert-manager.io/issuer: letsencrypt-staging
        ingress.pomerium.io/policy: '[{"allow":{"and":[{"domain":{"is":"example.com"}}]}}]'
    spec:
      ingressClassName: pomerium
      rules:
      - host: kuard.localhost.pomerium.io
        http:
          paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kuard
                port:
                  number: 80
      tls:
        - hosts:
            - kuard.localhost.pomerium.io
          secretName: kuard.localhost.pomerium.io-tls
    

    Again, change the references to localhost.pomerium.io to match your domain space.

  3. Apply the Ingress manifest to the cluster:

    kubectl apply -f example-ingress.yaml
    

The Pomerium Ingress Controller will use cert-manager to automatically provision a certificate from the letsencrypt-staging issuer for the route to kuard.localhost.pomerium.io.

Once you’ve configured all your application services correctly in the cluster, adjust the issuer for your Ingresses (including the Authenticate service) to use letsencrypt-prod.