# Kubernetes Networking

## Overview

By default, the Kubernetes Helm chart does not expose any of the Hatchet services over an ingress. There are three services which can possibly be exposed:

1. `hatchet-engine`
2. `hatchet-stack-api`
3. `hatchet-stack-frontend`

To expose these services, you will need to do the following:

1. Configure ingresses for `frontend` and `engine` services (and optionally the `api` service). We recommend configuring the ingress to reverse proxy `/api` endpoints to the `hatchet-stack-api` service, and configuring a separate ingress to proxy to `hatchet-engine`.

2. Update the following configuration variables:

```yaml
api:
  env:
    SERVER_AUTH_COOKIE_DOMAIN: "hatchet.example.com" # example.com should be replaced with your domain
    SERVER_URL: "https://hatchet.example.com" # example.com should be replaced with your domain
    SERVER_GRPC_BIND_ADDRESS: "0.0.0.0"
    SERVER_GRPC_INSECURE: "false"
    SERVER_GRPC_BROADCAST_ADDRESS: "hatchet-engine.example.com:443" # example.com should be replaced with your domain

engine:
  env:
    SERVER_AUTH_COOKIE_DOMAIN: "hatchet.example.com" # example.com should be replaced with your domain
    SERVER_URL: "https://hatchet.example.com" # example.com should be replaced with your domain
    SERVER_GRPC_BIND_ADDRESS: "0.0.0.0"
    SERVER_GRPC_INSECURE: "false"
    SERVER_GRPC_BROADCAST_ADDRESS: "engine.hatchet.example.com:443" # example.com should be replaced with your domain
```

## Example: `nginx-ingress`

Let's walk through an example of exposing Hatchet over `hatchet.example.com` (for the API and frontend) and `engine.hatchet.example.com` (for the engine).

We'll be deploying this with SSL enabled, which requires a valid certificate. We recommend using [cert-manager](https://cert-manager.io/docs/) to manage your certificates. This guide assumes that you have a cert-manager `ClusterIssuer` called `letsencrypt-prod` configured.

Here's an example `values.yaml` file for this setup:

```yaml
api:
  env:
    # TODO: insert these values from the output of the keyset generation command
    SERVER_AUTH_COOKIE_SECRETS: "$SERVER_AUTH_COOKIE_SECRET1 $SERVER_AUTH_COOKIE_SECRET2"
    SERVER_ENCRYPTION_MASTER_KEYSET: "$SERVER_ENCRYPTION_MASTER_KEYSET"
    SERVER_ENCRYPTION_JWT_PRIVATE_KEYSET: "$SERVER_ENCRYPTION_JWT_PRIVATE_KEYSET"
    SERVER_ENCRYPTION_JWT_PUBLIC_KEYSET: "$SERVER_ENCRYPTION_JWT_PUBLIC_KEYSET"
    SERVER_AUTH_COOKIE_DOMAIN: "hatchet.example.com" # example.com should be replaced with your domain
    SERVER_URL: "https://hatchet.example.com" # example.com should be replaced with your domain
    SERVER_GRPC_BIND_ADDRESS: "0.0.0.0"
    SERVER_GRPC_INSECURE: "false"
    SERVER_GRPC_BROADCAST_ADDRESS: "engine.hatchet.example.com:443" # example.com should be replaced with your domain

engine:
  env:
    # TODO: insert these values from the output of the keyset generation command
    SERVER_AUTH_COOKIE_SECRETS: "$SERVER_AUTH_COOKIE_SECRET1 $SERVER_AUTH_COOKIE_SECRET2"
    SERVER_ENCRYPTION_MASTER_KEYSET: "$SERVER_ENCRYPTION_MASTER_KEYSET"
    SERVER_ENCRYPTION_JWT_PRIVATE_KEYSET: "$SERVER_ENCRYPTION_JWT_PRIVATE_KEYSET"
    SERVER_ENCRYPTION_JWT_PUBLIC_KEYSET: "$SERVER_ENCRYPTION_JWT_PUBLIC_KEYSET"
    SERVER_AUTH_COOKIE_DOMAIN: "hatchet.example.com" # example.com should be replaced with your domain
    SERVER_URL: "https://hatchet.example.com" # example.com should be replaced with your domain
    SERVER_GRPC_BIND_ADDRESS: "0.0.0.0"
    SERVER_GRPC_INSECURE: "false"
    SERVER_GRPC_BROADCAST_ADDRESS: "engine.hatchet.example.com:443" # example.com should be replaced with your domain
  ingress:
    enabled: true
    ingressClassName: nginx
    labels: {}
    annotations:
      cert-manager.io/cluster-issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/auth-tls-verify-client: "optional"
      nginx.ingress.kubernetes.io/auth-tls-secret: "${kubernetes_namespace.cloud.metadata[0].name}/engine-cert"
      nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
      nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
      nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      nginx.ingress.kubernetes.io/grpc-backend: "true"
      nginx.ingress.kubernetes.io/server-snippet: |
        grpc_read_timeout 1d;
        grpc_send_timeout 1h;
        client_header_timeout 1h;
        client_body_timeout 1h;
    hosts:
      - host: engine.hatchet.example.com
        paths:
          - path: /
        backend:
          serviceName: hatchet-engine
          servicePort: 7070
    tls:
      - hosts:
          - engine.hatchet.example.com
        secretName: engine-cert
        servicePort: 7070

frontend:
  ingress:
    enabled: true
    ingressClassName: nginx
    labels: {}
    annotations:
      nginx.ingress.kubernetes.io/proxy-body-size: 50m
      nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
      nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
      nginx.ingress.kubernetes.io/proxy-connect-timeout: "60"
      cert-manager.io/cluster-issuer: letsencrypt-prod
    hosts:
      - host: hatchet.example.com
        paths:
          - path: /api
            backend:
              serviceName: hatchet-api
              servicePort: 8080
          - path: /
            backend:
              serviceName: hatchet-frontend
              servicePort: 8080
    tls:
      - secretName: hatchet-api
        hosts:
          - hatchet.example.com
```