Add new TCP / UDP services

Modified on Thu, 18 May, 2023 at 9:12 PM

When installing new listeners, e.g. the Direktiv SMTP listener, access from external environments into the Direktiv cluster is needed. As an example, the SMTP Listener listens on TCP port 2525 (by default):

To support the external port listener to the internal container TCP port, the APISIX gateway and load balancer is used (see https://apisix.apache.org/ for more details). To enabled this, a 2-step process needs to be completed.

Step 1: open port on ingress controller

The first step is to open the port on the ingress controller itself. To do this, add the gateway configuration to the apisix.yaml file (normally located in the ~/direktiv-ee/install/03_apisix/ directory for the enterprise edition):

gateway:
  type: LoadBalancer
  tls:
    enabled: true
    sslProtocols: "TLSv1 TLSv1.1 TLSv1.2"
  stream:
    enabled: true
    tcp:
    - 2525

etcd:
  replicaCount: 1
  podAffinityPreset: soft
  auth:
    rbac:
      create: true
      user: root
    token:
      privateKey:
        existingSecret: apisix-dep-jwt-token

ingress-controller:
  enabled: true
  image:
    repository: direktiv/apisix-ingress-controller
    tag: "1.6.0"
  config:
     apisix:
        serviceNamespace: apisix

configurationSnippet:
  httpStart:
    proxy_busy_buffers_size   512k;
    proxy_buffers   4 512k;
    proxy_buffer_size   256k;
  httpSrv: |
    error_page 500 502 503 504 /errorpage.html;
    location = /errorpage.html {
            root /usr/share/nginx/html;
            internal;
    }

extraVolumes:
  - name: error-volume
    configMap:
      name: apisix-dep-error
extraVolumeMounts:
  - name: error-volume
    mountPath: /usr/share/nginx/html

Once this file has been updated replace the APISIX services. A helm upgrade command does not replace any services, so the easiest approach is to uninstall and install APISIX with the following commands in the 03_apisix installation directory:

# helm uninstall -n apisix apisix
# helm uninstall -n apisix apisix-dep
# kubectl delete persistentvolumeclaims -n apisix data-apisix-etcd-0
# export DIREKTIV_HOST=<hostname to use>
# ./install.sh

Step 2: add the route for the internal port

After re-installing APISIX a route for the port can be added with a ApisixRoute YAML:

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: smtp-tcp-route
spec:
  stream:
    - name: smtp-route
      protocol: TCP
      match:
        ingressPort: 2525
      backend:
        serviceName: smtp-listener-service
        servicePort: 2525

For this, the following steps are needed:

Create a file called smtp-apisix-route.yaml with the configuration above (change the ingressPort, serviceName and servicePort as required.
Run the following command: kubectl apply -f smtp-apisix-route.yaml

# kubectl apply -f smtp-apisix-route.yaml
apisixroute.apisix.apache.org/smtp-tcp-route created

Alternative

The second alternative is to update the APISIX installation. For this the etcd password is required because it is stored in the PersistentVolumeClaim. The password can be fetched with the following command:

kubectl get secrets -n apisix apisix-etcd -o 'go-template={{index .data "etcd-root-password"}}' | base64 --decode

The output of that command has to be added to the etc section of the apisix.yaml file in 03_apisix/apisix.yaml:

etcd:
  replicaCount: 1
  podAffinityPreset: soft
  password: MyEtcdPassword
  auth:
    rbac:
      create: true
      user: root
    token:
      privateKey:
        existingSecret: apisix-dep-jwt-token