Using SignalR and other WebSockets in Kubernetes behind an NGINX Ingress Controller

When using Ingress in Kubernetes, the NGINX Ingress Controller presents a default options for many. When you application is using WebSocket and frameworks like SignalR, the NGINX should be adjusted for that use-case.

Scaling SignalR out to multiple instances

When running multiple instances of a SignalR server, you should make sure, they can all talk to and transfer state between each other. For that, you can back SignalR with a Redis Cache backplane.

Once that is done, you can scale out. Still, you want to ensure that an application holds a connection to the same instance, once established. For that, add the Session Affinity annotation to your Kubernetes Ingress.

nginx.ingress.kubernetes.io/affinity: cookie

You should also think about setting the Affinity Mode. By default, NGINX will re-distribute the load, if a deployment gets scaled up. When choosing persistent, NGINX will not rebalance sessions to new servers.

Troubleshooting

Some users run into these errors, when running a SignalR or similar WebSocket based application behind the NGINX Ingress Controller.

WebSocketTransport.js:86 WebSocket connection to 'wss://yourdomain.com/echo?id=_Zbe1Q5-pyRTFQiE3jWYHg' failed: Error during WebSocket handshake: Unexpected response code: 200

[2020-02-15T19:10:47.497Z] Error: Failed to start the transport 'WebSockets': Error: There was an error with the transport.

[2020-02-15T19:11:02.591Z] Error: Connection disconnected with error 'Error: Server returned handshake error: Handshake was canceled.'.

That usually implies, that you are using the nginx/inginx-ingress Helm Chart for deploying NGINX Ingress into your cluster. Turns out, that this variant of NGINX causes trouble to some customers. The official Helm Chart, that should be used is stable/nginx-ingress.

If you still want to use NGINX version, that the nginx/inginx-ingress Helm Chart deploys, you need to enable WebSocket support for your Service. To load balance Web Sockets, we have to add the following annotation to the Ingress resource:

nginx.org/websocket-services: "service1[,service2,...]"

The following example shows two load balances applications, one of which is using WebSockets:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    nginx.org/websocket-services: "web-chat"
spec:
  rules:
  - host: app.yourdomain.com
    http:
      paths:      
        - backend:
            serviceName: web-app
            servicePort: 80
  - host: chat.yourdomain.com
    http:
      paths:
        - backend:
            serviceName: web-chat
            servicePort: 80

☝️ Advertisement Block: I will buy myself a pizza every time I make enough money with these ads to do so. So please feed a hungry developer and consider disabling your Ad Blocker.