This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Traffic Management

1 - Outbound Traffic IP Range Exclusions

Excluding IP address ranges of outbound traffic from sidecar interception

This guide demonstrates how outbound IP address ranges can be excluded from being intercepted by FSM’s proxy sidecar, so as to not subject them to service mesh filtering and routing policies.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have FSM installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.

Demo

The following demo shows an HTTP curl client making HTTP requests to the httpbin.org website directly using its IP address. We will explicitly disable the egress functionality to ensure traffic to a non-mesh destination (httpbin.org in this demo) is not able to egress the pod.

  1. Disable mesh-wide egress passthrough.

    export fsm_namespace=fsm-system # Replace fsm-system with the namespace where FSM is installed
kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"traffic":{"enableEgress":false}}}'  --type=merge

  2. Deploy the curl client into the curl namespace after enrolling its namespace to the mesh.

    # Create the curl namespace
kubectl create namespace curl

# Add the namespace to the mesh
fsm namespace add curl

# Deploy curl client in the curl namespace
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/curl/curl.yaml -n curl

    Confirm the curl client pod is up and running.

    $ kubectl get pods -n curl
NAME                    READY   STATUS    RESTARTS   AGE
curl-54ccc6954c-9rlvp   2/2     Running   0          20s

  3. Retrieve the public IP address for the httpbin.org website. For the purpose of this demo, we will test with a single IP range to be excluded from traffic interception. In this example, we will use the IP address 54.91.118.50 represented by the IP range 54.91.118.50/32, to make HTTP requests with and without outbound IP range exclusions configured.

    $ nslookup httpbin.org
Server:		172.23.48.1
Address:	172.23.48.1#53

Non-authoritative answer:
Name:	httpbin.org
Address: 54.91.118.50
Name:	httpbin.org
Address: 54.166.163.67
Name:	httpbin.org
Address: 34.231.30.52
Name:	httpbin.org
Address: 34.199.75.4

    Note: Replace 54.91.118.50 with a valid IP address returned by the above command in subsequent steps.

  4. Confirm the curl client is unable to make successful HTTP requests to the httpbin.org website running on http://54.91.118.50:80.

    $ kubectl exec -n curl -ti "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')" -c curl -- curl -I http://54.91.118.50:80
curl: (7) Failed to connect to 54.91.118.50 port 80: Connection refused
command terminated with exit code 7

    The failure above is expected because by default outbound traffic is redirected via the Pipy proxy sidecar running on the curl client’s pod, and the proxy subjects this traffic to service mesh policies which does not allow this traffic.

  5. Program FSM to exclude the IP range 54.91.118.50/32 IP range

    kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"traffic":{"outboundIPRangeExclusionList":["54.91.118.50/32"]}}}'  --type=merge

  6. Confirm the MeshConfig has been updated as expected

    # 54.91.118.50 is one of the IP addresses of httpbin.org
$ kubectl get meshconfig fsm-mesh-config -n "$fsm_namespace" -o jsonpath='{.spec.traffic.outboundIPRangeExclusionList}{"\n"}'
["54.91.118.50/32"]

  7. Restart the curl client pod so the updated outbound IP range exclusions can be configured. It is important to note that existing pods must be restarted to pick up the updated configuration because the traffic interception rules are programmed by the init container only at the time of pod creation.

    kubectl rollout restart deployment curl -n curl

    Wait for the restarted pod to be up and running.

  8. Confirm the curl client is able to make successful HTTP requests to the httpbin.org website running on http://54.91.118.50:80

    # 54.91.118.50 is one of the IP addresses for httpbin.org
$ kubectl exec -n curl -ti "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')" -c curl -- curl -I http://54.91.118.50:80
HTTP/1.1 200 OK
Date: Thu, 18 Mar 2021 23:17:44 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 9593
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

  9. Confirm that HTTP requests to other IP addresses of the httpbin.org website that are not excluded fail

    # 34.199.75.4 is one of the IP addresses for httpbin.org
$ kubectl exec -n curl -ti "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')" -c curl -- curl -I http://34.199.75.4:80
curl: (7) Failed to connect to 34.199.75.4 port 80: Connection refused
command terminated with exit code 7

2 - Retry Policy

Using retries to enhance service availability

This guide demonstrates how to configure retry policy for a client and server application within the service mesh.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.

Demo

  1. Install FSM with permissive mode and retry policy enabled.

    fsm install --set=fsm.enablePermissiveTrafficPolicy=true --set=fsm.featureFlags.enableRetryPolicy=true

  2. Deploy the httpbin service into the httpbin namespace after enrolling its namespace to the mesh. The httpbin service runs on port 14001.

    kubectl create namespace httpbin

fsm namespace add httpbin

kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/httpbin/httpbin.yaml -n httpbin

    Confirm the httpbin service and pods are up and running.

    kubectl get svc,pod -n httpbin

    Should look similar to below

    NAME      TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)     AGE
httpbin   ClusterIP   10.96.198.23   <none>        14001/TCP   20s

NAME                     READY   STATUS    RESTARTS   AGE
httpbin-5b8b94b9-lt2vs   2/2     Running   0          20s

  3. Deploy the curl into the curl namespace after enrolling its namespace to the mesh.

    kubectl create namespace curl

fsm namespace add curl

kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/curl/curl.yaml -n curl

    Confirm the curl pod is up and running.

    kubectl get pods -n curl

    Should look similar to below.

    NAME                    READY   STATUS    RESTARTS   AGE
curl-54ccc6954c-9rlvp   2/2     Running   0          20s

  4. Apply the Retry policy to retry when the curl ServiceAccount receives a 5xx code when sending a request to httpbin Service.

    kubectl apply -f - <<EOF
kind: Retry
apiVersion: policy.flomesh.io/v1alpha1
metadata:
  name: retry
  namespace: curl
spec:
  source:
    kind: ServiceAccount
    name: curl
    namespace: curl
  destinations:
  - kind: Service
    name: httpbin
    namespace: httpbin
  retryPolicy:
    retryOn: "5xx"
    perTryTimeout: 1s
    numRetries: 5
    retryBackoffBaseInterval: 1s
EOF

  5. Send a HTTP request that returns status code 503 from the curl pod to the httpbin service.

    curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"
kubectl exec "$curl_client" -n curl -c curl -- curl -sI httpbin.httpbin.svc.cluster.local:14001/status/503

    Returned result might look like

    HTTP/1.1 503 SERVICE UNAVAILABLE
server: gunicorn
date: Tue, 14 Feb 2023 11:11:51 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
connection: keep-alive

  6. Query for the stats between curl to httpbin.

    curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"
fsm proxy get stats -n curl "$curl_client" | grep upstream_rq_retry

    The number of times the request from the curl pod to the httpbin pod was retried using the exponential backoff retry should be equal to the numRetries field in the retry policy.

    The upstream_rq_retry_limit_exceeded stat shows the number of requests not retried because it’s more than the maximum retries allowed - numRetries.

    cluster.httpbin/httpbin|14001.upstream_rq_retry: 4
cluster.httpbin/httpbin|14001.upstream_rq_retry_backoff_exponential: 4
cluster.httpbin/httpbin|14001.upstream_rq_retry_backoff_ratelimited: 0
cluster.httpbin/httpbin|14001.upstream_rq_retry_limit_exceeded: 1
cluster.httpbin/httpbin|14001.upstream_rq_retry_overflow: 0
cluster.httpbin/httpbin|14001.upstream_rq_retry_success: 0

  7. Send a HTTP request that returns a non-5xx status code from the curl pod to the httpbin service.

    curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"
kubectl exec "$curl_client" -n curl -c curl -- curl -sI httpbin.httpbin.svc.cluster.local:14001/status/404

    Returned result might look something like

    HTTP/1.1 404 NOT FOUND
server: gunicorn
date: Tue, 14 Feb 2023 11:18:56 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
connection: keep-alive

3 - TCP Traffic Routing

Set up TCP traffic routing

This guide demonstrates a TCP client and server application within the service mesh communicating using FSM’s TCP routing capability.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have FSM installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.

Demo

The following demo shows a TCP client sending data to a tcp-echo server, which then echoes back the data to the client over a TCP connection.

  1. Set the namespace where FSM is installed.

    fsm_namespace=fsm-system  # Replace fsm-system with the namespace where FSM is installed if different

  2. Deploy the tcp-echo service in the tcp-demo namespace. The tcp-echo service runs on port 9000 with the appProtocol field set to tcp, which indicates to FSM that TCP routing must be used for traffic directed to the tcp-echo service on port 9000.

    # Create the tcp-demo namespace
kubectl create namespace tcp-demo

# Add the namespace to the mesh
fsm namespace add tcp-demo

# Deploy the service
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/tcp-echo.yaml -n tcp-demo

    Confirm the tcp-echo service and pod is up and running.

    $ kubectl get svc,po -n tcp-demo
NAME               TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
service/tcp-echo   ClusterIP   10.0.216.68   <none>        9000/TCP   97s

NAME                            READY   STATUS    RESTARTS   AGE
pod/tcp-echo-6656b7c4f8-zt92q   2/2     Running   0          97s

  3. Deploy the curl client into the curl namespace.

    # Create the curl namespace
kubectl create namespace curl

# Add the namespace to the mesh
fsm namespace add curl

# Deploy curl client in the curl namespace
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/curl/curl.yaml -n curl

    Confirm the curl client pod is up and running.

    $ kubectl get pods -n curl
NAME                    READY   STATUS    RESTARTS   AGE
curl-54ccc6954c-9rlvp   2/2     Running   0          20s

Using Permissive Traffic Policy Mode

We will enable service discovery using permissive traffic policy mode, which allows application connectivity to be established without the need for explicit SMI policies.

  1. Enable permissive traffic policy mode

    kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":true}}}' --type=merge

  2. Confirm the curl client is able to send and receive a response from the tcp-echo service using TCP routing.

    $ kubectl exec -n curl -ti "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')" -c curl -- sh -c 'echo hello | nc tcp-echo.tcp-demo 9000'
echo response: hello

    The tcp-echo service should echo back the data sent by the client. In the above example, the client sends hello, and the tcp-echo service responds with echo response: hello.

Using SMI Traffic Policy Mode

When using SMI traffic policy mode, explicit traffic policies must be configured to allow application connectivity. We will set up SMI policies to allow the curl client to communicate with the tcp-echo service on port 9000.

  1. Enable SMI traffic policy mode by disabling permissive traffic policy mode

    kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":false}}}' --type=merge

  2. Confirm the curl client is unable to send and receive a response from the tcp-echo service in the absence of SMI policies.

    $ kubectl exec -n curl -ti "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')" -c curl -- sh -c 'echo hello | nc tcp-echo.tcp-demo 9000'
command terminated with exit code 1

  3. Configure SMI traffic access and routing policies.

    kubectl apply -f - <<EOF
# TCP route to allows access to tcp-echo:9000
apiVersion: specs.smi-spec.io/v1alpha4
kind: TCPRoute
metadata:
name: tcp-echo-route
namespace: tcp-demo
spec:
matches:
    ports:
    - 9000
---
# Traffic target to allow curl app to access tcp-echo service using a TCPRoute
kind: TrafficTarget
apiVersion: access.smi-spec.io/v1alpha3
metadata:
name: tcp-access
namespace: tcp-demo
spec:
destination:
    kind: ServiceAccount
    name: tcp-echo
    namespace: tcp-demo
sources:
- kind: ServiceAccount
    name: curl
    namespace: curl
rules:
- kind: TCPRoute
    name: tcp-echo-route
EOF

  4. Confirm the curl client is able to send and receive a response from the tcp-echo service using SMI TCP route.

    $ kubectl exec -n curl -ti "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')" -c curl -- sh -c 'echo hello | nc tcp-echo.tcp-demo 9000'
echo response: hello

4 - Canary Rollouts using SMI Traffic Split

Managing Canary rollouts using SMI Taffic Split

This guide demonstrates how to perform Canary rollouts using the SMI Traffic Split configuration.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have FSM installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.

Demonstration

Explanation

In this demo, we use two applications, curl and httpbin implemented with Pipy, to act as client and server respectively. The service has two versions, v1 and v2, which are simulated by deploying httpbin-v1 and httpbin-v2.

Observant viewers may notice the frequent use of Pipy to implement httpbin functionalities in demonstrations. This is because web services implemented with Pipy can easily customize response content, facilitating the observation of test results.

Prerequisites

  • Kubernetes cluster
  • kubectl CLI

Installing the Service Mesh

Download the FSM CLI.

system=$(uname -s | tr '[:upper:]' '[:lower:]')
arch=$(uname -m | sed -E 's/x86_/amd/' | sed -E 's/aarch/arm/')
release=v1.2.3
curl -L https://github.com/flomesh-io/fsm/releases/download/${release}/fsm-${release}-${system}-${arch}.tar.gz | tar -vxzf -
cp ./${system}-amd64/fsm /usr/local/bin/fsm

Install the service mesh and wait for all components to run successfully.

fsm install --timeout 120s

Deploying the Sample Application

The curl and httpbin applications run in their respective namespaces, which are managed by the service mesh through the fsm namespace add xxx command.

kubectl create ns httpbin
kubectl create ns curl
fsm namespace add httpbin curl

Deploy the v1 version of httpbin which returns Hi, I am v1! for all HTTP requests. Other applications access httpbin through the Service httpbin.

kubectl apply -n httpbin -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: httpbin
spec:
  ports:
    - name: pipy
      port: 8080
      targetPort: 8080
      protocol: TCP
  selector:
    app: pipy
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-v1
  labels:
    app: pipy
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pipy
      version: v1
  template:
    metadata:
      labels:
        app: pipy
        version: v1
    spec:
      containers:
        - name: pipy
          image: flomesh/pipy:latest
          ports:
            - name: pipy
              containerPort: 8080
          command:
            - pipy
            - -e
            - |
              pipy()
              .listen(8080)
              .serveHTTP(new Message('Hi, I am v1!\n'))
EOF

Deploy the curl application.

kubectl apply -n curl -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: curl
  labels:
    app: curl
    service: curl
spec:
  ports:
    - name: http
      port: 80
  selector:
    app: curl
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: curl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: curl
  template:
    metadata:
      labels:
        app: curl
    spec:
      containers:
      - image: curlimages/curl
        imagePullPolicy: IfNotPresent
        name: curl
        command: ["sleep", "365d"]
EOF

Wait for all applications to run successfully.

kubectl wait --for=condition=ready pod --all -A

Test application access.

curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"
# send four request
kubectl exec "$curl_client" -n curl -c curl -- curl -s httpbin.httpbin:8080 httpbin.httpbin:8080 httpbin.httpbin:8080 httpbin.httpbin:8080

Expected results are displayed.

Hi, I am v1!
Hi, I am v1!
Hi, I am v1!
Hi, I am v1!

Next, deploy the v2 version of httpbin.

Deploying Version v2

The v2 version of httpbin returns Hi, I am v2! for all HTTP requests. Before deployment, we need to set the default traffic split strategy, otherwise, the new version instances would be accessible through the Service httpbin.

Create Service httpbin-v1 with an additional version tag in its selector compared to Service httpbin. Currently, both have the same endpoints.

kubectl apply -n httpbin -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: httpbin-v1
spec:
  ports:
    - name: pipy
      port: 8080
      targetPort: 8080
      protocol: TCP
  selector:
    app: pipy
    version: v1
EOF

Apply the TrafficSplit strategy to route all traffic to httpbin-v1.

kubectl apply -n httpbin -f - <<EOF
apiVersion: split.smi-spec.io/v1alpha4
kind: TrafficSplit
metadata:
  name: httpbin-split
spec:
  service: httpbin
  backends:
  - service: httpbin-v1
    weight: 100
EOF

Then deploy the new version.

kubectl apply -n httpbin -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: httpbin-v2
spec:
  ports:
    - name: pipy
      port: 8080
      targetPort: 8080
      protocol: TCP
  selector:
    app: pipy
    version: v2
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-v2
  labels:
    app: pipy
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pipy
      version: v2
  template:
    metadata:
      labels:
        app: pipy
        version: v2
    spec:
      containers:
        - name: pipy
          image: flomesh/pipy:latest
          ports:
            - name: pipy
              containerPort: 8080
          command:
            - pipy
            - -e
            - |
              pipy()
              .listen(8080)
              .serveHTTP(new Message('Hi, I am v2!\n'))
EOF

Wait for the new version to run successfully.

kubectl wait --for=condition=ready pod -n httpbin -l version=v2

If you send requests again, v1 httpbin still handles them.

kubectl exec "$curl_client" -n curl -c curl -- curl -s httpbin.httpbin:8080 httpbin.httpbin:8080 httpbin.httpbin:8080 httpbin.httpbin:8080
Hi, I am v1!
Hi, I am v1!
Hi, I am v1!
Hi, I am v1!

Canary Release

Modify the traffic split strategy to direct 25% of the traffic to version v2.

kubectl apply -n httpbin -f - <<EOF
apiVersion: split.smi-spec.io/v1alpha4
kind: TrafficSplit
metadata:
  name: httpbin-split
spec:
  service: httpbin
  backends:
  - service: httpbin-v1
    weight: 75
  - service: httpbin-v2
    weight: 25
EOF

Sending requests again shows that 1/4 of the traffic is handled by version v2.

kubectl exec "$curl_client" -n curl -c curl -- curl -s httpbin.httpbin:8080 httpbin.httpbin:8080 httpbin.httpbin:8080 httpbin.httpbin:8080
Hi, I am v1!
Hi, I am v2!
Hi, I am v1!
Hi, I am v1!

Advanced Canary Release

Suppose in version v2, /test endpoint functionality has been updated. For risk control, we want only a portion of the traffic to access the /test endpoint of the v2 version, while other endpoints like /demo should only access the v1 version.

We need to introduce another resource to define the traffic visiting the /test endpoint: a route definition. Here we define two routes:

  • httpbin-test: Traffic using the GET method to access /test.
  • httpbin-all: Traffic using the GET method to access /, note this is a prefix match.
kubectl apply -n httpbin -f - <<EOF
apiVersion: specs.smi-spec.io/v1alpha4
kind: HTTPRouteGroup
metadata:
  name: httpbin-test
spec:
  matches:
  - name: test
    pathRegex: "/test"
    methods:
    - GET
---
apiVersion: specs.smi-spec.io/v1alpha4
kind: HTTPRouteGroup
metadata:
  name: httpbin-all
spec:
  matches:
  - name: test
    pathRegex: ".*"
    methods:
    - GET
EOF

Then

update the traffic split strategy, associating the routes to the split. Also, create a new policy.

kubectl apply -n httpbin -f - <<EOF
apiVersion: split.smi-spec.io/v1alpha4
kind: TrafficSplit
metadata:
  name: httpbin-split
spec:
  service: httpbin
  matches:
  - name: httpbin-test
    kind: HTTPRouteGroup
  backends:
  - service: httpbin-v1
    weight: 75
  - service: httpbin-v2
    weight: 25
---
apiVersion: split.smi-spec.io/v1alpha4
kind: TrafficSplit
metadata:
  name: httpbin-all
spec:
  service: httpbin
  matches:
  - name: httpbin-all
    kind: HTTPRouteGroup
  backends:
  - service: httpbin-v1
    weight: 100
EOF

Now, when accessing the /test endpoint, only 25% of the traffic goes to the new version.

kubectl exec "$curl_client" -n curl -c curl -- curl -s httpbin.httpbin:8080/test httpbin.httpbin:8080/test httpbin.httpbin:8080/test httpbin.httpbin:8080/test
Hi, I am v1!
Hi, I am v2!
Hi, I am v1!
Hi, I am v1!

And requests to the /demo endpoint all go to the old v1 version.

kubectl exec "$curl_client" -n curl -c curl -- curl -s httpbin.httpbin:8080/demo httpbin.httpbin:8080/demo httpbin.httpbin:8080/demo httpbin.httpbin:8080/demo
Hi, I am v1!
Hi, I am v1!
Hi, I am v1!
Hi, I am v1!

This meets the expectations.

5 - Circuit breaking for destinations within the mesh

Configuring circuit breaking for destinations within the mesh

This guide demonstrates how to configure circuit breaking for destinations that are a part of an FSM managed service mesh.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have FSM installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.
  • FSM version >= v1.0.0.

Demo

The following demo shows a load-testing client fortio sending traffic to the httpbin service. We will see how applying circuit breakers for traffic to the httpbin service impacts the fortio client when the configured circuit breaking limits trip.

For simplicity, enable permissive traffic policy mode so that explicit SMI traffic access policies are not required for application connectivity within the mesh.

export FSM_NAMESPACE=fsm-system # Replace fsm-system with the namespace where FSM is installed
kubectl patch meshconfig fsm-mesh-config -n "$FSM_NAMESPACE" -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":true}}}'  --type=merge

Deploy services

Deploy server service.

kubectl create namespace server
fsm namespace add server
kubectl apply -n server -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: fortio
  labels:
    app: fortio
    service: fortio
spec:
  ports:
  - port: 8080
    name: http-8080
  - port: 8078
    name: tcp-8078
  - port: 8079
    name: grpc-8079
  selector:
    app: fortio
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fortio
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fortio
  template:
    metadata:
      labels:
        app: fortio
    spec:
      containers:
      - name: fortio
        image: fortio/fortio:latest_release
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: http
        - containerPort: 8078
          name: tcp
        - containerPort: 8079
          name: grpc
EOF

Deploy client service.

kubectl create namespace client
fsm namespace add client

kubectl apply -n client -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: fortio-client
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fortio-client
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fortio-client
  template:
    metadata:
      labels:
        app: fortio-client
    spec:
      serviceAccountName: fortio-client
      containers:
      - name: fortio-client
        image: fortio/fortio:latest_release
        imagePullPolicy: Always
EOF

Test

Confirm the fortio-client is able to successfully make HTTP requests to the fortio-server service on port 8080. We call the service with 10 concurrent connections (-c 10) and send 1000 requests (-n 1000). Service is configured to return HTTP status code of 511 for 20% requests.

fortio_client=`kubectl get pod -n client -l app=fortio-client -o jsonpath='{.items[0].metadata.name}'`
kubectl exec "$fortio_client" -n client -c fortio-client -- fortio load -quiet -c 10 -n 1000 -qps 200 -p 99.99 http://fortio.server.svc.cluster.local:8080/echo?status=511:20

Returned result might look something like below:

Sockets used: 205 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false, Catchup allowed: true
IP addresses distribution:
10.43.43.151:8080: 205
Code 200 : 804 (80.4 %)
Code 511 : 196 (19.6 %)
All done 1000 calls (plus 0 warmup) 2.845 ms avg, 199.8 qps

Next, apply a circuit breaker configuration using the UpstreamTrafficSetting resource for traffic directed to the fortio-server service.

Policy: Error Request Count Triggers Circuit Breaker

The error request count trigger threshold is set to errorAmountThreshold=100, the circuit breaker is triggered when the error request count reaches 100, returning 503 Service Unavailable!, the circuit breaker lasts for 10s

kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: http-circuit-breaking
  namespace: server
spec:
  host: fortio.server.svc.cluster.local
  connectionSettings:
    http:
      circuitBreaking:
        statTimeWindow: 1m
        minRequestAmount: 200
        errorAmountThreshold: 100
        degradedTimeWindow: 10s
        degradedStatusCode: 503
        degradedResponseContent: 'Service Unavailable!'
EOF

Set 20% of the server’s responses to return error code 511. When the error request count reaches 100, the number of successful requests should be around 400.

kubectl exec "$fortio_client" -n client -c fortio-client -- fortio load -quiet -c 10 -n 1000 -qps 200 -p 99.99 http://fortio.server.svc.cluster.local:8080/echo\?status\=511:20

From the results, it meets the expectations. After circuit breaker, the requests return the 503 error code.

Sockets used: 570 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false, Catchup allowed: true
IP addresses distribution:
10.43.43.151:8080: 570
Code 200 : 430 (43.0 %)
Code 503 : 470 (47.0 %)
Code 511 : 100 (10.0 %)
All done 1000 calls (plus 0 warmup) 3.376 ms avg, 199.8 qps

Checking the sidecar logs, the error count reached 100 at the 530th request, triggering the circuit breaker.

2023-02-08 01:08:01.456 [INF] [circuit_breaker] total/slowAmount/errorAmount (open) server/fortio|8080 530 0 100

Policy: Error Rate Triggers Circuit Breaker

Here we change the trigger from error count to error rate: errorRatioThreshold=0.10, the circuit breaker is triggered when the error rate reaches 10%, it lasts for 10s and returns 503 Service Unavailable!. Note that the minimum number of requests is still 200.

kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: http-circuit-breaking
  namespace: server
spec:
  host: fortio.server.svc.cluster.local
  connectionSettings:
    http:
      circuitBreaking:
        statTimeWindow: 1m
        minRequestAmount: 200
        errorRatioThreshold: 0.10
        degradedTimeWindow: 10s
        degradedStatusCode: 503
        degradedResponseContent: 'Service Unavailable!'
EOF

Set 20% of the server’s responses to return error code 511.

kubectl exec "$fortio_client" -n client -c fortio-client -- fortio load -quiet -c 10 -n 1000 -qps 200 -p 99.99 http://fortio.server.svc.cluster.local:8080/echo\?status\=511:20

From the output results, it can be seen that after 200 requests, the circuit breaker was triggered, and the number of circuit breaker requests was 800.

Sockets used: 836 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false, Catchup allowed: true
IP addresses distribution:
10.43.43.151:8080: 836
Code 200 : 164 (16.4 %)
Code 503 : 800 (80.0 %)
Code 511 : 36 (3.6 %)
All done 1000 calls (plus 0 warmup) 3.605 ms avg, 199.8 qps

Checking the sidecar logs, after satisfying the minimum number of requests to trigger the circuit breaker (200), the error rate also reached the threshold, triggering the circuit breaker.

2023-02-08 01:19:25.874 [INF] [circuit_breaker] total/slowAmount/errorAmount (close) server/fortio|8080 200 0 36

Policy: Slow Call Request Count Triggers Circuit Breaker

Testing slow calls, add a 200ms delay to 20% of the requests.

kubectl exec "$fortio_client" -n client -c fortio-client -- fortio load -quiet -c 10 -n 1000 -qps 200 -p 50,78,79,80,81,82,90,95 http://fortio.server.svc.cluster.local:8080/echo\?delay\=200ms:20

A similar effect is achieved, with nearly 80% of requests taking less than 200ms.

# target 50% 0.000999031
# target 78% 0.0095
# target 79% 0.200175
# target 80% 0.200467
# target 81% 0.200759
# target 82% 0.20105
# target 90% 0.203385
# target 95% 0.204844

285103 max 0.000285103 sum 0.000285103
Sockets used: 10 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false, Catchup allowed: true
IP addresses distribution:
10.43.43.151:8080: 10
Code 200 : 1000 (100.0 %)
All done 1000 calls (plus 0 warmup) 44.405 ms avg, 149.6 qps

Setting strategy:

  • Set the slow call request time threshold to 200ms
  • Set the number of slow call requests to 100
kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: http-circuit-breaking
  namespace: server
spec:
  host: fortio.server.svc.cluster.local
  connectionSettings:
    http:
      circuitBreaking:
        statTimeWindow: 1m
        minRequestAmount: 200
        slowTimeThreshold: 200ms
        slowAmountThreshold: 100
        degradedTimeWindow: 10s
        degradedStatusCode: 503
        degradedResponseContent: 'Service Unavailable!'
EOF

Inject a 200ms delay for 20% of requests.

kubectl exec "$fortio_client" -n client -c fortio-client -- fortio load -quiet -c 10 -n 1000 -qps 200 -p 50,78,79,80,81,82,90,95 http://fortio.server.svc.cluster.local:8080/echo\?delay\=200ms:20

The number of successful requests is 504, with 20% taking 200ms, and the number of slow requests has reached the threshold to trigger a circuit breaker.

# target 50% 0.00246111
# target 78% 0.00393846
# target 79% 0.00398974
# target 80% 0.00409756
# target 81% 0.00421951
# target 82% 0.00434146
# target 90% 0.202764
# target 95% 0.220036

Sockets used: 496 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false, Catchup allowed: true
IP addresses distribution:
10.43.43.151:8080: 496
Code 200 : 504 (50.4 %)
Code 503 : 496 (49.6 %)
All done 1000 calls (plus 0 warmup) 24.086 ms avg, 199.8 qps

Checking the logs of the sidecar, the number of slow calls reached 100 on the 504th request, triggering the circuit breaker.

2023-02-08 07:27:01.106 [INF] [circuit_breaker] total/slowAmount/errorAmount (open)  server/fortio|8080 504 100 0

Policy: Slow Call Rate Triggers Circuit Breaking

The slow call rate is set to 0.10, meaning that the circuit breaker will be triggered when it is expected that 10% of the requests within the statistical window will take more than 200ms.

kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: http-circuit-breaking
  namespace: server
spec:
  host: fortio.server.svc.cluster.local
  connectionSettings:
    http:
      circuitBreaking:
        statTimeWindow: 1m
        minRequestAmount: 200
        slowTimeThreshold: 200ms
        slowRatioThreshold: 0.1
        degradedTimeWindow: 10s
        degradedStatusCode: 503
        degradedResponseContent: 'Service Unavailable!'
EOF

Add 200ms delay to 20% of requests.

kubectl exec "$fortio_client" -n client -c fortio-client -- fortio load -quiet -c 10 -n 1000 -qps 200 -p 50,78,79,80,81,82,90,95 http://fortio.server.svc.cluster.local:8080/echo\?delay\=200ms:20

From the output results, there are 202 successful requests which meets the minimum number of requests required for the configuration to take effect. Among them, 20% are slow calls, reaching the threshold for triggering circuit breaker.

# target 50% 0.00305539
# target 78% 0.00387172
# target 79% 0.00390087
# target 80% 0.00393003
# target 81% 0.00395918
# target 82% 0.00398834
# target 90% 0.00458915
# target 95% 0.00497674

Sockets used: 798 (for perfect keepalive, would be 10)
Uniform: false, Jitter: false, Catchup allowed: true
IP addresses distribution:
10.43.43.151:8080: 798
Code 200 : 202 (20.2 %)
Code 503 : 798 (79.8 %)
All done 1000 calls (plus 0 warmup) 10.133 ms avg, 199.8 qps

Check the logs of the sidecar, at the 202nd request, the number of slow requests was 28, which triggered the circuit breaker.

2023-02-08 07:38:25.284 [INF] [circuit_breaker] total/slowAmount/errorAmount (open)  server/fortio|8080 202 28 0

6 - Local rate limiting of L4 connections

Configuring local rate limiting for L4 connections

This guide demonstrates how to configure rate limiting for L4 TCP connections destined to a target host that is a part of a FSM managed service mesh.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have FSM installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.
  • FSM version >= v1.2.0.

Demo

The following demo shows a client fortio-client sending TCP traffic to the fortio TCP echo service. The fortio service echoes TCP messages back to the client. We will see the impact of applying local TCP rate limiting policies targeting the fortio service to control the throughput of traffic destined to the service backend.

  1. For simplicity, enable permissive traffic policy mode so that explicit SMI traffic access policies are not required for application connectivity within the mesh.

    export FSM_NAMESPACE=fsm-system # Replace fsm-system with the namespace where FSM is installed
kubectl patch meshconfig fsm-mesh-config -n "$FSM_NAMESPACE" -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":true}}}'  --type=merge

  2. Deploy the fortio TCP echo service in the demo namespace after enrolling its namespace to the mesh. The fortio TCP echo service runs on port 8078.

    # Create the demo namespace
kubectl create namespace demo

# Add the namespace to the mesh
fsm namespace add demo

# Deploy fortio TCP echo in the demo namespace
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/fortio/fortio.yaml -n demo

    Confirm the fortio service pod is up and running.

    kubectl get pods -n demo
NAME                            READY   STATUS    RESTARTS   AGE
fortio-c4bd7857f-7mm6w          2/2     Running   0          22m

  3. Deploy the fortio-client app in the demo namespace. We will use this client to send TCP traffic to the fortio TCP echo service deployed previously.

    kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/fortio/fortio-client.yaml -n demo

    Confirm the fortio-client pod is up and running.

    NAME                            READY   STATUS    RESTARTS   AGE
fortio-client-b9b7bbfb8-prq7r   2/2     Running   0          7s

  4. Confirm the fortio-client app is able to successfully make TCP connections and send data to the frotio TCP echo service on port 8078. We call the fortio service with 3 concurrent connections (-c 3) and send 10 calls (-n 10).

    fortio_client="$(kubectl get pod -n demo -l app=fortio-client -o jsonpath='{.items[0].metadata.name}')"

kubectl exec "$fortio_client" -n demo -c fortio-client -- fortio load -qps -1 -c 3 -n 10 tcp://fortio.demo.svc.cluster.local:8078

Fortio 1.32.3 running at -1 queries per second, 8->8 procs, for 10 calls: tcp://fortio.demo.svc.cluster.local:8078
20:41:47 I tcprunner.go:238> Starting tcp test for tcp://fortio.demo.svc.cluster.local:8078 with 3 threads at -1.0 qps
Starting at max qps with 3 thread(s) [gomax 8] for exactly 10 calls (3 per thread + 1)
20:41:47 I periodic.go:723> T001 ended after 34.0563ms : 3 calls. qps=88.0894283876992
20:41:47 I periodic.go:723> T000 ended after 35.3117ms : 4 calls. qps=113.2769025563765
20:41:47 I periodic.go:723> T002 ended after 44.0273ms : 3 calls. qps=68.13954069406937
Ended after 44.2097ms : 10 calls. qps=226.19
Aggregated Function Time : count 10 avg 0.01096615 +/- 0.01386 min 0.001588 max 0.0386716 sum 0.1096615
# range, mid point, percentile, count
>= 0.001588 <= 0.002 , 0.001794 , 40.00, 4
> 0.002 <= 0.003 , 0.0025 , 60.00, 2
> 0.003 <= 0.004 , 0.0035 , 70.00, 1
> 0.025 <= 0.03 , 0.0275 , 90.00, 2
> 0.035 <= 0.0386716 , 0.0368358 , 100.00, 1
# target 50% 0.0025
# target 75% 0.02625
# target 90% 0.03
# target 99% 0.0383044
# target 99.9% 0.0386349
Error cases : no data
Sockets used: 3 (for perfect no error run, would be 3)
Total Bytes sent: 240, received: 240
tcp OK : 10 (100.0 %)
All done 10 calls (plus 0 warmup) 10.966 ms avg, 226.2 qps

    As seen above, all the TCP connections from the fortio-client pod succeeded.

    Total Bytes sent: 240, received: 240
tcp OK : 10 (100.0 %)
All done 10 calls (plus 0 warmup) 10.966 ms avg, 226.2 qps

  5. Next, apply a local rate limiting policy to rate limit L4 TCP connections to the fortio.demo.svc.cluster.local service to 1 connection per minute.

    kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: tcp-rate-limit
  namespace: demo
spec:
  host: fortio.demo.svc.cluster.local
  rateLimit:
    local:
      tcp:
        connections: 1
        unit: minute
EOF

    Confirm no traffic has been rate limited yet by examining the stats on the fortio backend pod.

    fortio_server="$(kubectl get pod -n demo -l app=fortio -o jsonpath='{.items[0].metadata.name}')"
fsm proxy get stats "$fortio_server" -n demo | grep fortio.*8078.*rate_limit

no matches found: fortio.*8078.*rate_limit

  6. Confirm TCP connections are rate limited.

    kubectl exec "$fortio_client" -n demo -c fortio-client -- fortio load -qps -1 -c 3 -n 10 tcp://fortio.demo.svc.cluster.local:8078

Fortio 1.32.3 running at -1 queries per second, 8->8 procs, for 10 calls: tcp://fortio.demo.svc.cluster.local:8078
20:49:38 I tcprunner.go:238> Starting tcp test for tcp://fortio.demo.svc.cluster.local:8078 with 3 threads at -1.0 qps
Starting at max qps with 3 thread(s) [gomax 8] for exactly 10 calls (3 per thread + 1)
20:49:38 E tcprunner.go:203> [2] Unable to read: read tcp 10.244.1.19:59244->10.96.83.254:8078: read: connection reset by peer
20:49:38 E tcprunner.go:203> [0] Unable to read: read tcp 10.244.1.19:59246->10.96.83.254:8078: read: connection reset by peer
20:49:38 E tcprunner.go:203> [2] Unable to read: read tcp 10.244.1.19:59258->10.96.83.254:8078: read: connection reset by peer
20:49:38 E tcprunner.go:203> [0] Unable to read: read tcp 10.244.1.19:59260->10.96.83.254:8078: read: connection reset by peer
20:49:38 E tcprunner.go:203> [2] Unable to read: read tcp 10.244.1.19:59266->10.96.83.254:8078: read: connection reset by peer
20:49:38 I periodic.go:723> T002 ended after 9.643ms : 3 calls. qps=311.1065021258944
20:49:38 E tcprunner.go:203> [0] Unable to read: read tcp 10.244.1.19:59268->10.96.83.254:8078: read: connection reset by peer
20:49:38 E tcprunner.go:203> [0] Unable to read: read tcp 10.244.1.19:59274->10.96.83.254:8078: read: connection reset by peer
20:49:38 I periodic.go:723> T000 ended after 14.8212ms : 4 calls. qps=269.8836801338623
20:49:38 I periodic.go:723> T001 ended after 20.3458ms : 3 calls. qps=147.45057948077735
Ended after 20.5468ms : 10 calls. qps=486.69
Aggregated Function Time : count 10 avg 0.00438853 +/- 0.004332 min 0.0014184 max 0.0170216 sum 0.0438853
# range, mid point, percentile, count
>= 0.0014184 <= 0.002 , 0.0017092 , 20.00, 2
> 0.002 <= 0.003 , 0.0025 , 50.00, 3
> 0.003 <= 0.004 , 0.0035 , 70.00, 2
> 0.004 <= 0.005 , 0.0045 , 90.00, 2
> 0.016 <= 0.0170216 , 0.0165108 , 100.00, 1
# target 50% 0.003
# target 75% 0.00425
# target 90% 0.005
# target 99% 0.0169194
# target 99.9% 0.0170114
Error cases : count 7 avg 0.0034268714 +/- 0.0007688 min 0.0024396 max 0.0047932 sum 0.0239881
# range, mid point, percentile, count
>= 0.0024396 <= 0.003 , 0.0027198 , 42.86, 3
> 0.003 <= 0.004 , 0.0035 , 71.43, 2
> 0.004 <= 0.0047932 , 0.0043966 , 100.00, 2
# target 50% 0.00325
# target 75% 0.00409915
# target 90% 0.00451558
# target 99% 0.00476544
# target 99.9% 0.00479042
Sockets used: 8 (for perfect no error run, would be 3)
Total Bytes sent: 240, received: 72
tcp OK : 3 (30.0 %)
tcp short read : 7 (70.0 %)
All done 10 calls (plus 0 warmup) 4.389 ms avg, 486.7 qps

    As seen above, only 30% of the 10 calls succeeded, while the remaining 70% was rate limitied. This is because we applied a rate limiting policy of 1 connection per minute at the fortio backend service, and the fortio-client was able to use 1 connection to make 3/10 calls, resulting in a 30% success rate.

    Examine the sidecar stats to further confirm this.

    fsm proxy get stats "$fortio_server" -n demo | grep 'fortio.*8078.*rate_limit'
local_rate_limit.inbound_demo/fortio_8078_tcp.rate_limited: 7

  7. Next, let’s update our rate limiting policy to allow a burst of connections. Bursts allow a given number of connections over the baseline rate of 1 connection per minute defined by our rate limiting policy.

    kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: tcp-echo-limit
  namespace: demo
spec:
  host: fortio.demo.svc.cluster.local
  rateLimit:
    local:
      tcp:
        connections: 1
        unit: minute
        burst: 10
EOF

  8. Confirm the burst capability allows a burst of connections within a small window of time.

    kubectl exec "$fortio_client" -n demo -c fortio-client -- fortio load -qps -1 -c 3 -n 10 tcp://fortio.demo.svc.cluster.local:8078

Fortio 1.32.3 running at -1 queries per second, 8->8 procs, for 10 calls: tcp://fortio.demo.svc.cluster.local:8078
20:56:56 I tcprunner.go:238> Starting tcp test for tcp://fortio.demo.svc.cluster.local:8078 with 3 threads at -1.0 qps
Starting at max qps with 3 thread(s) [gomax 8] for exactly 10 calls (3 per thread + 1)
20:56:56 I periodic.go:723> T002 ended after 5.1568ms : 3 calls. qps=581.7561278312132
20:56:56 I periodic.go:723> T001 ended after 5.2334ms : 3 calls. qps=573.2411052088509
20:56:56 I periodic.go:723> T000 ended after 5.2464ms : 4 calls. qps=762.4275693809088
Ended after 5.2711ms : 10 calls. qps=1897.1
Aggregated Function Time : count 10 avg 0.00153124 +/- 0.001713 min 0.00033 max 0.0044054 sum 0.0153124
# range, mid point, percentile, count
>= 0.00033 <= 0.001 , 0.000665 , 70.00, 7
> 0.003 <= 0.004 , 0.0035 , 80.00, 1
> 0.004 <= 0.0044054 , 0.0042027 , 100.00, 2
# target 50% 0.000776667
# target 75% 0.0035
# target 90% 0.0042027
# target 99% 0.00438513
# target 99.9% 0.00440337
Error cases : no data
Sockets used: 3 (for perfect no error run, would be 3)
Total Bytes sent: 240, received: 240
tcp OK : 10 (100.0 %)
All done 10 calls (plus 0 warmup) 1.531 ms avg, 1897.1 qps

    As seen above, all the TCP connections from the fortio-client pod succeeded.

    Total Bytes sent: 240, received: 240
tcp OK : 10 (100.0 %)
All done 10 calls (plus 0 warmup) 1.531 ms avg, 1897.1 qps

    Further, examine the stats to confirm the burst allows additional connections to go through. The number of connections rate limited hasn’t increased since our previous rate limit test before we configured the burst setting.

    fsm proxy get stats "$fortio_server" -n demo | grep 'fortio.*8078.*rate_limit'
local_rate_limit.inbound_demo/fortio_8078_tcp.rate_limited: 0

7 - Local rate limiting of HTTP requests

Configuring local rate limiting for HTTP requests

This guide demonstrates how to configure rate limiting for HTTP requests destined to a target host that is a part of a FSM managed service mesh.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have FSM installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.
  • FSM version >= v1.2.0.

Demo

The following demo shows a client sending HTTP requests to the fortio service. We will see the impact of applying local HTTP rate limiting policies targeting the fortio service to control the throughput of requests destined to the service backend.

  1. For simplicity, enable permissive traffic policy mode so that explicit SMI traffic access policies are not required for application connectivity within the mesh.

    export FSM_NAMESPACE=fsm-system # Replace fsm-system with the namespace where FSM is installed
kubectl patch meshconfig fsm-mesh-config -n "$FSM_NAMESPACE" -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":true}}}'  --type=merge

  2. Deploy the fortio HTTP service in the demo namespace after enrolling its namespace to the mesh. The fortio HTTP service runs on port 8080.

    # Create the demo namespace
kubectl create namespace demo

# Add the namespace to the mesh
fsm namespace add demo

# Deploy fortio TCP echo in the demo namespace
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/fortio/fortio.yaml -n demo

    Confirm the fortio service pod is up and running.

    kubectl get pods -n demo
NAME                            READY   STATUS    RESTARTS   AGE
fortio-c4bd7857f-7mm6w          2/2     Running   0          22m

  3. Deploy the fortio-client app in the demo namespace. We will use this client to send TCP traffic to the fortio TCP echo service deployed previously.

    kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/fortio/fortio-client.yaml -n demo

    Confirm the fortio-client pod is up and running.

    kubectl get pods -n demo
NAME                            READY   STATUS    RESTARTS   AGE
fortio-client-b9b7bbfb8-prq7r   2/2     Running   0          7s

  4. Confirm the fortio-client app is able to successfully make HTTP requests to the fortio HTTP service on port 8080. We call the fortio service with 3 concurrent connections (-c 3) and send 10 requests (-n 10).

    fortio_client="$(kubectl get pod -n demo -l app=fortio-client -o jsonpath='{.items[0].metadata.name}')"

kubectl exec "$fortio_client" -n demo -c fortio-client -- fortio load -c 3 -n 10 http://fortio.demo.svc.cluster.local:8080

    You will get the result as below.

    Fortio 1.33.0 running at 8 queries per second, 8->8 procs, for 10 calls: http://fortio.demo.svc.cluster.local:8080
20:58:07 I httprunner.go:93> Starting http test for http://fortio.demo.svc.cluster.local:8080 with 3 threads at 8.0 qps and parallel warmup
Starting at 8 qps with 3 thread(s) [gomax 8] : exactly 10, 3 calls each (total 9 + 1)
20:58:08 I periodic.go:723> T002 ended after 1.1273523s : 3 calls. qps=2.661102478790348
20:58:08 I periodic.go:723> T001 ended after 1.1273756s : 3 calls. qps=2.661047480537986
20:58:08 I periodic.go:723> T000 ended after 1.5023464s : 4 calls. qps=2.662501803844972
Ended after 1.5024079s : 10 calls. qps=6.656
Sleep times : count 7 avg 0.52874391 +/- 0.03031 min 0.4865562 max 0.5604152 sum 3.7012074
Aggregated Function Time : count 10 avg 0.0050187 +/- 0.005515 min 0.0012575 max 0.0135401 sum 0.050187
# range, mid point, percentile, count
>= 0.0012575 <= 0.002 , 0.00162875 , 70.00, 7
> 0.012 <= 0.0135401 , 0.01277 , 100.00, 3
# target 50% 0.0017525
# target 75% 0.0122567
# target 90% 0.0130267
# target 99% 0.0134888
# target 99.9% 0.013535
Error cases : no data
20:58:08 I httprunner.go:190> [0] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
20:58:08 I httprunner.go:190> [1] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
20:58:08 I httprunner.go:190> [2] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
Sockets used: 3 (for perfect keepalive, would be 3)
Uniform: false, Jitter: false
IP addresses distribution:
10.96.189.159:8080: 3
Code 200 : 10 (100.0 %)
Response Header Sizes : count 10 avg 124.3 +/- 0.4583 min 124 max 125 sum 1243
Response Body/Total Sizes : count 10 avg 124.3 +/- 0.4583 min 124 max 125 sum 1243
All done 10 calls (plus 0 warmup) 5.019 ms avg, 6.7 qps

    As seen above, all the HTTP requests from the fortio-client pod succeeded.

    Code 200 : 10 (100.0 %)

  5. Next, apply a local rate limiting policy to rate limit HTTP requests at the virtual host level to 3 requests per minute.

    kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: http-rate-limit
  namespace: demo
spec:
  host: fortio.demo.svc.cluster.local
  rateLimit:
    local:
      http:
        requests: 3
        unit: minute
EOF

    Confirm no HTTP requests have been rate limited yet by examining the stats on the fortio backend pod.

    fortio_server="$(kubectl get pod -n demo -l app=fortio -o jsonpath='{.items[0].metadata.name}')"

fsm proxy get stats "$fortio_server" -n demo | grep 'http_local_rate_limiter.http_local_rate_limit.rate_limited'
http_local_rate_limiter.http_local_rate_limit.rate_limited: 0

  6. Confirm HTTP requests are rate limited.

    kubectl exec "$fortio_client" -n demo -c fortio-client -- fortio load -c 3 -n 10 http://fortio.demo.svc.cluster.local:8080

Fortio 1.33.0 running at 8 queries per second, 8->8 procs, for 10 calls: http://fortio.demo.svc.cluster.local:8080
21:06:36 I httprunner.go:93> Starting http test for http://fortio.demo.svc.cluster.local:8080 with 3 threads at 8.0 qps and parallel warmup
Starting at 8 qps with 3 thread(s) [gomax 8] : exactly 10, 3 calls each (total 9 + 1)
21:06:37 W http_client.go:838> [0] Non ok http code 429 (HTTP/1.1 429)
21:06:37 W http_client.go:838> [1] Non ok http code 429 (HTTP/1.1 429)
21:06:37 W http_client.go:838> [2] Non ok http code 429 (HTTP/1.1 429)
21:06:37 W http_client.go:838> [0] Non ok http code 429 (HTTP/1.1 429)
21:06:37 W http_client.go:838> [1] Non ok http code 429 (HTTP/1.1 429)
21:06:37 I periodic.go:723> T001 ended after 1.1269827s : 3 calls. qps=2.661975201571417
21:06:37 W http_client.go:838> [2] Non ok http code 429 (HTTP/1.1 429)
21:06:37 I periodic.go:723> T002 ended after 1.1271942s : 3 calls. qps=2.66147572441377
21:06:38 W http_client.go:838> [0] Non ok http code 429 (HTTP/1.1 429)
21:06:38 I periodic.go:723> T000 ended after 1.5021191s : 4 calls. qps=2.662904692444161
Ended after 1.5021609s : 10 calls. qps=6.6571
Sleep times : count 7 avg 0.53138026 +/- 0.03038 min 0.4943128 max 0.5602373 sum 3.7196618
Aggregated Function Time : count 10 avg 0.00318326 +/- 0.002431 min 0.0012651 max 0.0077951 sum 0.0318326
# range, mid point, percentile, count
>= 0.0012651 <= 0.002 , 0.00163255 , 60.00, 6
> 0.002 <= 0.003 , 0.0025 , 70.00, 1
> 0.005 <= 0.006 , 0.0055 , 80.00, 1
> 0.006 <= 0.007 , 0.0065 , 90.00, 1
> 0.007 <= 0.0077951 , 0.00739755 , 100.00, 1
# target 50% 0.00185302
# target 75% 0.0055
# target 90% 0.007
# target 99% 0.00771559
# target 99.9% 0.00778715
Error cases : count 7 avg 0.0016392143 +/- 0.000383 min 0.0012651 max 0.0023951 sum 0.0114745
# range, mid point, percentile, count
>= 0.0012651 <= 0.002 , 0.00163255 , 85.71, 6
> 0.002 <= 0.0023951 , 0.00219755 , 100.00, 1
# target 50% 0.00163255
# target 75% 0.00188977
# target 90% 0.00211853
# target 99% 0.00236744
# target 99.9% 0.00239233
21:06:38 I httprunner.go:190> [0] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
21:06:38 I httprunner.go:190> [1] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
21:06:38 I httprunner.go:190> [2] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
Sockets used: 7 (for perfect keepalive, would be 3)
Uniform: false, Jitter: false
IP addresses distribution:
10.96.189.159:8080: 3
Code 200 : 3 (30.0 %)
Code 429 : 7 (70.0 %)
Response Header Sizes : count 10 avg 37.2 +/- 56.82 min 0 max 124 sum 372
Response Body/Total Sizes : count 10 avg 166 +/- 27.5 min 124 max 184 sum 1660
All done 10 calls (plus 0 warmup) 3.183 ms avg, 6.7 qps

    As seen above, only 3 out of 10 HTTP requests succeeded, while the remaining 7 requests were rate limited as per the rate limiting policy.

    Code 200 : 3 (30.0 %)
Code 429 : 7 (70.0 %)

    Examine the stats to further confirm this.

    fsm proxy get stats "$fortio_server" -n demo | grep 'http_local_rate_limiter.http_local_rate_limit.rate_limited'

http_local_rate_limiter.http_local_rate_limit.rate_limited: 7

  7. Next, let’s update our rate limiting policy to allow a burst of requests. Bursts allow a given number of requests over the baseline rate of 3 requests per minute defined by our rate limiting policy.

    kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: http-rate-limit
  namespace: demo
spec:
  host: fortio.demo.svc.cluster.local
  rateLimit:
    local:
      http:
        requests: 3
        unit: minute
        burst: 10
EOF

  8. Confirm the burst capability allows a burst of requests within a small window of time.

    kubectl exec "$fortio_client" -n demo -c fortio-client -- fortio load -c 3 -n 10 http://fortio.demo.svc.cluster.local:8080

Fortio 1.33.0 running at 8 queries per second, 8->8 procs, for 10 calls: http://fortio.demo.svc.cluster.local:8080
21:11:04 I httprunner.go:93> Starting http test for http://fortio.demo.svc.cluster.local:8080 with 3 threads at 8.0 qps and parallel warmup
Starting at 8 qps with 3 thread(s) [gomax 8] : exactly 10, 3 calls each (total 9 + 1)
21:11:05 I periodic.go:723> T002 ended after 1.127252s : 3 calls. qps=2.6613392568831107
21:11:05 I periodic.go:723> T001 ended after 1.1273028s : 3 calls. qps=2.661219328116634
21:11:05 I periodic.go:723> T000 ended after 1.5019947s : 4 calls. qps=2.663125242718899
Ended after 1.5020768s : 10 calls. qps=6.6574
Sleep times : count 7 avg 0.53158916 +/- 0.03008 min 0.4943959 max 0.5600713 sum 3.7211241
Aggregated Function Time : count 10 avg 0.00318637 +/- 0.002356 min 0.0012401 max 0.0073302 sum 0.0318637
# range, mid point, percentile, count
>= 0.0012401 <= 0.002 , 0.00162005 , 60.00, 6
> 0.002 <= 0.003 , 0.0025 , 70.00, 1
> 0.005 <= 0.006 , 0.0055 , 80.00, 1
> 0.007 <= 0.0073302 , 0.0071651 , 100.00, 2
# target 50% 0.00184802
# target 75% 0.0055
# target 90% 0.0071651
# target 99% 0.00731369
# target 99.9% 0.00732855
Error cases : no data
21:11:05 I httprunner.go:190> [0] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
21:11:05 I httprunner.go:190> [1] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
21:11:05 I httprunner.go:190> [2] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
Sockets used: 3 (for perfect keepalive, would be 3)
Uniform: false, Jitter: false
IP addresses distribution:
10.96.189.159:8080: 3
Code 200 : 10 (100.0 %)
Response Header Sizes : count 10 avg 124 +/- 0 min 124 max 124 sum 1240
Response Body/Total Sizes : count 10 avg 124 +/- 0 min 124 max 124 sum 1240
All done 10 calls (plus 0 warmup) 3.186 ms avg, 6.7 qps

    As seen above, all HTTP requests succeeded as we allowed a burst of 10 requests with our rate limiting policy.

    Code 200 : 10 (100.0 %)

    Further, examine the stats to confirm the burst allows additional requests to go through. The number of requests rate limited hasn’t increased since our previous rate limit test before we configured the burst setting.

    fsm proxy get stats "$fortio_server" -n demo | grep 'http_local_rate_limiter.http_local_rate_limit.rate_limited'

http_local_rate_limiter.http_local_rate_limit.rate_limited: 0

  9. Next, let’s configure the rate limting policy for a specific HTTP route allowed on the upstream service.

    Note: Since we are using permissive traffic policy mode in the demo, an HTTP route with a wildcard path regex .* is allowed on the upstream backend, so we will configure a rate limiting policy for this route. However, when using SMI policies in the mesh, paths corresponding to matching allowed SMI HTTP routing rules can be configured.

    kubectl apply -f - <<EOF
apiVersion: policy.flomesh.io/v1alpha1
kind: UpstreamTrafficSetting
metadata:
  name: http-rate-limit
  namespace: demo
spec:
  host: fortio.demo.svc.cluster.local
  httpRoutes:
    - path: .*
      rateLimit:
        local:
          requests: 3
          unit: minute
EOF

  10. Confirm HTTP requests are rate limited at a per-route level.

    kubectl exec "$fortio_client" -n demo -c fortio-client -- fortio load -c 3 -n 10 http://fortio.demo.svc.cluster.local:8080

Fortio 1.33.0 running at 8 queries per second, 8->8 procs, for 10 calls: http://fortio.demo.svc.cluster.local:8080
21:19:34 I httprunner.go:93> Starting http test for http://fortio.demo.svc.cluster.local:8080 with 3 threads at 8.0 qps and parallel warmup
Starting at 8 qps with 3 thread(s) [gomax 8] : exactly 10, 3 calls each (total 9 + 1)
21:19:35 W http_client.go:838> [0] Non ok http code 429 (HTTP/1.1 429)
21:19:35 W http_client.go:838> [2] Non ok http code 429 (HTTP/1.1 429)
21:19:35 W http_client.go:838> [1] Non ok http code 429 (HTTP/1.1 429)
21:19:35 W http_client.go:838> [0] Non ok http code 429 (HTTP/1.1 429)
21:19:35 W http_client.go:838> [1] Non ok http code 429 (HTTP/1.1 429)
21:19:35 W http_client.go:838> [2] Non ok http code 429 (HTTP/1.1 429)
21:19:35 I periodic.go:723> T001 ended after 1.126703s : 3 calls. qps=2.6626360274180505
21:19:35 I periodic.go:723> T002 ended after 1.1267472s : 3 calls. qps=2.6625315776245104
21:19:36 W http_client.go:838> [0] Non ok http code 429 (HTTP/1.1 429)
21:19:36 I periodic.go:723> T000 ended after 1.5027817s : 4 calls. qps=2.6617305760377574
Ended after 1.5028359s : 10 calls. qps=6.6541
Sleep times : count 7 avg 0.53089959 +/- 0.03079 min 0.4903791 max 0.5604715 sum 3.7162971
Aggregated Function Time : count 10 avg 0.00369734 +/- 0.003165 min 0.0011174 max 0.0095033 sum 0.0369734
# range, mid point, percentile, count
>= 0.0011174 <= 0.002 , 0.0015587 , 60.00, 6
> 0.002 <= 0.003 , 0.0025 , 70.00, 1
> 0.007 <= 0.008 , 0.0075 , 90.00, 2
> 0.009 <= 0.0095033 , 0.00925165 , 100.00, 1
# target 50% 0.00182348
# target 75% 0.00725
# target 90% 0.008
# target 99% 0.00945297
# target 99.9% 0.00949827
Error cases : count 7 avg 0.0016556 +/- 0.0004249 min 0.0011174 max 0.0025594 sum 0.0115892
# range, mid point, percentile, count
>= 0.0011174 <= 0.002 , 0.0015587 , 85.71, 6
> 0.002 <= 0.0025594 , 0.0022797 , 100.00, 1
# target 50% 0.0015587
# target 75% 0.00186761
# target 90% 0.00216782
# target 99% 0.00252024
# target 99.9% 0.00255548
21:19:36 I httprunner.go:190> [0] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
21:19:36 I httprunner.go:190> [1] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
21:19:36 I httprunner.go:190> [2] fortio.demo.svc.cluster.local:8080 resolved to 10.96.189.159:8080
Sockets used: 7 (for perfect keepalive, would be 3)
Uniform: false, Jitter: false
IP addresses distribution:
10.96.189.159:8080: 3
Code 200 : 3 (30.0 %)
Code 429 : 7 (70.0 %)
Response Header Sizes : count 10 avg 37.2 +/- 56.82 min 0 max 124 sum 372
Response Body/Total Sizes : count 10 avg 166 +/- 27.5 min 124 max 184 sum 1660
All done 10 calls (plus 0 warmup) 3.697 ms avg, 6.7 qps

    As seen above, only 3 out of 10 HTTP requests succeeded, while the remaining 7 requests were rate limited as per the rate limiting policy.

    Code 200 : 3 (30.0 %)
Code 429 : 7 (70.0 %)

    Examine the stats to further confirm this. 7 additional requests have been rate limited after configuring HTTP route level rate limiting since our previous test, indicated by the total of 14 HTTP requests rate limited in the stats.

    fsm proxy get stats "$fortio_server" -n demo | grep 'http_local_rate_limiter.http_local_rate_limit.rate_limited'
http_local_rate_limiter.http_local_rate_limit.rate_limited: 14