This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Extending FSM with Plugins

Demos of multiple plugins to extend FSM functionality

1 - Identity and Access Management

Adding Identity and Access Management functionality to fsm sidecar via Plugins

In this demonstration, we will extend the IAM (Identity and Access Management) feature for the service mesh to enhance the security of the service. When service A accesses service B, it will carry the obtained token. After receiving the request, service B verifies the token through the authentication service, and based on the verification result, decides whether to serve the request or not.

Two plugins are required here:

  • token-injector to inject the token into the request from service A
  • token-verifyer to verify the identity of the request accessing service B.

Both of them handle outbound and inbound traffic, respectively.

Corresponding to this are two PluginChains:

  • token-injector-chain
  • token-verifier-chain

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have fsm installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.
  • Have jq command available.

Deploy demo services

kubectl create namespace curl
fsm namespace add curl
kubectl apply -n curl -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/curl/curl.yaml

kubectl create namespace httpbin
fsm namespace add httpbin
kubectl apply -n httpbin -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/httpbin/httpbin.yaml

sleep 2
kubectl wait --for=condition=ready pod -n curl -l app=curl --timeout=90s
kubectl wait --for=condition=ready pod -n httpbin -l app=httpbin --timeout=90s

curl_pod=`kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}'`
httpbin_pod=`kubectl get pod -n httpbin -l app=httpbin -o jsonpath='{.items..metadata.name}'`

To view the content of the plugin chains for both services. The built-in plugins are located in the modules directory.

These built-in plugins are native functions provided by the service mesh and are not configured through plugin mechanism, but can be overridden via plugin mechanism.

fsm proxy get config_dump -n curl $curl_pod | jq '.Chains."outbound-http"'
[
  "modules/outbound-http-routing.js",
  "modules/outbound-metrics-http.js",
  "modules/outbound-tracing-http.js",
  "modules/outbound-logging-http.js",
  "modules/outbound-circuit-breaker.js",
  "modules/outbound-http-load-balancing.js",
  "modules/outbound-http-default.js"
]

fsm proxy get config_dump -n httpbin $httpbin_pod | jq '.Chains."inbound-http"'
[
  "modules/inbound-tls-termination.js",
  "modules/inbound-http-routing.js",
  "modules/inbound-metrics-http.js",
  "modules/inbound-tracing-http.js",
  "modules/inbound-logging-http.js",
  "modules/inbound-throttle-service.js",
  "modules/inbound-throttle-route.js",
  "modules/inbound-http-load-balancing.js",
  "modules/inbound-http-default.js"
]

Test communication between the applications.

kubectl exec $curl_pod -n curl -c curl -- curl -Is http://httpbin.httpbin:14001/get
HTTP/1.1 200 OK
server: gunicorn/19.9.0
date: Sun, 05 Feb 2023 05:42:51 GMT
content-type: application/json
content-length: 304
access-control-allow-origin: *
access-control-allow-credentials: true
connection: keep-alive

Deploy authentication service

Deploy a standalone authentication service to authenticate requests and return 200 or 401. For simplicity, here we have hard-coded a valid token as 2f1acc6c3a606b082e5eef5e54414ffb

kubectl create namespace auth

kubectl apply -n auth -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: ext-auth
  name: ext-auth
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ext-auth
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: ext-auth
    spec:
      containers:
      - command:
        - pipy
        - -e
        - |2-

          pipy({
            _acceptTokens: ['2f1acc6c3a606b082e5eef5e54414ffb'],
            _allow: false,
          })

            // Pipeline layouts go here, e.g.:
            .listen(8079)
            .demuxHTTP().to($ => $
              .handleMessageStart(
                msg => ((token = msg?.head?.headers?.['x-iam-token']) =>
                  _allow = token && _acceptTokens?.find(el => el == token)
                )()
              )
              .branch(() => _allow, $ => $.replaceMessage(new Message({ status: 200 })),
                $ => $.replaceMessage(new Message({ status: 401 }))
              )
            )
        image: flomesh/pipy:latest
        name: pipy
        resources: {}
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: ext-auth
  name: ext-auth
spec:
  ports:
  - port: 8079
    protocol: TCP
    targetPort: 8079
  selector:
    app: ext-auth
EOF

Enable plugin policy mode

To enable plugin policy, the mesh configuration needs to be modified as plugin policies are not enabled by default.

export fsm_namespace=fsm-system
kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"featureFlags":{"enablePluginPolicy":true}}}' --type=merge

Declaring a plugin

Plugin token-injector:

  • metadata.name: the name of the plugin, which is also the name of the plugin script. For example, this plugin will be saved as token-injector.js stored in the plugins directory of the code repository.
  • spec.pipyscript: the PipyJS script content, which is the functional logic code, stored in the script file plugins/token-injector.js. Context metadata that is built-in to the system can be used within the script.
  • spec.priority: the priority of the plugin, with optional values of 0-65535. The higher the value, the higher the priority, and the earlier the plugin is positioned in the plugin chain. The value here is 115, which, based on the built-in plugin list in Helm values.yaml, will be positioned between modules/outbound-circuit-breaker.js and modules/outbound-http-load-balancing.js, executed after the circuit breaker logic is processed and before the load balancer forwards to the upstream.
kubectl apply -f - <<EOF
kind: Plugin
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: token-injector
spec:
  priority: 115
  pipyscript: |+
    (
    pipy({
      _pluginName: '',
      _pluginConfig: null,
      _accessToken: null,
    })
    .import({
        __service: 'outbound-http-routing',
    })
    .pipeline()
    .onStart(
        () => void (
            _pluginName = __filename.slice(9, -3),
            _pluginConfig = __service?.Plugins?.[_pluginName],
            _accessToken = _pluginConfig?.AccessToken
        )
    )
    .handleMessageStart(
        msg => _accessToken && (msg.head.headers['x-iam-token'] = _accessToken)
    )
    .chain()
    )
EOF

Plugin token-verifier

kubectl apply -f - <<EOF
kind: Plugin
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: token-verifier
spec:
  priority: 115
  pipyscript: |+
    (
    pipy({
        _pluginName: '',
        _pluginConfig: null,
        _verifier: null,
        _authPaths: null,
        _authRequred: false,
        _authSuccess: undefined,
    })
    .import({
        __service: 'inbound-http-routing',
    })
    .pipeline()
    .onStart(
        () => void (
            _pluginName = __filename.slice(9, -3),
            _pluginConfig = __service?.Plugins?.[_pluginName],
            _verifier = _pluginConfig?.Verifier,
            _authPaths = _pluginConfig?.Paths && _pluginConfig.Paths?.length > 0 && (
              new algo.URLRouter(Object.fromEntries(_pluginConfig.Paths.map(path => [path, true])))
            )
        )
    )
    .handleMessageStart(
        msg => _authRequred = (_verifier && _authPaths?.find(msg.head.headers.host, msg.head.path))
    )
    .branch(
        () => _authRequred, (
        $ => $
          .fork().to($ => $
            .muxHTTP().to($ => $.connect(()=> _verifier))
            .handleMessageStart(
              msg => _authSuccess = (msg.head.status == 200)
            )
          )
          .wait(() => _authSuccess !== undefined)
          .branch(() => _authSuccess, $ => $.chain(),
            $ => $.replaceMessage(
              () => new Message({ status: 401 }, 'Unauthorized!')
            )
          )
      ),
        $ => $.chain()
      )
    )
EOF

Setting up plugin-chain

plugin chain token-injector-chain:

  • metadata.name: name of plugin chain resource token-injector-chain
  • spec.chains
    • name: name of the plugin chain, one of the 4 plugin chains, here it is outbound-http which is the HTTP protocol processing stage for outbound traffic.
    • plugins: list of plugins to be inserted into the plugin chain, here token-injector is inserted into the plugin chain.
  • spec.selectors: target of the plugin chain, using Kubernetes label selector scheme.
    • podSelector: pod selector, selects pods with label app=curl.
    • namespaceSelector: namespace selector, selects namespaces managed by the mesh, i.e., flomesh.io/monitored-by=fsm.
kubectl apply -n curl -f - <<EOF
kind: PluginChain
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: token-injector-chain
spec:
  chains:
    - name: outbound-http
      plugins:
        - token-injector
  selectors:
    podSelector:
      matchLabels:
        app: curl
      matchExpressions:
        - key: app
          operator: In
          values: ["curl"]
    namespaceSelector:
      matchExpressions:
        - key: flomesh.io/monitored-by
          operator: In
          values: ["fsm"]
EOF

plugin chain token-verifier-chain:

kubectl apply -n httpbin -f - <<EOF
kind: PluginChain
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: token-verifier-chain
spec:
  chains:
    - name: inbound-http
      plugins:
        - token-verifier
  selectors:
    podSelector:
      matchLabels:
        app: httpbin
    namespaceSelector:
      matchExpressions:
        - key: flomesh.io/monitored-by
          operator: In
          values: ["fsm"]
EOF

After applying the plugin chain configuration, the plugin chains of the two applications can be viewed now. From the results, we can see the two plugins located in the plugins directory. Our declared plugins have been configured in the two applications through the configuration of the plugin chain.

fsm proxy get config_dump -n curl $curl_pod | jq '.Chains."outbound-http"'
[
  "modules/outbound-http-routing.js",
  "modules/outbound-metrics-http.js",
  "modules/outbound-tracing-http.js",
  "modules/outbound-logging-http.js",
  "modules/outbound-circuit-breaker.js",
  "plugins/token-injector.js",
  "modules/outbound-http-load-balancing.js",
  "modules/outbound-http-default.js"
]

fsm proxy get config_dump -n httpbin $httpbin_pod | jq '.Chains."inbound-http"'
[
  "modules/inbound-tls-termination.js",
  "modules/inbound-http-routing.js",
  "modules/inbound-metrics-http.js",
  "modules/inbound-tracing-http.js",
  "modules/inbound-logging-http.js",
  "modules/inbound-throttle-service.js",
  "modules/inbound-throttle-route.js",
  "plugins/token-verifier.js",
  "modules/inbound-http-load-balancing.js",
  "modules/inbound-http-default.js"
]

After applying the plugin configuration, but we haven’t yet make changes to plugin configuration, the application curl can still access httpbin.

kubectl exec $curl_pod -n curl -c curl -- curl -Is http://httpbin.httpbin:14001/get
HTTP/1.1 200 OK
server: gunicorn/19.9.0
date: Sun, 05 Feb 2023 06:34:33 GMT
content-type: application/json
content-length: 304
access-control-allow-origin: *
access-control-allow-credentials: true
connection: keep-alive

Setting up plugin configuration

We will first apply the configuration of the plugin token-verifier. Here, the authentication service ext-auth.auth:8079 and the request /get that needs to be authenticated are configured.

  • spec.config contains the contents of the plugin configuration, which will be converted to JSON format. For example, the configuration applied to the token-verifier plugin exists in the following JSON form:

    {
  "Plugins": {
    "token-verifier": {
      "Paths": [
        "/get"
      ],
      "Verifier": "ext-auth.auth:8079"
    }
  }
}

kubectl apply -n httpbin -f - <<EOF
kind: PluginConfig
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: token-verifier-config
spec:
  config:
    Verifier: 'ext-auth.auth:8079'
    Paths:
      - "/get"
  plugin: token-verifier
  destinationRefs:
    - kind: Service
      name: httpbin
      namespace: httpbin
EOF

At this time, the application curl cannot access the httbin /get path because a access token has not been configured for curl yet.

kubectl exec $curl_pod -n curl -c curl -- curl -Is http://httpbin.httpbin:14001/get
HTTP/1.1 401 Unauthorized
content-length: 13
connection: keep-alive

But accessing /headers path doesn’t require any authentication.

kubectl exec $curl_pod -n curl -c curl -- curl -Is http://httpbin.httpbin:14001/headers
HTTP/1.1 200 OK
server: gunicorn/19.9.0
date: Sun, 05 Feb 2023 06:37:05 GMT
content-type: application/json
content-length: 217
access-control-allow-origin: *
access-control-allow-credentials: true
connection: keep-alive

Next, the configuration of the plugin token-injector is applied to configure the access token 2f1acc6c3a606b082e5eef5e54414ffb for the application’s requests. This token is also a valid token hardcoded in the authentication service.

kubectl apply -n curl -f - <<EOF
kind: PluginConfig
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: token-injector-config
spec:
  config:
    AccessToken: '2f1acc6c3a606b082e5eef5e54414ffb'
  plugin: token-injector
  destinationRefs:
    - kind: Service
      name: httpbin
      namespace: httpbin
EOF

After applying the configuration for the token-injector plugin, the requests from the curl application will now have the access token 2f1acc6c3a606b082e5eef5e54414ffb configured. As a result, when accessing the /get path of httpbin, the requests will pass authentication and be accepted by httpbin.

kubectl exec $curl_pod -n curl -c curl -- curl -Is http://httpbin.httpbin:14001/get
HTTP/1.1 200 OK
server: gunicorn/19.9.0
date: Sun, 05 Feb 2023 06:39:54 GMT
content-type: application/json
content-length: 360
access-control-allow-origin: *
access-control-allow-credentials: true
connection: keep-alive

2 - HTTP Header Modifier

Modify HTTP Request or Response header via Plugins

In daily network interactions, HTTP headers play a very important role. They can pass various information about the request or response, such as authentication, cache control, content type, etc. This allows users to precisely control incoming and outgoing request and response headers to meet various security, performance and business needs.

FSM does not provide HTTP header control functionality out of box, but we can provide with Plugin Extending feature easily.

Enable plugin policy mode

To utilize plugin to extend mesh, we should enable the plugin policy mode first, because it’s disabled by default.

Execute the command below to enable it.

kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"featureFlags":{"enablePluginPolicy":true}}}' --type=merge

Deploy sample application

Deploy client application:

# Create the curl namespace
kubectl create namespace curl
# Add the namespace to the mesh
fsm namespace add curl
# Deploy curl client in the curl namespace
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/curl.yaml -n curl

Deploy service application:

# Create a namespace
kubectl create ns httpbin
# Add the namespace to the mesh
fsm namespace add httpbin
# Deploy the application
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/httpbin/httpbin.yaml -n httpbin

Check the communication between apps.

curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"
kubectl exec ${curl_client} -n curl -c curl -- curl -s http://httpbin.httpbin:14001/headers

Declaring a plugin

For header modification, we provide a script which can be applied with command bellow.

kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/header-modifier.yaml

Check the plugin declared or not.

kubectl get plugin
NAME              AGE
header-modifier   5s

Setting up plugin-chain

Following the PluginChain API doc, the new plugin works in chain inbound-http and available on resources which is labelled by app=httpbin or app=curl in namespace monitored by mesh.

kubectl apply -f - <<EOF
kind: PluginChain
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: header-modifier-chain
  namespace: pipy
spec:
  chains:
    - name: inbound-http
      plugins:
        - header-modifier
  selectors:
    podSelector:
      matchExpressions:
        - key: app
          operator: In
          values:
            - httpbin
            - curl
    namespaceSelector:
      matchExpressions:
        - key: flomesh.io/monitored-by
          operator: In
          values: ["fsm"]
EOF

Apply plugin configuration

Once applied Plugin and PluginChain, we need to configure it. In the command bellow, we make Service httpbin to modify HTTP headers for request owning header version=v2. When route matched, the plugin will remove header version and add a new header x-canary-tag=v2 from/in request, and add a new header x-carary=true in response.

kubectl apply -f - <<EOF
kind: PluginConfig
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: header-modifier-config
  namespace: httpbin
spec:
  config:
    Matches:
    - Headers:
        Exact:
          version: 'v2'
      Filters: 
      - Type: RequestHeaderModifier
        RequestHeaderModifier:
          Remove:
          - version
          Add:
          - Name: x-canary-tag
            Value: v2
      - Type: ResponseHeaderModifier
        ResponseHeaderModifier:
          Add:
          - Name: x-canary
            Value: 'true'
  plugin: header-modifier
  destinationRefs:
    - kind: Service
      name: httpbin
      namespace: httpbin
EOF

Testing

Let’s send a request again, but attach a header this time.

kubectl exec ${curl_client} -n curl -c curl -- curl -ksi http://httpbin.httpbin:14001/headers -H "version:v2"

In the request which httpbin received, the header version is removed and new header X-Canary-Tag appeared.

In response, we can get the new header x-canary.

HTTP/1.1 200 OK
server: gunicorn
date: Fri, 01 Dec 2023 10:37:14 GMT
content-type: application/json
access-control-allow-origin: *
access-control-allow-credentials: true
x-canary: true
content-length: 211
connection: keep-alive

{
  "headers": {
    "Accept": "*/*",
    "Connection": "keep-alive",
    "Host": "httpbin.httpbin:14001",
    "Serviceidentity": "curl.curl",
    "User-Agent": "curl/8.4.0",
    "X-Canary-Tag": "v2"
  }
}

3 - Fault Injection

Adding Fault Injection functionality to fsm sidecar via Plugins

Fault injection testing is a software testing technique that intentionally introduces errors into a system to verify its ability to handle and bounce back from error conditions. This testing method is usually performed before deployment to identify any possible faults that may have arisen during production. This demo demonstrates on how to implement Fault Injection functionality via a plugin.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have fsm installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.

Deploy demo services


kubectl create namespace curl
fsm namespace add curl
kubectl apply -n curl -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/curl.yaml

kubectl create namespace pipy
fsm namespace add pipy
kubectl apply -n pipy -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/pipy-ok.pipy.yaml

# Wait for pods to be up and ready

sleep 2
kubectl wait --for=condition=ready pod -n curl -l app=curl --timeout=180s
kubectl wait --for=condition=ready pod -n pipy -l app=pipy-ok -l version=v1 --timeout=180s
kubectl wait --for=condition=ready pod -n pipy -l app=pipy-ok -l version=v2 --timeout=180s

Enable plugin policy mode

kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"featureFlags":{"enablePluginPolicy":true}}}' --type=merge

Declaring a plugin

kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/fault-injection.yaml

Setting up plugin-chain

kubectl apply -f - <<EOF
kind: PluginChain
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: http-fault-injection-chain
  namespace: pipy
spec:
  chains:
    - name: inbound-http
      plugins:
        - http-fault-injection
  selectors:
    podSelector:
      matchLabels:
        app: pipy-ok
      matchExpressions:
        - key: app
          operator: In
          values: ["pipy-ok"]
    namespaceSelector:
      matchExpressions:
        - key: flomesh.io/monitored-by
          operator: In
          values: ["fsm"]
EOF

Setting up plugin configuration

In below configuration, we need to have either of delay or abort

kubectl apply -f - <<EOF
kind: PluginConfig
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: http-fault-injection-config
  namespace: pipy
spec:
  config:
    delay:
      percentage:
        value: 0.5
      fixedDelay: 5s
    abort:
      percentage:
        value: 0.5
      httpStatus: 400
  plugin: http-fault-injection
  destinationRefs:
    - kind: Service
      name: pipy-ok
      namespace: pipy
EOF

Test

Use the below command to perform a test

curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"

date;  kubectl exec ${curl_client} -n curl -c curl -- curl -ksi http://pipy-ok.pipy:8080 ;  echo "";  date

Run the above command a few times, and you will see that after a few more visits, and there is approximately a 50% chance of receiving the following result (HTTP status code 400, with a delay of 5 seconds).

Thu Mar 30 06:47:58 UTC 2023
HTTP/1.1 400 Bad Request
content-length: 0
connection: keep-alive


Thu Mar 30 06:48:04 UTC 2023

4 - Traffic Mirroring

Adding Traffic Shadowing functionality to fsm sidecar via Plugins

Traffic mirroring, also known as shadowing, is a technique used to test new versions of an application in a safe and efficient manner. It involves creating a mirrored service that receives a copy of live traffic for testing and troubleshooting purposes. This approach is especially useful for acceptance testing, as it can help identify issues in advance, before they impact end-users.

One of the key benefits of traffic mirroring is that it occurs outside the primary request path for the main service. This means that end-users are not affected by any changes or issues that may occur during the testing process. As such, traffic mirroring is a powerful and low-risk approach for validating new versions of an application.

By using traffic mirroring, you can get valuable insights into how your application will perform in a live environment, without putting your users at risk. This approach can help you identify and address issues quickly and efficiently, which can ultimately improve the overall performance and reliability of your application.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have fsm installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.

Deploy demo services

kubectl create namespace curl
fsm namespace add curl
kubectl apply -n curl -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/curl.yaml

kubectl create namespace pipy
fsm namespace add pipy
kubectl apply -n pipy -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/pipy-ok.pipy.yaml

# Wait for pods to be up and ready

sleep 2
kubectl wait --for=condition=ready pod -n curl -l app=curl --timeout=180s
kubectl wait --for=condition=ready pod -n pipy -l app=pipy-ok -l version=v1 --timeout=180s
kubectl wait --for=condition=ready pod -n pipy -l app=pipy-ok -l version=v2 --timeout=180s

Enable plugin policy mode

kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"featureFlags":{"enablePluginPolicy":true}}}' --type=merge

Declaring a plugin

kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/traffic-mirror.yaml

Setting up plugin-chain

kubectl apply -f - <<EOF
kind: PluginChain
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: traffic-mirror-chain
  namespace: pipy
spec:
  chains:
    - name: outbound-http
      plugins:
        - traffic-mirror
  selectors:
    podSelector:
      matchLabels:
        app: curl
      matchExpressions:
        - key: app
          operator: In
          values: ["curl"]
    namespaceSelector:
      matchExpressions:
        - key: flomesh.io/monitored-by
          operator: In
          values: ["fsm"]
EOF

Setting up plugin configuration

kubectl apply -f - <<EOF
kind: PluginConfig
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: traffic-mirror-config
  namespace: curl
spec:
  config:
    namespace: pipy
    service: pipy-ok-v2
    port: 8080
    percentage:
      value: 1.0
  plugin: traffic-mirror
  destinationRefs:
    - kind: Service
      name: pipy-ok-v1
      namespace: pipy
EOF

Test

Use the below command to perform a test

curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"
kubectl exec ${curl_client} -n curl -c curl -- curl -ksi http://pipy-ok-v1.pipy:8080

Accessing service pipy-ok-v1 should be mirrored to pipy-ok-v2, so we should be seeing access logs in both services.

Testing pipy-ok-v1 logs

pipy_ok_v1="$(kubectl get pod -n pipy -l app=pipy-ok,version=v1 -o jsonpath='{.items[0].metadata.name}')"
kubectl logs pod/${pipy_ok_v1} -n pipy -c pipy

You will see something similar

[2023-03-29 08:41:04 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2023-03-29 08:41:04 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2023-03-29 08:41:04 +0000] [1] [INFO] Using worker: sync
[2023-03-29 08:41:04 +0000] [14] [INFO] Booting worker with pid: 14
127.0.0.6 - - [29/Mar/2023:08:45:35 +0000] "GET / HTTP/1.1" 200 9593 "-" "curl/7.85.0-DEV"

Testing pipy-ok-v2 logs

pipy_ok_v2="$(kubectl get pod -n pipy -l app=pipy-ok,version=v2 -o jsonpath='{.items[0].metadata.name}')"
kubectl logs pod/${pipy_ok_v2} -n pipy -c pipy

You will see something similar

[2023-03-29 08:41:09 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2023-03-29 08:41:09 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2023-03-29 08:41:09 +0000] [1] [INFO] Using worker: sync
[2023-03-29 08:41:09 +0000] [15] [INFO] Booting worker with pid: 15
127.0.0.6 - - [29/Mar/2023:08:45:35 +0000] "GET / HTTP/1.1" 200 9593 "-" "curl/7.85.0-DEV"

5 - Cross-Origin Resource Sharing (CORS)

Adding CORS functionality to FSM sidecar via Plugins

CORS stands for Cross-Origin Resource Sharing. It is a security feature implemented by web browsers to prevent web pages from making requests to a different domain than the one that served the web page.

The same-origin policy is a security feature that allows web pages to access resources only from the same origin, which includes the same domain, protocol, and port number. This policy is designed to protect users from malicious scripts that can steal sensitive data from other websites.

CORS allows web pages to make cross-origin requests by adding specific headers to the HTTP response from the server. The headers indicate which domains are allowed to make cross-origin requests, and what type of requests are allowed.

However, configuring CORS can be challenging, especially when dealing with complex web applications that involve multiple domains and servers. One way to simplify the CORS configuration is to use a proxy server.

A proxy server acts as an intermediary between the web application and the server. The web application sends requests to the proxy server, which then forwards the requests to the server. The server responds to the proxy server, which then sends the response back to the web application.

By using a proxy server, you can configure the CORS headers on the proxy server instead of configuring them on the server that serves the web page. This way, the web application can make cross-origin requests without violating the same-origin policy.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have fsm installed.
  • Have kubectl available to interact with the API server.
  • Have fsm CLI available for managing the service mesh.

Deploy demo services

kubectl create namespace curl
fsm namespace add curl
kubectl apply -n curl -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/curl.yaml

kubectl create namespace pipy
fsm namespace add pipy
kubectl apply -n pipy -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/pipy-ok.pipy.yaml

# Wait for pods to be up and ready

sleep 2
kubectl wait --for=condition=ready pod -n curl -l app=curl --timeout=180s
kubectl wait --for=condition=ready pod -n pipy -l app=pipy-ok -l version=v1 --timeout=180s
kubectl wait --for=condition=ready pod -n pipy -l app=pipy-ok -l version=v2 --timeout=180s

Enable plugin policy mode

kubectl patch meshconfig fsm-mesh-config -n "$fsm_namespace" -p '{"spec":{"featureFlags":{"enablePluginPolicy":true}}}' --type=merge

Declaring a plugin

kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/plugins/cors.yaml

Setting up plugin-chain

kubectl apply -f - <<EOF
kind: PluginChain
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: cors-policy-chain
  namespace: pipy
spec:
  chains:
    - name: inbound-http
      plugins:
        - cors-policy
  selectors:
    podSelector:
      matchLabels:
        app: pipy-ok
      matchExpressions:
        - key: app
          operator: In
          values: ["pipy-ok"]
    namespaceSelector:
      matchExpressions:
        - key: flomesh.io/monitored-by
          operator: In
          values: ["fsm"]
EOF

Setting up plugin configuration

kubectl apply -f - <<EOF
kind: PluginConfig
apiVersion: plugin.flomesh.io/v1alpha1
metadata:
  name: cors-policy-config
  namespace: pipy
spec:
  config:
    allowCredentials: true
    allowHeaders:
    - X-Foo-Bar-1
    allowMethods:
    - POST
    - GET
    - PATCH
    - DELETE
    allowOrigins:
    - regex: http.*://www.test.cn
    - exact: http://www.aaa.com
    - prefix: http://www.bbb.com
    exposeHeaders:
    - Content-Encoding
    - Kuma-Revision
    maxAge: 24h
  plugin: cors-policy
  destinationRefs:
    - kind: Service
      name: pipy-ok
      namespace: pipy
EOF

Test

Use the below command to perform a test

curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"
kubectl exec ${curl_client} -n curl -c curl -- curl -ksi http://pipy-ok.pipy:8080 -H "Origin: http://www.bbb.com"

You will see response similar to:

HTTP/1.1 200 OK
access-control-allow-credentials: true
access-control-expose-headers: Content-Encoding,Kuma-Revision
access-control-allow-origin: http://www.bbb.com
content-length: 20
connection: keep-alive

Hi, I am PIPY-OK v1!

Run another command to perform a test

curl_client="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].metadata.name}')"
kubectl exec ${curl_client} -n curl -c curl -- curl -ksi http://pipy-ok.pipy:8080 -H "Origin: http://www.bbb.com" -X OPTIONS

You will see response similar to:

HTTP/1.1 200 OK
access-control-allow-origin: http://www.bbb.com
access-control-allow-credentials: true
access-control-expose-headers: Content-Encoding,Kuma-Revision
access-control-allow-methods: POST,GET,PATCH,DELETE
access-control-allow-headers: X-Foo-Bar-1
access-control-max-age: 86400
content-length: 0
connection: keep-alive