This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

FSM quick start

Try FSM in 5 minutes

This guide shows how to download, install, and run FSM, deploy a demo application, and complete SMI standard functionality like link encryption, access control, and traffic splitting in less than 5 minutes. This demo assumes you are running Ubuntu 21 on x86 architecture, running the k3s version ‘V1.23.8 + K3S1’. For more version and platform support, please refer to the complete Beginner’s Guide.

Pre-requisites

Running Kubernetes cluster. If you don’t have one, you can use below script to install k3s:

export INSTALL_K3S_VERSION=v1.23.8+k3s1
curl -sfL https://get.k3s.io | sh -s - --disable traefik --write-kubeconfig-mode 644 --write-kubeconfig ~/.kube/config

Minimum Kubernetes version supported by FSM is v1.19.0

Download and install FSM CLI

GNU/Linux

Download the 64-bit GNU/Linux or macOS binary of FSM v1.3.3:

system=$(uname -s | tr '[:upper:]' '[:lower:]')
arch=$(uname -m | sed -E 's/x86_/amd/' | sed -E 's/aarch/arm/')
release=v1.3.3
curl -L https://github.com/flomesh-io/fsm/releases/download/${release}/fsm-${release}-${system}-${arch}.tar.gz | tar -vxzf -
./${system}-${arch}/fsm version
cp ./${system}-${arch}/fsm /usr/local/bin/

macOS

Download the 64-bit macOS binaries for FSM v1.3.3

system=$(uname -s | tr "[:upper:]" "[:lower:]")
arch=$(uname -m)
release=v1.3.3
curl -L https://github.com/flomesh-io/fsm/releases/download/$release/fsm-$release-$system-$arch.tar.gz | tar -vxzf -
./$system-$arch/fsm version
cp ./$system-$arch/fsm /usr/local/bin/

Install FSM on Kubernetes cluster

Below command also installs and enable Prometheus, Grafana and Jaeger.

Read more on FSM’s integrations with Prometheus, Grafana, and Jaeger in the observability documentation.

export fsm_namespace=fsm-system 
export fsm_mesh_name=fsm 

fsm install \
    --mesh-name "$fsm_mesh_name" \
    --fsm-namespace "$fsm_namespace" \
    --set=fsm.enablePermissiveTrafficPolicy=true \
    --set=fsm.deployPrometheus=true \
    --set=fsm.deployGrafana=true \
    --set=fsm.deployJaeger=true \
    --set=fsm.tracing.enable=true

Deploy Applications

In this section we will deploy 5 different Pods, and we will apply policies to control the traffic between them.

  • bookbuyer is an HTTP client making requests to bookstore. This traffic is permitted.
  • bookthief is an HTTP client and much like bookbuyer also makes HTTP requests to bookstore. This traffic should be blocked.
  • bookstore is a server, which responds to HTTP requests. It is also a client making requests to the bookwarehouse service. This traffic is permitted.
  • bookwarehouse is a server and should respond only to bookstore. Both bookbuyer and bookthief should be blocked.
  • mysql is a MySQL database only reachable by bookwarehouse.

Use below script to install:

kubectl create namespace bookstore
kubectl create namespace bookbuyer
kubectl create namespace bookthief
kubectl create namespace bookwarehouse
fsm namespace add bookstore bookbuyer bookthief bookwarehouse
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/bookbuyer.yaml
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/bookthief.yaml
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/bookstore.yaml
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/bookwarehouse.yaml
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/mysql.yaml

Expose the GUI ports of each service, so that with a browser we can access these ports of demo application.

git clone https://github.com/flomesh-io/fsm.git -b main
cd fsm
cp .env.example .env
./scripts/port-forward-all.sh

In a browser, open the following URL.

_Note: If you need to access from the host, you need to replace localhost with the IP address of the virtual machine; or run the port-forward-all.sh script on the host. _

Access Control

By installing FSM with the above command, all services are without access control (permissive traffic policy mode), or all access is allowed. The situation when there is no access control can be seen by looking at the growth in the number of books counts per service in the browser.

The counts in the bookbuyer, bookthief UI correspond to the number of books purchased and stolen, respectively, while in bookstore-v1 these should be increasing by.

The count for book sales in the bookstore UI should also be increasing.

The following demonstrates denying access to the bookstore service by disabling the permissive traffic policy mode.

kubectl patch meshconfig fsm-mesh-config -n fsm-system -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":false}}}'  --type=merge

You will see that the count is no longer increasing.

Execute below command to allow bookbuyer privileges to access bookstore:

kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/access/traffic-access-v1.yaml

Here we go back to the bookbuyer and bookstore UI and see that the count resumes increasing while the count for the bookthief UI remains stopped.

With access control, we have successfully prevented bookthief from stealing books from bookstore, while normal purchases are unaffected.

Observability

Metrics

Use below command to enable namespace metrics generation and capturing, or else metrics generated by Pods won’t be gathered.

fsm metrics enable --namespace "bookstore,bookbuyer,bookthief,bookwarehouse"

After running port-forwarding script, open url http://localhost:3000 in browser to access Grafan console. Dashboard default username and passwords are admin, admin.

FSM has several built-in dashboards to provide visualization of metrics in the control plane and data plane. For example, the following figure shows the metrics of pod http://localhost:3000 of the bookthief service accessing other services.

image

The following figure shows the metrics of bookthief accessing other services at the granularity of deployment. The difference from the previous figure is that if bookthief has multiple replicas, the aggregate data for all replicas is shown here: !

image

The next metrics for the FSM component, and for the mesh base information are shown here.

image

Tracing

Jaeger’s dashboard can be accessed by typing http://localhost:16686/search in your browser: !

image

The dashboard allows you to look up service-related tracing information: !

image

Show service topology diagram.

image

Logging

The FSM control plane outputs diagnostic logs to the standard output for service mesh management, and the output of logging information can be controlled by adjusting the level of logging. The logs output to the standard output can be aggregated and stored by the log collection tool.

Uninstall Service Mesh

To uninstall all resources associated with FSM after completing the quick experience with FSM, you will need to delete these sample applications and associated SMI resources and uninstall the FSM control plane and cluster-wide FSM resources.

To delete the sample applications.

kubectl delete ns bookbuyer bookthief bookstore bookwarehouse

Uninstall the control plane.

fsm uninstall mesh

1 - eBPF setup quickstart

Learn how to use eBPF with FSM

This quick start guide demonstrates how to setup environment and configure FSM to use eBPF as its interception method.

For more details refer to Learn how to use eBPF with FSM

Prerequisites

  • Ubuntu 20.04
  • Kernel 5.15.0-1034 (5.4 or higher)
  • 2c4g VM * 3:master、node1、node2

Install CNI Plugin

Execute the following command on all nodes to download the CNI plugin.

sudo mkdir -p /opt/cni/bin
curl -sSL https://github.com/containernetworking/plugins/releases/download/v1.1.1/cni-plugins-linux-amd64-v1.1.1.tgz | sudo tar -zxf - -C /opt/cni/bin

Master Node

Get the IP address of the master node. (Your machine IP might be different)

export MASTER_IP=10.0.2.6

Kubernetes cluster uses the k3s distribution, but when installing the cluster, you need to disable the flannel integrated by k3s and use independently installed flannel for validation. This is because k3s’s doesn’t follow Flannel directory structure /opt/cni/bin and store its CNI bin directory at /var/lib/rancher/k3s/data/xxx/bin where xxx is some randomly generated text.

export INSTALL_K3S_VERSION=v1.23.8+k3s2
curl -sfL https://get.k3s.io | sh -s - --disable traefik --disable servicelb --flannel-backend=none --advertise-address $MASTER_IP --write-kubeconfig-mode 644 --write-kubeconfig ~/.kube/config

Install Flannel. Note that the default Pod CIDR of Flannel is 10.244.0.0/16, and we will modify it to k3s’s default 10.42.0.0/16.

curl -s https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml | sed 's|10.244.0.0/16|10.42.0.0/16|g' | kubectl apply -f -

Get the access token of the API server for initializing worker nodes.

sudo cat /var/lib/rancher/k3s/server/node-token

Worker Node

Use the IP address of the master node and the token obtained earlier to initialize the node.

export INSTALL_K3S_VERSION=v1.23.8+k3s2
export NODE_TOKEN=K107c1890ae060d191d347504740566f9c506b95ea908ba4795a7a82ea2c816e5dc::server:2757787ec4f9975ab46b5beadda446b7
curl -sfL https://get.k3s.io | K3S_URL=https://${MASTER_IP}:6443 K3S_TOKEN=${NODE_TOKEN} sh -

Download FSM CLI

system=$(uname -s | tr [:upper:] [:lower:])
arch=$(dpkg --print-architecture)
release=v1.3.3
curl -L https://github.com/flomesh-io/fsm/releases/download/${release}/fsm-${release}-${system}-${arch}.tar.gz | tar -vxzf -
./${system}-${arch}/fsm version
sudo cp ./${system}-${arch}/fsm /usr/local/bin/

Install FSM

export fsm_namespace=fsm-system 
export fsm_mesh_name=fsm 

fsm install \
    --mesh-name "$fsm_mesh_name" \
    --fsm-namespace "$fsm_namespace" \
    --set=fsm.trafficInterceptionMode=ebpf \
    --timeout=900s

Deploy Sample Application

Before execute commands bellow, please update the node name “node1”, “node2” with the real node names in kubectl patch commands.

#Sample services
kubectl create namespace ebpf
fsm namespace add ebpf

kubectl apply -n ebpf -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/interceptor/curl.yaml
kubectl apply -n ebpf -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/samples/interceptor/pipy-ok.yaml

#Schedule Pods to Different Nodes
kubectl patch deployments curl -n ebpf -p '{"spec":{"template":{"spec":{"nodeName":"node1"}}}}'
kubectl patch deployments pipy-ok-v1 -n ebpf -p '{"spec":{"template":{"spec":{"nodeName":"node1"}}}}'
kubectl patch deployments pipy-ok-v2 -n ebpf -p '{"spec":{"template":{"spec":{"nodeName":"node2"}}}}'

sleep 5

#Wait for dependent Pods to start successfully
kubectl wait --for=condition=ready pod -n ebpf -l app=curl --timeout=180s
kubectl wait --for=condition=ready pod -n ebpf -l app=pipy-ok -l version=v1 --timeout=180s
kubectl wait --for=condition=ready pod -n ebpf -l app=pipy-ok -l version=v2 --timeout=180s

Testing

During testing, you can view the debug logs of BPF program execution by viewing the kernel tracing logs on the worker node using the following command. To avoid interference caused by sidecar communication with the control plane, first obtain the IP address of the control plane.

kubectl get svc -n fsm-system fsm-controller -o jsonpath='{.spec.clusterIP}'
10.43.241.189

Execute the following command on both worker nodes.

sudo cat /sys/kernel/debug/tracing/trace_pipe | grep bpf_trace_printk | grep -v '10.43.241.189'

Execute the following command on both worker nodes.

curl_client="$(kubectl get pod -n ebpf -l app=curl -o jsonpath='{.items[0].metadata.name}')"
kubectl exec ${curl_client} -n ebpf -c curl -- curl -s pipy-ok:8080

You should receive results similar to the following, and the kernel tracing logs should also output the debug logs of the BPF program accordingly (the content is quite long, so it will not be shown here).

Hi, I am pipy ok v1 !
Hi, I am pipy ok v2 !

2 - Quick Start on OpenShift

How to quickly setup FSM on Redhat OpenShift

If you already have running OpenShift platform, follow these steps to get started quickly.

Prerequisites

Helm Chart Repository

OpenShift starting with version 4.8 comes with Helm Chart Repository installed, and if you are using an older version of OpenShift, you can add the repo via helm command:

# curl -L https://mirror.openshift.com/pub/openshift-v4/clients/helm/latest/helm-linux-amd64 -o /usr/local/bin/helm

# chmod +x /usr/local/bin/helm

helm repo add openshift-helm-charts https://charts.openshift.io/

To install the repo to be used from the OpenShift console run the following command as and OpenShift admin:

oc apply -f https://charts.openshift.io/openshift-charts-repo.yaml

The Permissions

The account to deploy the charts need to be bound to clusterrole cluster-admin, you could achieve by the command:

oc adm policy add-cluster-role-to-user cluster-admin <your OCP account name>

Installation

Install via Helm Cli

Run the following helm command to install FSM

helm install \
        --devel \
        --namespace fsm-system \
        --create-namespace \
        --set=fsm.controllerLogLevel=warn \
        FSM \
        openshift-helm-charts/flomesh-FSM

Install via OpenShift Console

  1. Create project named fsm-system

  2. Find and search FSM in the Developer Catalog

  3. Before starting install the chart, switch configuration style to YAML view, as the JSON Schema version is much newer than OpenShift console supported version. You could change the configs as well if needed.

  4. Click Install to start installation and wait the pods to be ready.

Deploy Applications

In this section we will deploy 5 different Pods, and we will apply policies to control the traffic between them.

  • bookbuyer is an HTTP client making requests to bookstore. This traffic is permitted.
  • bookthief is an HTTP client and much like bookbuyer also makes HTTP requests to bookstore. This traffic should be blocked.
  • bookstore is a server, which responds to HTTP requests. It is also a client making requests to the bookwarehouse service. This traffic is permitted.
  • bookwarehouse is a server and should respond only to bookstore. Both bookbuyer and bookthief should be blocked.
  • mysql is a MySQL database only reachable by bookwarehouse.

Use below script to install:

kubectl create namespace bookstore
kubectl create namespace bookbuyer
kubectl create namespace bookthief
kubectl create namespace bookwarehouse
fsm namespace add bookstore bookbuyer bookthief bookwarehouse
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/bookbuyer.yaml
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/bookthief.yaml
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/bookstore.yaml
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/bookwarehouse.yaml
kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/apps/mysql.yaml

Expose the GUI ports of each service, so that with a browser we can access these ports of demo application.

git clone https://github.com/flomesh-io/fsm.git -b main
cd FSM
cp .env.example .env
./scripts/port-forward-all.sh #可以忽略错误信息

In a browser, open the following URL.

_Note: If you need to access from the host, you need to replace localhost with the IP address of the virtual machine; or run the port-forward-all.sh script on the host. _

Access Control

By installing FSM with the above command, all services are without access control (permissive traffic policy mode), or all access is allowed. The situation when there is no access control can be seen by looking at the growth in the number of books counts per service in the browser.

The counts in the bookbuyer, bookthief UI correspond to the number of books purchased and stolen, respectively, while in bookstore-v1 these should be increasing by.

The count for book sales in the bookstore UI should also be increasing.

The following demonstrates denying access to the bookstore service by disabling the permissive traffic policy mode.

kubectl patch meshconfig fsm-mesh-config -n fsm-system -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":false}}}'  --type=merge

You will see that the count is no longer increasing.

Execute below command to allow bookbuyer privileges to access bookstore:

kubectl apply -f https://raw.githubusercontent.com/flomesh-io/fsm-docs/main/manifests/access/traffic-access-v1.yaml

Here we go back to the bookbuyer and bookstore UI and see that the count resumes increasing while the count for the bookthief UI remains stopped.

With access control, we have successfully prevented bookthief from stealing books from bookstore, while normal purchases are unaffected.

Observability

Metrics

Use below command to enable namespace metrics generation and capturing, or else metrics generated by Pods won’t be gathered.

fsm metrics enable --namespace "bookstore,bookbuyer,bookthief,bookwarehouse"

After running port-forwarding script, open url http://localhost:3000 in browser to access Grafan console. Dashboard default username and passwords are admin, admin.

FSM has several built-in dashboards to provide visualization of metrics in the control plane and data plane. For example, the following figure shows the metrics of pod http://localhost:3000 of the bookthief service accessing other services.

image

The following figure shows the metrics of bookthief accessing other services at the granularity of deployment. The difference from the previous figure is that if bookthief has multiple replicas, the aggregate data for all replicas is shown here: !

image

The next metrics for the FSM component, and for the mesh base information are shown here.

image

Tracing

Jaeger’s dashboard can be accessed by typing http://localhost:16686/search in your browser: !

image

The dashboard allows you to look up service-related tracing information: !

image

Show service topology diagram.

image

Logging

The FSM control plane outputs diagnostic logs to the standard output for service mesh management, and the output of logging information can be controlled by adjusting the level of logging. The logs output to the standard output can be aggregated and stored by the log collection tool.