Egress

Enable access to the Internet and services external to the service mesh.

1: Egress
2: Egress Gateway

1 - Egress

Enable access to the Internet and services external to the service mesh.

Allowing access to the Internet and out-of-mesh services (Egress)

This document describes the steps required to enable access to the Internet and services external to the service mesh, referred to as Egress traffic.

FSM redirects all outbound traffic from a pod within the mesh to the pod’s sidecar proxy. Outbound traffic can be classified into two categories:

Traffic to services within the mesh cluster, referred to as in-mesh traffic
Traffic to services external to the mesh cluster, referred to as egress traffic

While in-mesh traffic is routed based on L7 traffic policies, egress traffic is routed differently and is not subject to in-mesh traffic policies. FSM supports access to external services as a passthrough without subjecting such traffic to filtering policies.

Configuring Egress

There are two mechanisms to configure Egress:

Using the Egress policy API: to provide fine grained access control over external traffic
Using the mesh-wide global egress passthrough setting: the setting is toggled on or off and affects all pods in the mesh, enabling which allows traffic destined to destinations outside the mesh to egress the pod.

1. Configuring Egress policies

FSM supports configuring fine grained policies for traffic destined to external endpoints using its Egress policy API. To use this feature, enable it if not enabled:

# Replace fsm-system with the namespace where FSM is installed
kubectl patch meshconfig fsm-mesh-config -n fsm-system -p '{"spec":{"featureFlags":{"enableEgressPolicy":true},"traffic":{"enableEgress":false}}}' --type=merge

Remember to disable egress passthrough with set traffic.enableEgress: false.

Refer to the Egress policy demo and API documentation on how to configure policies for routing egress traffic for various protocols.

2. Configuring mesh-wide Egress passthrough

Enabling mesh-wide Egress passthrough to external destinations

Egress can be enabled mesh-wide during FSM install or post install. When egress is enabled mesh-wide, outbound traffic from pods are allowed to egress the pod as long as the traffic does not match in-mesh traffic policies that otherwise deny the traffic.

During FSM installation, the egress feature is enabled by default. You can disabled via options as below.
```
fsm install --set fsm.enableEgress=false
```
After FSM has been installed:
fsm-controller retrieves the egress configuration from the fsm-mesh-config MeshConfig custom resource in the fsm mesh control plane namespace (fsm-system by default). Use kubectl patch to set enableEgress to true in the fsm-mesh-config resource.
```
# Replace fsm-system with the namespace where FSM is installed
kubectl patch meshconfig fsm-mesh-config -n fsm-system -p '{"spec":{"traffic":{"enableEgress":true}}}' --type=merge
```
With kubectl patching, it could be disabled too.

Disabling mesh-wide Egress passthrough to external destinations

Similar to enabling egress, mesh-wide egress can be disabled during FSM install or post install.

During FSM install:

fsm install --set fsm.enableEgress=false

After FSM has been installed: Use kubectl patch to set enableEgress to false in the fsm-mesh-config resource.

# Replace fsm-system with the namespace where FSM is installed
kubectl patch meshconfig fsm-mesh-config -n fsm-system -p '{"spec":{"traffic":{"enableEgress":false}}}'  --type=merge

With egress disabled, traffic from pods within the mesh will not be able to access external services outside the cluster.

How it works

When egress is enabled mesh-wide, FSM controller programs every Pipy proxy sidecar in the mesh with a wildcard rule that matches outbound destinations that do not correspond to in-mesh services. The wildcard rule that matches such external traffic simply proxies the traffic as is to its original destination without subjecting them to L4 or L7 traffic policies.

FSM supports egress for traffic that uses TCP as the underlying transport. This includes raw TCP traffic, HTTP, HTTPS, gRPC etc.

Since mesh-wide egress is a global setting and operates as a passthrough to unknown destinations, fine grained access control (such as applying TCP or HTTP routing policies) over egress traffic is not possible.

Refer to the Egress passthrough demo to learn more.

Pipy configurations

When egress is enabled globally in the mesh, the FSM controller issues the following configuration for each Pipy proxy sidecar.

{
  "Spec": {
    "SidecarLogLevel": "error",
    "Traffic": {
      "EnableEgress": true
    }
  }
}

The Pipy script for EnableEgress=true will use the original destination logic to route the request to proxy it to the original destination.

2 - Egress Gateway

Mannage access to the Internet and services external to the service mesh with Egress gateway.

Egress Gateway

Egress gateway is another approach to manage access to services external to the service mesh.

In this mode, the sidecar forwards egress traffic to the Egress gateway, and Egress gateway completes the forwarding to external services.

Using an Egress Gateway provides unified egress management, although it is an extra hop from the network perspective. The security team can set network rules on a fixed device to allow access to external services. The node selector is then used when the egress gateway is dispatched to these devices. Both approaches have their advantages and disadvantages and need to be chosen based on specific scenarios.

Configuration Egress Gateway

Egress gateway also supports the enable and disable mesh-wide passthrough, you can refer to configuration section of Egress.

First of all, it’s required to deploy the egress gateway. Refer to Egress Gateway Demo for egress gateway installation.

Once we have the gateway, we need to add a global egress policy. The spec of EgressGateway declares that egress traffic can be forwarded to the Service global-egress-gateway under the namespace egress-gateway.

kind: EgressGateway
apiVersion: policy.flomesh.io/v1alpha1
metadata:
  name: global-egress-gateway
  namespace: fsm
spec:
  global:
    - service: fsm-egress-gateway
      namespace: fsm

global-egress-gateway created above is a global egress gateway. By default, all egress traffic will be redirected to this global egress gateway by sidecar.

More configuration for Egress gateway

As we know, the sidecar will forward egress traffic to egress gateway and the latter one will complete the forwarding to services external to mesh.

The transmission between sidecar and egress gateway has two modes: http2tunnel and socks5. This can be set during the deployment of egress gateway and it will use http2tunnel if omitted.

Demo

To learn more about configuration for egress gateway, refer to following demo guides: