【Envoy-01】Architecture Overview and Fundamentals

I’ve been re-learning Envoy recently since it’s a powerful L4/L7 proxy widely used in multiple opensource projects(e.g. Istio, Cilium, Envoy Gateway). Back in 2019, I first got to know Service Mesh, the first Service Mesh opensource project I got involve is Istio, already using Envoy as L4/L7 proxy for traffic management. At that time, I wasn’t interested in Envoy presumably because it’s written in C++, which is considered as a ‘The deeper you get, the harder it gets’ language.

Fast-forward to 2023, the Gateway API gradually became GA, Cilium graduated from CNCF, more and more opensource projects use Envoy as network proxy(at lease use it as L7 proxy). Besides, thanks to the WASM extension support, users can easily extend their own traffic policy without breaking the Envoy code itself. In foreseeable future, Envoy will be commonly used to handle inbound/outbound traffic for all services in the community.

This series of Envoy learning articles are trying to explain why envoy is so powerful and what we need to know about Envoy.

1. What can Envoy do

Cloud Native L4/L7 Proxy
- manageable, observable
Extendability
- extendable, easily adapt and extend
Control via API
- Initially config by config file
- Mostly controlled by central control plane
Observability
- tracing/metric
- access log

2. Core Concept of Envoy

Listener
- listen on a specific port
- accepting incoming connection and do something with them(send them to filters defined by users)
Cluster
- upstream/destination of the traffic
- composed of endpoint and cluster member
Endpoint/Cluster Member/Cluster Load Assigment
- ip addresses and ports
Routes
- decides which cluster/endpoint to send data based on the http property
Filters
- decides what to do with the data(e.g. add/remove a header)

3. Flow of the Data

Downstream Client sends data to the Envoy proxy
Listener listens a specific port and receive the data, sends it to a chain of filters
Filters modify the data based on the configuration and make routing decision
Send the data to upstream Cluster/Endpoint

How the routing decision is done

TCP filters
- read bytes, modify bytes and write bytes in TCP level
HCM(HTTP Connection Manager) filters (end of tcp filter)
- convert bytes into http requests with headers and body(HTTP connection manager filter)
- compare the headers to the routing rules and select the destination(call r oute filter)
HTTP filters
- read bytes, modify bytes and write bytes in HTTP level
Route filters (end of http filter)
- send the request to upstream

4. Example

TCP Filters

Here is an example of Envoy TCP proxy configuration down below:

# https://github.com/solo-io/hoot/blob/master/01-intro/simple_tcp.yaml
admin:
  access_log_path: /dev/stdout
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address: { address: 0.0.0.0, port_value: 10000 }
      filter_chains:
        - filters:
            - name: envoy.filters.network.tcp_proxy
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
                stat_prefix: ingress_tcp
                cluster: somecluster
  clusters:
    - name: somecluster
      connect_timeout: 0.25s
      type: STRICT_DNS
      lb_policy: ROUND_ROBIN
      hosts: [{ socket_address: { address: 127.0.0.1, port_value: 8082 }}]

a list of listeners
- the ports envoy listens on
- a list of tcp filters(only has one in example, which is a tcp_proxy filter, configured to send traffic to the cluster somecluster)
a list of clusters
- the destination of the traffic

HTTP Filters

# https://github.com/solo-io/hoot/blob/master/01-intro/simple.yaml
admin:
  access_log_path: /dev/stdout
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address: { address: 0.0.0.0, port_value: 10000 }
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: namespace.local_service
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/" }
                          route: { cluster: somecluster }
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
    - name: somecluster
      connect_timeout: 0.25s
      type: STRICT_DNS
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: somecluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 8082

a list of listeners
- the ports envoy listens on
- a list of http filters(a http_connection_manager tcp filter uses router http filter, configured to send traffic with / prefix to the cluster somecluster)
a list of clusters
- the destination of the traffic
- a list of endpoints send to specific upstream socket

【Envoy-01】Architecture Overview and Fundamentals

1. What can Envoy do

2. Core Concept of Envoy

3. Flow of the Data

How the routing decision is done

4. Example

TCP Filters

HTTP Filters

5. Reference

CATALOG

FEATURED TAGS

FRIENDS