Hello everyone, I am Lou Tsai.

I found that many of the recent articles published by the main account of the gateway have uneven quality. It is recommended to read this article directly. It has theory and actual combat.

Not BB, on the article directory:

1 API Gateway Basics

1.1 What is API Gateway?

The API gateway is a server and the only entrance to the system. From an object-oriented design perspective, it is similar to the Facade pattern.

The API gateway encapsulates the internal architecture of the system and provides a customized API for each client. It may also have other responsibilities, such as authentication, monitoring, load balancing, caching, protocol conversion, throttling and circuit breaking, and static response handling.

The core point of the API gateway method is that all clients and consumers access microservices through a unified gateway, and all non-business functions are processed at the gateway layer. Usually, the gateway is also an access API that provides REST/HTTP.

1.2 Main functions of the gateway

As the unified entrance of microservice backend services, the microservice gateway can coordinate and manage backend services, mainly divided into data plane and control plane:

• The main function of the data plane is to access the user's HTTP request and aggregate after the microservices are split. Using the microservice gateway to expose the API and contract of the backend service uniformly, the routing and filtering functions are the core capability modules of the gateway. In addition, the microservice gateway can implement the interception mechanism and focus on cross-cutting functions, including protocol conversion, security authentication, circuit breaking and current limiting, grayscale release, log management, traffic monitoring, etc.
• The main function of the control plane is to perform unified control and configuration management on back-end services. For example, the elastic scaling of the gateway can be controlled; the configuration can be delivered uniformly; tags can be added to the gateway service; the API contract of the backend service can be exposed to the user by configuring the Swagger function on the microservice gateway to complete the document service and improve Work efficiency and reduce communication costs.

• Routing function: Routing is the core capability of the microservice gateway. The microservice gateway can forward the request to the target microservice through the routing function. In the microservice architecture, the gateway can combine the dynamic service discovery of the registration center to realize the discovery of the backend service. The caller only needs to know the service API exposed by the gateway to transparently access the backend microservice.
• Load balancing: API gateway combines load balancing technology, uses service discovery tools such as Eureka or Consul, and realizes load balancing of downstream services through mechanisms such as polling, specified weight, and IP address hashing.
• Unified authentication: Generally speaking, user identity authentication is required for both internal and external network interfaces, and user authentication will use a unified single sign-on (Single Sign On) system in some large-scale systems. Each microservice needs to connect to the single sign-on system, which is obviously a waste of resources and low development efficiency. The API gateway is an excellent place for unified management of security. It can extract the authentication part to the gateway layer. The microservice system does not need to pay attention to the logic of authentication, but only to focus on its own business.
• Protocol conversion: A major role of the API gateway is to build heterogeneous systems. The API gateway serves as a single entry point and integrates microservices based on different styles and implementation technologies such as REST, AMQP, and Dubbo through protocol conversion. It is oriented to Web Mobile, open platforms, etc. The client provides unified services.
• Indicator monitoring: The gateway can count the number of back-end service requests, and can update the current traffic health status in real time. It can perform delay statistics on services at the URL granularity, and can also use the Hystrix Dashboard to check the traffic status of the back-end service and whether there is a fuse occur.
• Current-limiting fuse: In some scenarios, it is necessary to control the number and frequency of client visits, and some high-concurrency systems sometimes require limited flow. A threshold can be configured on the gateway. When the number of requests exceeds the threshold, an error will be returned directly without continuing to access the background service. When there is a traffic flood or when the back-end service is delayed or fails, the gateway can take the initiative to fuse, protect the back-end service, and keep the front-end user experience good.
• Black and white list: The microservice gateway can use the system blacklist to filter HTTP request characteristics and intercept abnormal client requests, such as DDoS attacks and other behaviors that erode bandwidth or force services to be interrupted, and can be intercepted and filtered at the gateway level. A more common interception strategy is to add blacklists based on IP addresses. In routing services with authentication management, you can skip authentication management by setting a whitelist and directly access backend service resources.
• Grayscale release: The microservice gateway can perform flow control according to the special tags in the HTTP request and the metadata identifier of the backend service list, so as to realize the grayscale release without the user's perception.
• Traffic coloring: Similar to the principle of gray release, the gateway can color the request according to the host, head, agent and other identifiers of the HTTP request. With the traffic coloring function of the gateway, we can track the subsequent call links of the service and Service latency and service operation status for further link analysis.
• Documentation Center: The gateway combines Swagger to expose the backend microservices to the gateway. The gateway serves as a unified entrance to provide API specifications for viewing backend services to users of the interface. It is not necessary to know the Swagger address of each backend microservice. The gateway has the effect of aggregating the back-end API.
• Log auditing: The microservice gateway can act as a unified log recorder and collector to intercept log request information and response information at the service URL granularity.

2 API gateway selection

2.1 Commonly used API gateways

Let’s take a brief look at the commonly used API gateways on the market:

nginx

Nginx is a high-performance HTTP and reverse proxy server. On the one hand, Nginx can be used as a reverse proxy, and on the other hand, it can be used as a static resource server. The interface uses Lua dynamic language to complete flexible customization functions.

After Nginx starts, there will be a Master process and multiple Worker processes. The Master process and Worker processes interact through inter-process communication, as shown in the figure. The blocking point of the Worker worker process is at I/O multiplexing function calls like select(), epoll_wait(), etc., to wait for data read/write events to occur. Nginx uses an asynchronous non-blocking method to process requests, that is to say, Nginx can process thousands of requests at the same time.

Zuul

Zuul is an API gateway component open sourced by Netflix, which can be used in conjunction with Eureka, Ribbon, Hystrix and other components. The community is active and integrated into the complete ecology of SpringCloud. It is one of the best options for building front-end gateway services for micro-service systems.

The core of Zuul is a series of filters that can perform the following functions:

• Unified authentication + dynamic routing + load balancing + stress testing
• Auditing and Monitoring: Track meaningful data and statistics with edge locations, resulting in an accurate view of production.
• Multi-regional elasticity: request routing across AWS Regions, aiming at diversifying the use of ELB (Elastic Load Balancing, Elastic Load Balancing) and making the edge of the system closer to the users of the system.

Zuul currently has two major versions: Zuul1 and Zuul2

Zuul1 is built based on the Servlet framework. As shown in the figure, it adopts blocking and multi-threading methods, that is, one thread processes a connection request. This method will cause more surviving connections and A thread increase occurs.

Zuul2 released by Netflix has a major update. It runs on an asynchronous and non-blocking framework. Each CPU core has one thread to handle all requests and responses. The life cycle of requests and responses is handled through events and callbacks. way reduces the number of threads, so the overhead is less.

Spring Cloud GetWay

Spring Cloud Gateway is a brand new API gateway project of Spring Cloud, the purpose is to replace Zuul1, it is based on Spring5.0 + SpringBoot2.0 + WebFlux (based on high-performance Reactor mode responsive communication framework Netty, asynchronous non-blocking model) And other technical development, the performance is higher than Zuul, the official test, Spring Cloud GateWay is 1.6 times that of Zuul, aiming to provide a simple and effective unified API routing management method for microservice architecture.

Spring Cloud Gateway can be used in conjunction with Spring Cloud Discovery Client (such as Eureka), Ribbon, Hystrix and other components to implement routing forwarding, load balancing, fusing, authentication, path rewriting, log monitoring, etc., and Gateway also has built-in current limiting The filter realizes the function of current limiting.

Kong

Kong is a high-availability, easy-to-extend API Gateway project written by Mashape based on OpenResty (Nginx + Lua module). Kong is built based on NGINX and Apache Cassandra or PostgreSQL. It can provide an easy-to-use RESTful API to operate and configure the API management system, so it can horizontally expand multiple Kong servers and evenly distribute requests to Each Server, to deal with a large number of network requests.

Kong has three main components:

• Kong Server: Nginx-based server used to receive API requests.
• Apache Cassandra/PostgreSQL: Used to store operational data.
• Kong dashboard: The official UI management tool is recommended, and the admin api can also be managed in a restfull way.

Kong adopts the plug-in mechanism for function customization, and the plug-in set (can be 0 or N) is executed in the life cycle of the API request response cycle. The plug-in is written in Lua and currently has several basic functions: HTTP basic authentication, key authentication, CORS (Cross-Origin Resource Sharing, cross-domain resource sharing), TCP, UDP, file logs, API request current limiting, request forwarding and Nginx monitoring.

Kong gateway has the following features:

• Scalability: Easily scale out by simply adding more servers, which means your platform can handle any request with a lower load;
• Modularity: can be extended by adding new plugins, which can be easily configured through the RESTful Admin API;
• Runs on any infrastructure: Kong gateways can run anywhere. You can deploy Kong in cloud or internal network environments, including single or multiple data center setups, and with public, private or invite-only APIs.

Traefik

Træfɪk is a modern HTTP reverse proxy and load balancing tool that was born to make deploying microservices easier. It supports multiple backends (Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, Zookeeper, BoltDB, Rest API, file…) to automatically and dynamically apply its configuration file settings.

Important features:

• It is very fast, no need to install other dependencies, a single executable file written in Go language;
• Various background support: Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd;
• Support Rest API, Websocket, HTTP/2, Docker image;
• Listen for changes in the background and automatically apply new configuration file settings;
• Hot update of configuration files without restarting the process;
• Backend circuit breaker, load balancing, fault tolerance mechanism;
• Refreshing front-end page that can monitor service indicators.

For more information about Traefik, you can view the official website: https://traefik.cn/

2.2 API Gateway Comparison

The above is a screenshot of the gateway comparison. If you are lazy, you can mainly focus on Kong, Traefik and Zuul:

• From the perspective of open source community activity, Kong and Traefik are undoubtedly better;
• From the perspective of maturity, the better ones are Kong, Tyk, and Traefik;
• From a performance point of view, Kong is ahead of the others;
• From the perspective of the extensibility of architectural advantages, Kong and Tyk have a wealth of plug-ins, Ambassador also has plug-ins but not many, and Zuul requires self-development, but Zuul is also highly used due to its deep integration with Spring Cloud. In recent years, Istio With the popularity of service mesh, Ambassador is also a considerable advantage because it can seamlessly integrate with Istio.

The following are the thoughts and conclusions of other netizens for reference:

• Performance: The form of Nginx+Lua must be higher than the gateway implemented by the Java language. Zuul1.0 in the Java technology stack is implemented based on Servlet, and the rest are implemented based on webflux, and the performance is higher than that based on Servlet. In terms of performance, I think the choice of gateway may not be so important, and it can be done by adding a few more machines.
• Maintainability and scalability: Not many people have mastered the combination of Nginx+Lua. If the team has a master, the masters can do whatever they want. If you don’t see this passage, for the general team, choose what your team is good at. language is more important. For the three gateways under the Java technology stack, Zuul and Spring Cloud Getway need more or less integration and configuration pages to maintain, but for Soul, I have no brains to read the article, which one needs to be moved, especially It’s great to be able to connect to Dubbo without a brain. In addition, Soul2.0 and later versions can get rid of ZK. There is no criticism in my heart, and I like brainless operation.
• High availability: The strategy for high availability of the gateway is basically a unified strategy, all of which adopt the method of multi-machine deployment. A load is hung in front of it. Please pay attention to some components that need to be used outside.

3 Microservice gateway based on Traefik self-developed

This is our self-developed microservice gateway, which is developed based on Traefik. The following is an explanation from technology selection, gateway framework, gateway background, and protocol conversion. It is absolutely dry!

3.1 Technology stack selection

• Traefik: An open source reverse proxy and load balancing tool. Its biggest advantage is that it can be directly integrated with common microservice systems and can realize automatic dynamic configuration. Traefik is relatively lightweight, very easy to use and set up, and has relatively good performance. It has been used in production environments around the world.
• Etcd: A distributed, highly available consistent key-value storage system written in Go, used to provide reliable distributed key-value storage, configuration sharing and service discovery.
• Go: Strong concurrency, performance comparable to C, processing capacity 4 times that of PHP, high efficiency, simple syntax, easy to use, and development efficiency close to PHP.

3.3 Gateway framework

The whole gateway framework is divided into 3 parts:

• Gateway background (hal-fe and hal-admin): used for the configuration of applications, services and plug-ins, and then publish the configuration information to ETCD;
• Traefik: Read the ETCD configuration, and route and distribute requests according to the configuration information. If authentication is required, it will directly perform unified authentication through the hal-agent module. After the authentication is completed, if it is an Http request, it will directly hit the downstream service. If it is a Grpc and Thrift protocol, the protocol conversion will be performed through the hal-proxy module.
• Protocol conversion module: read the ETCD configuration, perform Grpc and Thrift protocol conversion on the requests distributed by Traefik, obtain service downstream machines through the service discovery mechanism, and send the converted data to the downstream service machines through load balancing.

3.4 Gateway background

It mainly consists of 3 modules:

• Application: mainly includes application name, domain name, path prefix, group to which it belongs, status, etc., such as Indian Overseas Mall and Indian Community;
• Service: mainly includes service name, registration method, agreement type, group to which it belongs, status, etc., such as comment service, address service, and search service.
• Plug-in: mainly includes the plug-in name, plug-in type, plug-in attribute configuration, etc., such as path prefix replacement plug-in, authentication plug-in.

An application can only be bound to one service, but multiple plug-ins can be bound. After the gateway configuration is completed through the background, the configuration information is generated into a Config file and published to ETCD. The Config file needs to follow a strict data format. For example, the Traefix configuration needs to follow the official file configuration format to be recognized by Traefik.

3.5 Protocol conversion module

The hal-proxy module is the most complex and the most technical module of the entire microservice gateway, so I will explain it to you in detail.

problem introduction

Before talking about this module, let's look at the following questions:

• When a request comes from the upstream trafik, it is necessary to know the IP and port of the downstream machine before sending the request to the downstream. How do these machines obtain it?
• After we have the machine, we need to establish a connection with the downstream machine. If the connection is released directly after one use, it will definitely put a lot of pressure on the service. This requires the introduction of the Client cache pool. How can we realize this Client cache pool? ?
• Finally, it is necessary to convert the protocol, because different downstream services support different types of protocols. How does this gateway support it dynamically?

Realization principle

Let's take a look at what modules are inside hal-proxy. The first is the Resolver module. What is the function of this module? Let me briefly introduce here. At present, there are many ways to obtain the machine list through services in the company, such as MIS platform, service tree, etc., that is, some are configured through the platform, and some are directly hung under the service tree. No matter which way, We all use the service name to find all the hosts under the service in a certain way.

So the role of the Resolver module is actually to find the IPs and service ports of all machines under the service through the service name, and then persist them into the memory and update them regularly.

The protocol module supports different protocol conversions. The conversion of each protocol type needs to be implemented separately. These protocol conversions are nothing more than initializing the Client through the machine IP and port, and then converting the data and sending it directly to the downstream machine. .

The last is the connection pool. We actually used the pool that comes with go to do it before, but when updating the pool data, it needs to be locked, so the performance has been unable to improve. Later, it was changed to a ring queue, and then the data All operations are performed through atomic operations, which realizes lock-free operations and greatly improves concurrency performance. The code of the ring queue is also arranged for you. You can directly read this article Go Language Core Manual-10. Atomic Operations ​​.

implement logic

This is the logic implementation diagram of hal-proxy. It has been drawn for 2 days, including the interaction methods of all core objects. I won't go into details here. How much you can master depends on your own comprehension. If you have any questions (or can't see the picture clearly) , you can follow my official account and add me to communicate on WeChat.