Practical example: How to prevent unexpectedly high concurrent traffic from overwhelming the system?

In network applications, high-concurrency systems face a major challenge, which is a large number of concurrent accesses, such as Tmall's Double Eleven, JD's 618, flash sales, and rush sales, which are typical high-traffic and high-concurrency scenarios.

HTTP interface extreme practice
Here, we implement Web interface current limiting, specifically: using custom annotations to encapsulate the token bucket-based current limiting algorithm to implement interface current limiting.

Implement interface current limiting without annotations
Build a project

Here, we use the SpringBoot project to build the Http interface ratelimiter project. The SpringBoot project is essentially a Maven project. Therefore, you can directly create a Maven project. My project name is mykit-ratelimiter-test. Next, add the following dependencies to the pom.xml file to build the project into a SpringBoot project.

You can see that in addition to referencing the SpringBoot-related Jar package in the project, I also referenced the guava framework, version 28.2-jre.


Create core classes
Here, I mainly simulate a flow-limiting scenario for a payment interface. First, we define a PayService interface and a MessageService interface. The PayService interface is mainly used to simulate subsequent payment services, and the MessageService interface simulates sending messages. The definitions of the interfaces are as follows.

PayService

• MessageService

Next, create the implementation classes of the two, as follows.

MessageServiceImpl

• PayServiceImpl

Since it is simulating payment and sending messages, I printed out the relevant logs in the specific implementation method and did not implement the specific business logic.

Next, create our Controller class PayController. In the pay() method of the PayController class interface, current limiting is used to put 2 tokens into the bucket per second, and the client obtains the token from the bucket. If the token is not obtained within 500 milliseconds, we can directly go to service degradation processing.


The code of PayController is as follows.

At this point, we have basically completed the implementation of the current limiting Web application without using comments.

Run the project
After the project is created, we run the project. Running the SpringBoot project is relatively simple. Just run the main() method of the MykitLimiterApplication class.

After the project runs successfully, we enter the link in the browser address bar: http://localhost:8080/boot/pay. The page will output the words "Payment successful", indicating that the project has been successfully built. As shown in the figure below.

At this time, I only visited once and did not trigger the current limit. Next, we kept refreshing the browser. At this time, the browser will output the words "Payment failed, try again...", as shown below.

There is also a sendMessage() method in the PayController class, which simulates the interface for sending messages and also uses the current limit operation. The specific code is shown below.

The code logic and operation effect of the sendMessage() method are the same as the pay() method. I will no longer visit the browser to access the http://localhost:8080/boot/send/message address. Friends can verify it by themselves.

Disadvantages of implementing current limit without using annotations

Through project writing, we can find that when limiting the flow of the interface in the project, not using annotations for development will lead to a lot of redundant code. Almost every method has to write the same flow limiting logic, and the code is very redundant.

How to solve the problem of redundant code? We can use custom annotations for implementation.

Use annotations to implement interface flow limiting
Using custom annotations, we can encapsulate some common business logic into the annotation aspect, and add corresponding annotations to the method that needs to add the annotated business logic. For our current limiting example, it can be implemented based on custom annotations.


Implementing custom annotations
To implement, let's create a custom annotation, as shown below.

Custom Annotation Aspect Implementation
Next, we also need to implement an aspect class MyRateLimiterAspect, as shown below.

Next, we modify the sendMessage() method in the PayController class. The modified method snippet code is shown below.



Run the deployment project
Deploying the project is relatively simple. You only need to execute the main() method under the MykitLimiterApplication class. Here, for simplicity, we still directly enter the link address from the browser to access it.

The effect is shown in the figure below.

Next, we refresh the browser continuously. The words "Message transmission failed, try again..." will appear, indicating that the current limiting operation has been triggered.


Disadvantages of implementing current limiting based on current limiting algorithms
The current limiting methods described above can only be used in a single-machine deployment environment. If the application is deployed to multiple servers for distribution or clustering, the current limiting methods above are not applicable. In this case, we need to use distributed current limiting. As for how to implement current limiting operations in a distributed scenario, we will introduce it in the next article.