How to implement business and test performance based on Reactor network model

2023.12.12

How to implement business and test performance based on Reactor network model

In this article, we implement a simple HTTP service to illustrate how to apply the Reactor network model to business. This is something that will not be discussed in schools and online 10-hour introductory C language, but it is very important.

For implementing an http service, an http request can normally be divided into two parts: request and response. We can open a web page at will, such as:

picture

We can see that an http request contains Request Header and Response Header. Based on this feature of the HTTP protocol, we can abstract two methods, http_request and http_response, which are used to process HTTP requests and responses respectively.

Then, the final process is to receive the data packet from recv_callback, then call the http_request method to parse the HTTP protocol, process the corresponding business logic, and after processing, write the response result into wbuf through http_response, then trigger the EPOLLOUT event, and finally call send_callback to The data in wbuf is sent out, and the process is roughly as follows:

picture

Let's implement http_request and http_response. The code is as follows:

int http_request(connection_t *conn) {
    // TODO parse http headr and body 
}


int http_response(connection_t *conn) {
    conn->wlen = sprintf(conn->wbuf, 
    "HTTP/1.1 200 OK\\r\\n"
    "Accept-Ranges: bytes\\r\\n"
    "Content-Length: 47\\r\\n"
    "Content-Type: text/html\\r\\n"
    "Date: Sta, 06 Aug 2023 13:16:46 GMT\\r\\n\\r\\n"
    "<html><body><h1>Hello Server</h1></body></html>");


    return conn->wlen;
}1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.

The code in http_request is omitted here. The focus of this article is how to insert business code into the event-based network model. You may feel confused. Isn’t HTTP service also a basic service? How can it be called business code? This is relative. For the TCP protocol, HTTP is actually just an application of TCP, and of course it can be regarded as business code.

The above code defines a parameter of type connection_t. In fact, the prototype of this parameter is conn_channel, which is defined as follows:

typedef struct conn_channel connection_t;1.

At this point, I don’t know if you have a feeling. As for the upper-layer business code, it does not care about the lower-layer network IO. We have not seen any direct operation of network IO in http_request and http_reponse. They only use the corresponding Reading and writing in rbuf and wbuf, it doesn't matter how the data is received or sent out. In the last article, we talked about the separation of network IO. Through this article, I believe you will have a more intuitive understanding.

I've pasted the complete code directly below.

#include <sys/socket.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <netinet/in.h>
#include <sys/types.h>
#include <string.h>
#include <fcntl.h>
#include <sys/epoll.h>
#include <error.h>


#define BUFFER_LEN 1024


int epfd = 0;


typedef int(*callback)(int);


struct conn_channel {
    int fd;


    char wbuf[BUFFER_LEN];
    int wlen;
    char rbuf[BUFFER_LEN];
    int rlen;


    union {
        callback recv_call;
        callback accept_call;
    } call_t;


    callback send_call;
};


struct conn_channel conn_map[1024] = {0};


typedef struct conn_channel connection_t;


int create_serv(int port) {
    int sockfd = socket(AF_INET, SOCK_STREAM, 0);


    struct sockaddr_in servaddr;
    memset(&servaddr, 0, sizeof(struct sockaddr_in));


    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(port);


    int on = 1;
    setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on));


    int ret = bind(sockfd, (struct sockaddr*)&servaddr, sizeof(servaddr));
    if (ret == -1) {
        perror("socket-bind-fail");
        return -1;
    } 


    listen(sockfd, 1024);


    return sockfd;
}


void make_nonblocking(int fd) {
    fcntl(fd, F_SETFL, O_NONBLOCK);
}


int http_request(connection_t *conn) {
    // TODO parse http headr and body 
}


int http_response(connection_t *conn) {
    conn->wlen = sprintf(conn->wbuf, 
    "HTTP/1.1 200 OK\\r\\n"
    "Accept-Ranges: bytes\\r\\n"
    "Content-Length: 47\\r\\n"
    "Content-Type: text/html\\r\\n"
    "Date: Sta, 06 Aug 2023 13:16:46 GMT\\r\\n\\r\\n"
    "<html><body><h1>Hello Server</h1></body></html>");


    return conn->wlen;
}


int recv_callback(int fd) {
    char *buffer = conn_map[fd].rbuf;
    int idx = conn_map[fd].rlen;


    int count = recv(fd, buffer+idx, BUFFER_LEN - idx, 0);
    if (count == 0) {
        printf("discounnect: %d\\n", fd);
        return -1;
    }


    conn_map[fd].rlen = count;


    // do http request and response
    http_request(&conn_map[fd]);


    http_response(&conn_map[fd]);


    struct epoll_event ev;
    ev.events = EPOLLOUT;
    ev.data.fd = fd;


    epoll_ctl(epfd, EPOLL_CTL_MOD, fd, &ev);


    return count;
}


int send_callback(int fd) {
   int count = send(fd, conn_map[fd].wbuf, conn_map[fd].wlen, 0);
   printf("send-count:%d\\n", count);


   struct epoll_event ev;
   ev.events = EPOLLIN;
   ev.data.fd = fd;


   epoll_ctl(epfd, EPOLL_CTL_MOD, fd, &ev);


   return count; 
}


int accept_callback(int fd) {
    struct sockaddr_in clientaddr;
    socklen_t len = sizeof(clientaddr);


    int clientfd = accept(fd, (struct sockaddr*)&clientaddr, &len);


    struct epoll_event ev;
    ev.events = EPOLLIN;
    ev.data.fd = clientfd;


    conn_map[clientfd].wlen = 0;
    conn_map[clientfd].rlen = 0;
    conn_map[clientfd].call_t.recv_call = recv_callback;
    conn_map[clientfd].send_call = send_callback;


    memset(conn_map[clientfd].wbuf, 0, BUFFER_LEN);
    memset(conn_map[clientfd].rbuf, 0, BUFFER_LEN);


    epoll_ctl(epfd, EPOLL_CTL_ADD, clientfd, &ev);


    return clientfd;
}


int main() {
    int sockfd = create_serv(2048);
    if (sockfd == -1) {
        perror("sockfd-create-fail");
        return -1;
    }


    make_nonblocking(sockfd);


    epfd = epoll_create1(0);


    printf("epfd:%d, sockfd: %d\\n", epfd, sockfd);


    conn_map[sockfd].wlen = 0;
    conn_map[sockfd].rlen = 0;
    conn_map[sockfd].call_t.recv_call = accept_callback;
    conn_map[sockfd].send_call = send_callback;


    memset(conn_map[sockfd].wbuf, 0, BUFFER_LEN);
    memset(conn_map[sockfd].rbuf, 0, BUFFER_LEN);


    struct epoll_event ev;
    ev.events = EPOLLIN;
    ev.data.fd = sockfd;


    epoll_ctl(epfd, EPOLL_CTL_ADD, sockfd, &ev);


    struct epoll_event events[1024] = {0};


    while(1) {
        int nready = epoll_wait(epfd, events, 1024, -1);


        int i = 0;
        for (i = 0; i < nready; i++) {
            int connfd = events[i].data.fd;


            if (events[i].events & EPOLLIN) {
                int ret = conn_map[connfd].call_t.recv_call(connfd);            
                printf("epollin-ret: %d\\n", ret); 
            } else if (events[i].events & EPOLLOUT) {
                int count = conn_map[connfd].send_call(connfd);
                printf("send-count: %d\\n", count);
            }
        }
    }
}1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
twenty one.
twenty two.
twenty three.
twenty four.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
91.
92.
93.
94.
95.
96.
97.
98.
99.
100.
101.
102.
103.
104.
105.
106.
107.
108.
109.
110.
111.
112.
113.
114.
115.
116.
117.
118.
119.
120.
121.
122.
123.
124.
125.
126.
127.
128.
129.
130.
131.
132.
133.
134.
135.
136.
137.
138.
139.
140.
141.
142.
143.
144.
145.
146.
147.
148.
149.
150.
151.
152.
153.
154.
155.
156.
157.
158.
159.
160.
161.
162.
163.
164.
165.
166.
167.
168.
169.
170.
171.
172.
173.
174.
175.
176.
177.
178.
179.
180.
181.
182.
183.
184.
185.
186.
187.
188.
189.
190.
191.
192.
193.
194.
195.
196.
197.
198.
199.
200.
201.
202.
203.
204.
205.
206.
207.
208.
209.
210.
211.
212.
213.
214.
215.
216.
217.
218.
219.
220.
221.
222.
223.
224.
225.
226.
227.
228.
229.
230.
231.
232.
233.
234.
235.
236.
237.
238.
239.
240.

The above code is less than 200 lines, and it does contain the core idea of the Reactor network model and the implementation method. If you are interested, you can further expand based on this. You can also refer to our previous series of network programming articles for a more complete implementation.

After the code is written, we need to further verify the performance. Here is a performance testing tool wrk recommended. Wrk is an open source performance testing tool implemented in C. The address is: https://github.com/wg/wrk. I am ready. I have two machines configured as follows:

ubuntu20.4 8C8G 192.168.56.2

ubuntu20.4-1 4C4G 192.168.56.3

Our service runs on the 8C8G machine, and another 4C4G machine is used to run the wrk tool for performance testing.

In order to have a comparison, we first installed an Nginx on the 8C8G machine for comparison. The version is 1.25.4

picture

Then we compile wrk on the 4C4G machine. The downloaded wrk directory structure is as follows:

picture

You can see that there is already a Makefile written. We only need to make it. After make is completed, a wrk binary file will be compiled.

picture

Next, we started Nginx on the 8C8G machine and the httpserver we just wrote. Our nginx runs on port 80 and httpserver runs on port 2048.

Next, we run wrk on the 4C4G machine to test nginx first, as follows:

picture

Then test the httpserver we wrote ourselves

picture

We used wrk to start 50 threads, 100 concurrency for 10 seconds, and tested Nginx and our own httpserver respectively. From the final results, we can see that without any optimization of Nginx, the httpserver we wrote is obviously better than Nginx. Performance is better. Of course, Nginx actually writes a lot of logs, and our httpserver writes almost no logs. You can try turning off the Nginx logs yourself and compare the results.

Summarize

In the process of learning C/C++, I believe many people will have such a feeling, that is, they can speak C/C++ syntax, but it is difficult to write a complete project. Personally, I feel that this is the current domestic IT education industry. Most courses spend a lot of time talking about if-else, and even talk about various variable types, but they just don't tell you how to write a complete project.

NEWS

How to implement business and test performance based on Reactor network model

How to implement business and test performance based on Reactor network model

Summarize

Exploring the core ideas of the Reactor network model

Goodbye, Citrix! Domestic cloud desktop players reshuffle their cards, who can take more cake

NEWS

How to implement business and test performance based on Reactor network model

How to implement business and test performance based on Reactor network model

Summarize

Exploring the core ideas of the Reactor network model

Goodbye, Citrix! Domestic cloud desktop players reshuffle their cards, who can take more cake​

Goodbye, Citrix! Domestic cloud desktop players reshuffle their cards, who can take more cake