foreword

With the advancement of the new four modernizations (electrification, intelligence, networking and sharing) in the entire automotive travel field, various auto manufacturers are gradually building a car networking system with intelligent driving and intelligent networking as the core.  The new generation of Internet of Vehicles system puts forward higher requirements for the platform architecture of underlying message collection, transmission and processing.

In the previous article of this series, "MQTT Protocol in the Internet of Vehicles Scenario", we have mentioned that the MQTT protocol is currently the most suitable communication protocol for building a data platform in the Internet of Vehicles scenario.  Based on this, in this article, we will continue to discuss the collection and transmission of MQTT messages in the Internet of Vehicles scenario, and how to build a MQTT messaging platform for the Internet of Vehicles with a scale of 10 million, in order to provide a platform architecture design reference for enterprise users who are engaged in the Internet of Vehicles business.

The basis of the Internet of Vehicles: message collection and transmission

Evolution of Internet of Vehicles Transmission Protocols

As we all know, vehicle-to-everything (V2X) refers to the interaction between vehicle and cloud, vehicle and network, vehicle and vehicle, vehicle and road, vehicle and human, vehicle and sensor equipment, etc., to realize the communication between vehicle and public network. The dynamic mobile communication system is a heterogeneous communication network established to meet the elements of efficiency, safety and management in every link related to the vehicle. The communication protocol running in it has become the key and core of the construction of the Internet of Vehicles system.

In the course of the development of the Internet of Vehicles, there are mainly two mainstream communication technologies, which have played a role in promoting the overall development of the Internet of Vehicles:

• DSRC (DeDICated Short Range Communication): It was developed by ASTM in 1992 for the business scenario of ETC. After years of improvement and iteration, it evolved into the IEEE (802.1X) vehicle networking communication technology standard. For quite a long time, DSRC technology has been the mainstream car networking communication protocol used in the mainstream production and consumer markets of the international automobile.
• **C-V2X (Cellular Vehicle to Everything, cellular vehicle networking communication): **C-V2X relies on the existing cellular base station, in addition to supporting the direct communication of PC5, RSU and vehicles can pass the 4/5G channel (using Uu interface) is connected to the V2X platform to realize vehicle-road collaborative communication. Compared with DSRC, C-V2X is technically better, it enhances the security and confidentiality of communication, supports high network capacity, and can support high bandwidth and large data volume requirements.

The competition between DSRC and C-V2X technologies is fierce, and both are expected to become mainstream vehicle networking communication standards. At present, my country has the most complete 5G communication network infrastructure, so it is more inclined to adopt C-V2X (LTE-V, 5G-V2X) communication technology. Next-generation car networking architecture for autonomous driving.

The significance of message platform construction for the Internet of Vehicles

With the rapid development of the Internet of Vehicles construction today, all OEMs have formed a consensus: the purpose of the Internet of Vehicles construction is not networking for networking, nor networking for in-vehicle entertainment, networking is for data. With the Internet of Vehicles, there is data. With data, supplemented by a complete data governance and application system, everything is available.

And the target data of this business is not limited to the relevant data on the car side. In the V2X framework, it is necessary to solve the interconnection between vehicle and vehicle (V2V), vehicle and road (V2R), vehicle and network (V2I), vehicle and cloud (V2C), vehicle and human (V2H), etc. Comprehensive data collection and analysis of roads, clouds, networks, and people. The 5G-based C-V2X protocol and communication method provide basic capability guarantee for the construction of the entire system.

From traditional OTA applications to many new intelligent application scenarios such as smart cockpit, high-precision map adaptation, centimeter-level positioning, long-distance connection between vehicle and mobile terminal, mobile terminal message collection, vehicle-road cloud map, vehicle-road collaboration and many other new intelligent application scenarios, the Internet of Vehicles business is very important to the message platform. The demand for and data processing system has expanded from the original vehicle cloud to the overall architecture construction of people-vehicle-road-network-cloud, which has put forward higher requirements for the construction of the entire message platform.

How to build a message communication and transmission system architecture with massive connections, high concurrent throughput, and low latency to ensure the ubiquity, convenience, high availability, reliability, security and high concurrency of the entire system has become a The key to the construction of a new generation of Internet of Vehicles systems in autonomous driving and vehicle-road collaboration scenarios.

Architecture design of tens of millions of car networking message platform

Next, we will take EMQ's IoV message platform and data processing overall solution as an example to introduce how to build a ten million-level IoV message platform.

business challenges

(1) Secure access to vehicle, road test unit and mobile phone system

The car side needs to cover new car-connected services such as car-machine data reporting, POI distribution, file push, configuration distribution, push message, operation care, etc. The mass message topics generated require more secure and stable access and transmission to realize message subscription and publishing . The road end needs to realize the secure access of the roadside RSU, the collection and transmission of messages, and the transmission of map data.

(2) Real-time and reliability of large concurrent message delivery

Application scenarios such as high-precision maps, centimeter-level positioning, and vehicle-road coordination all need to solve the millisecond-level low-latency and high-reliability transmission capability guarantee of massive vehicle-road map messages. The message processing platform requires high performance, low latency, and high reliability. The ability to support tens of millions of connections and millions of concurrent business scenarios.

(3) Rich application scenario integration

In the Internet of Vehicles system with autonomous driving as the core, it is necessary to use the message platform to connect various applications based on people, roads, maps, and clouds. Connect the vehicle-end data with application scenarios such as high-precision maps, centimeter-level positioning, vehicle-road coordination, and mobile phone connection through the message platform, ensure the consumption supply of applications through the message platform, and provide high-performance, low-latency and high-reliability. data schema.

(4) Mass data storage, processing and distribution

After the massive IoT data from people, vehicles, roads, clouds, graphs, and networks is collected, it is necessary to conduct full life cycle management for the access, storage, processing, and distribution of these large-scale real-time data streams to provide applications. Database support for dynamic and continuous data flow supports applications to deeply use car networking data to serve consumers and make business decisions.

Overall solution

In the solution, we mainly use EMQ's cloud-native distributed IoT access platform - EMQX, to realize the data connection, movement and processing of the vehicle-end, people and road-end in the Internet of Vehicles system. EMQX's integrated distributed MQTT message service and powerful IoT rule engine can provide a basic capability base for highly reliable, high-performance IoT real-time data movement, processing and integration, helping enterprises to quickly build critical business IoT platforms and applications.

(1) Message processing for the vehicle end

EMQX uses the MQTT protocol to access the connected car system. The vehicle terminal is connected to the EMQX distributed cluster through load balancing. The horizontal expansion capability of EMQX can realize the data communication capability of tens of millions of vehicle connections and millions of concurrent responses. Through the rule engine, one-stop capabilities such as massive message bridging message queues, persistent storage, offline message storage, etc. can be realized, while providing rich API atomic capabilities northbound integration.

In terms of security, EMQX not only supports TLS/DTLS or national secret GMSSL security protocol to ensure the reliability and stability of the system; it also provides multiple guarantee mechanisms such as heartbeat monitoring, will message, QoS level, etc., through offline message storage to achieve real-time in a complex network environment , Safe and reliable vehicle-machine message communication.

(2) Message processing for people and roadside

EMQX provides a message collection and processing platform for mobile APP, RSU and other terminals for people and road ends. Based on the 5G network slicing capability, through the nearby access of personal terminals and roadside units, ultra-low latency traffic information services are realized. Through MQTT and other protocols, the road condition information perceived by the human terminal and roadside facilities is pushed to the cloud control platform, and the cloud control platform integrates V2X algorithm to realize intelligent traffic scenarios such as road collaborative perception, safety reminder, and remote collaborative control.

In terms of security, it supports the international standard TLS/DTLS encryption or the national encryption algorithm GMSSL encryption, and guarantees the secure communication of the coordination of the human-vehicle-road information system by extending the PKI/CA certificate authentication system.

10 million message access framework model

For the new generation of Internet of Vehicles scenarios, the overall message access and data processing platform reference architecture of EMQ with tens of millions of connections and millions of concurrency is as follows:

• Business scenario: Vehicles, mobile APPs, roadside RSUs and other equipment in the Internet of Vehicles system are connected through MQTT, enabling concurrent access to more than ten million terminals.
• **System architecture:** Terminal devices are accessed through protocols such as MQTT and HTTP, and are connected to the distributed messaging platform EMQ X through load balancing components. Through distributed multi-cluster deployment to meet the requirements of tens of millions of concurrent connections, according to the throughput capacity of millions of messages, the rules engine is connected to the Kafka cluster to realize data forwarding. The Internet of Vehicles service platform, high-precision map service, V2X cloud control service, positioning service and other applications related to the vehicle network can be directly consumed by subscribing to Kafka data. Meanwhile, EMQ provides three southbound interface services of REST, MQTT and MQ message queue. Two-way communication of vehicle control (remote control) messages.

Through the above reference framework, EMQ can achieve the business requirements of tens of millions of connections and millions of concurrent throughputs in the Internet of Vehicles scenario through the EMQX cloud-native distributed IoT access platform.

Tens of millions of messages access test

Test environment and purpose

A car company plans to verify the following capabilities of the EMQX cluster based on the test environment in the Internet of Vehicles scenario, and prepare the corresponding technical architecture and capability support for subsequent business growth:

• It can support 10 million concurrent connections, and support QoS 0 messages of 100,000 to 150,000 per second, with a payload of 100 bytes, bridged to Kafka through the rule engine;
• 10 million concurrent connections to subscribe and consume OTA broadcast topics;
• 3 million users connecting at the same time will not cause cluster avalanche and test the time it takes to connect.

In addition, after completing all the above tests, continue to explore the highest throughput of message sending and bridge forwarding to Kafka that can be supported under the current configuration with 10 million concurrency (increase the client message sending frequency according to the EMQX cluster resource usage), and Tested to meet the highest message throughput with a QoS of 2 and an average response time of 50 ms under 10 million connections.

test preparation

The client connects to the load balancing ELB through TLS encryption, then TLS terminates the client at HAProxy, and finally connects to the EMQX cluster through TCP. By terminating TLS on HAProxy, the support capability of the EMQX cluster can be improved. In this deployment mode, the processing capability of EMQX is exactly the same as the client's direct connection through MQTT TCP. On the other hand, compared with the MQTT TCP connection, the client needs to consume more resources through the TLS connection. The scale of this test is tens of millions, and the number of test machines required is large. In order to reduce the required test resources, it does not Affects the test target of EMQX cluster, this test will use TCP connection directly.

testing scenarios

Test Results

The following are the results of this test:

summary

As shown in the above results, under the current deployment architecture, it can meet the verification requirements of the car company for 10 million concurrent connections + 200,000 messages bridged to Kafka, message broadcasting, and 3 million instantaneous concurrent connections. In the exploratory test, the maximum TPS of 1.2 million messages (QoS 0, payload 1kB) was tested under 10 million connections. The test lasted for 10 hours. The EMQX cluster was stable, the CPU idle was as low as 20%, and the memory usage was stable.

It can be seen from the above that EMQX has outstanding performance in supporting tens of millions of connections in the Internet of Vehicles scenario, and the architecture is stable and reliable.

Introduction and use of stress testing tools

Due to the large number of test machines required and the complicated management, this test was conducted using the XMeter performance test platform and the JMeter-MQTT plug-in of the commercial version of EMQ's test software.

• XMeter is an extended performance testing platform based on the open source testing tool JMeter. In view of the characteristics of the Internet of Things, such as large access scale, elastic expansion requirements, multiple access protocols, and mixed scenarios, XMeter has transformed JMeter to support large-scale and high-concurrency performance testing, such as the realization of 10 million-level MQTT Concurrent connections and message throughput testing. In addition to testing the MQTT protocol, it can also support the testing of mainstream applications such as HTTP/HTTPS.
• The JMeter-MQTT plug-in is an open source MQTT performance test plug-in implemented by XMeter, which has been used in many projects and is currently the most popular MQTT plug-in in the JMeter community.

write at the end

Through this article, we introduce the architecture design of the MQTT messaging platform for the Internet of Vehicles with tens of millions of levels based on the cloud-native distributed IoT access platform EMQX, and verify the performance of the architecture in the environment of tens of millions of concurrent connection scenarios. The construction of the system's message data platform provides a possible design reference.

As the world's leading provider of IoT data infrastructure software, EMQ is committed to building high-performance, low-latency, high-availability, and high-reliability products, and providing overall solutions for message collection, movement, processing, and analysis for the next-generation Internet of Vehicles system The solution provides infrastructure service guarantee for the autonomous driving and intelligent networked vehicle business of vehicle manufacturers, T1 suppliers, after-market service providers, travel service companies and government management agencies, and realizes the intelligent connection of people, vehicles, roads and clouds .