Author: Yang Ailin, Intel Engineer (Cloud Orchestration Engineer)

The main content of this article is translated from CryptoMB - TLShandshake acceleration for Istio ​​blog

In the service mesh Istio, a large amount of mTLS (Mutual Transport Layer Security) authentication is used in the communication between microservices and microservices at the ingress gateway, and a large amount of computing resources are required to process mTLS, which makes the mTLS processing process It may become a performance bottleneck. This article discusses how to use the CryptoMB Private Key Provider configuration to accelerate the TLS (Transport Layer Security) handshake in the service mesh ingress gateway and the sidecar proxy.

When it comes to mTLS, the TLS secure connection protocol, encryption and decryption operations are one of its most computationally intensive and critical operations. The service mesh Istio uses the proxy Envoy as a "gateway/sidecar" to handle secure connections and intercept network traffic.

Depending on the usage scenario, the Envoy proxy load increases and performance degrades when the ingress gateway has to handle a large number of TLS connection requests, as well as secure connections between services through the sidecar proxy mode. Of course the performance degradation depends on many factors such as the size of the cpuset running the Envoy proxy, incoming traffic patterns and key size. These factors must affect the corresponding time of the Envoy proxy for new TLS requests. To improve and speed up handshake performance, a new feature was introduced in Envoy 1.20 and Istio 1.14 for TLS acceleration. It can be achieved by utilizing the 3rd Generation Intel® Xeon® Scalable processor directive AVX512, the Intel® Integrated Performance Primitives (Intel®IPP) cryptographic library, the CryptoMB Private Key Provider in Envoy, and the ProxyConfig configuration in Istio.

Introduction to CryptoMB

The Intel® IPP cryptographic library Crypto library (https://github.com/intel/ipp-crypto/tree/develop/sources/ippcp/crypto_mb) supports multi-buffer cryptographic operations. Simply put, multi-buffer encryption refers to using the SIMD (Single Instruction Multiple Data) mechanism, implemented through Intel® Advanced Vector Extensions 512 (Intel® AVX-512) instructions. Up to eight RSA or ECDSA operations are collected into a single buffer and processed simultaneously, greatly improving the performance of cryptographic operations. The third-generation Intel® Xeon® Scalable processors (Ice Lake servers) launched by Intel in recent years support Intel® AVX-512 instructions and their extended instructions, such as AVX512 IFMA, AVX512VAES.

The idea behind implementing the CryptoMB Privatekey provider in Envoy is to use Intel® AVX-512 multi-buffer instructions during the TLS handshake to speed up RSA operations.

Accelerating Envoy with Intel AVX-512 instructions

Envoy uses BoringSSL as the default TLS protocol library. BoringSSL achieves offloading asynchronous private key operations by supporting the method of setting private keys. For this reason, Envoy implements a private key provider framework, Private Key Provider, to allow the creation of Envoy extensions that use BoringSSL hooks (Hooks) to handle TLS handshake private keys. Operations (signing and decrypting).

The CryptoMB Private Key Provider is a concrete implementation of the Envoy extension Private Key Provider that uses Intel® AVX-512 multi-buffer acceleration technology to handle operations involving TLS RSA in BoringSSL. When a new handshake occurs, BoringSSL calls the private key provider to request a cryptographic operation, then returns control to Envoy, and the RSA request is collected in a buffer. When the buffer is full or the timer expires, the private key provider will call Intel AVX-512 to process the buffer. When processing is complete, Envoy will be notified that the encryption/decryption operation is complete and the handshake can continue. The following figure shows the procedure call to implement TLS acceleration:

Envoy <-> BoringSSL <-> PrivateKeyProvider

The Envoy worker thread has a buffer size for 8 RSA requests. When the first RSA request is stored in the buffer, a timer is started (the timer duration is set by the poll_delay field in the CryptoMB configuration).

Buffer timer started

When the buffer is full or the timer expires, all RSA requests are encrypted simultaneously. SIMD (Single Instruction Multiple Data) processing has potential performance advantages over the non-accelerated case.

Buffer timer expired

Envoy CryptoMB Private Key Provider Configuration

In Envoy use, regular TLS configuration only uses private keys. When using Private Key Provider, the private key filed will be replaced by the private key provider field. The Private Key Provider field contains two fields: the provider name "providername" and the type configuration "typed config". The type configuration is set to CryptoMbPrivateKeyMethodConfig. This configuration parameter is used to specify the private key and the polling delay. The polling delay is to set the "poll_delay" mentioned above. The specific TLS configuration is as follows:

TLS configuration using only the private key (multi-buffer acceleration is not enabled in this case)

tls_certificates:
certificate_chain: { "filename":
"/path/cert.pem" }
private_key: { "filename": "/path/key.pem" }
1.
2.
3.
4.

TLS configuration using CryptoMB private key provider (with multi-buffer acceleration enabled)

tls_certificates:
certificate_chain: { "filename": "/path/cert.pem" }
private_key_provider:
provider_name: cryptomb
typed_config:  
"@type":
type.googleapis.com/envoy.extensions.private_key_providers.cryptomb.v3alpha.CryptoMbPrivateKeyMethodConfig 
private_key: { "filename": "/path/key.pem" }
     
poll_delay: 10ms
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

 CryptoMB Private Key Provider Configuration in Istio

In Istio, sidecar proxy configuration includes several types:

Pod level: Pod level configuration is set through resource annotations pod annotations

Mesh level: The mesh level is set with the global mesh option proxyConfig

EnvoyFilter: Provides a mechanism to customize the Envoy configuration generated by Istio Pilot

CryptoMB Private Key Provider configuration can be applied to the entire mesh, a specific gateway, or a specific pod's configuration using pod annotations. Users can also set the "poll_delay" value for PrivateKeyProvider through ProxyConfig, this configuration can also be applied to the whole network, that is, the ingress gateway and all sidecar proxy sidecars.

A simple network-wide configuration is as follows:

Sample mesh wide configuration

Istio Mesh full network configuration example:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
namespace: istio-system
  name:
example-istiocontrolplane
spec:
  profile:
demo
components:
egressGateways:    -
name: istio-egressgateway     
enabled: true   
ingressGateways:
-
name: istio-ingressgateway    
enabled: true
meshConfig:
defaultConfig: 
privateKeyProvider:       
cryptomb:         
pollDelay: 10ms
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
twenty one.
twenty two.

Istio Ingress Gateway Configuration

If the user only wants to configure the ingress gateway as the Private key Provider configuration, the example is as follows:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
namespace: istio-system
  name:
example-istiocontrolplane
spec:
  profile:
demo
components:
egressGateways:
-
name: istio-egressgateway     
enabled: true  
ingressGateways:
    -
name: istio-ingressgateway    
enabled: true
      k8s:       
podAnnotations:       
proxy.istio.io/config: |        
privateKeyProvider:         
cryptomb:
                pollDelay: 10ms
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
twenty one.
twenty two.
twenty three.
twenty four.

Using pod annotations to configure the Istio sidecar proxy

If the user only wants to configure specific application Pods as private key provider configuration, then the easiest way is to use pod annotations, for example:

apiVersion: v1
kind: ServiceAccount
metadata:
  name:
httpbin
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin
  labels:
    app:
httpbin
   
service: httpbin
spec:
  ports:
  - name:
http
    port:
8000
   
targetPort: 80
 
selector:
    app:
httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name:
httpbin
spec:
replicas: 1 
selector:
    matchLabels:
      app:
httpbin     
version: v1
template:
 
metadata:
     
labels:
       
app: httpbin
       
version: v1
     
annotations:
       
proxy.istio.io/config: |
         
privateKeyProvider:
           
cryptomb:
             
pollDelay: 10ms
    spec:
     
serviceAccountName: httpbin
     
containers:
      -
image: docker.io/kennethreitz/httpbin
       
imagePullPolicy: IfNotPresent
       
name: httpbin
      
ports:
       -
containerPort: 80
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
twenty one.
twenty two.
twenty three.
twenty four.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.

 Applications

From September 2021, the CryptoMB code has been integrated into the mainline of the Envoy community. For details, please read the official document Envoy 1.23.0

api-v3 extension module ​​CryptoMb private key provider​ ​, if you are using Istio 1.13, Istio 1.14 or later versions, it already includes the corresponding version of Envoy, such as Envoy 1.22, which already includes CryptoMB by default, without the need for Compile by yourself, this acceleration capability based on CryptoMB encryption/decryption can be applied to ingress gateways

istio-ingress-gateway , can also work with microservice proxy isti o-proxy sidecar containers. Check if your system is 3rd Generation Intel® Xeon® Scalable Processor (codename Ice Lake) before use, this processor supports

AVX512 extended instructions, multi-cache instructions. And this acceleration capability includes:

• AVX-512 crypto acceleration for TLS connections
• AVX-512 vector AES for symmetric data encryption

CryptoMB accelerated TLS has been adopted by Alibaba's Alibaba Cloud Service Grid product ASM. Alibaba Cloud Service Grid ASM combines this Multi-Buffer technology and configures and enables this function to provide TLS acceleration. The usage methods are as follows:

1) When Multi-Buffer is not enabled, the TLS configuration only needs to include a private key. Once Multi-Buffer is enabled, a private key provider needs to be provided in the TLS configuration, which is the cryptoMB private key provider mentioned above. The message processed by the provider is of type CryptoMbPrivateKeyMethodConfig, including two main fields, one is The private key used, and the other one is used to set the waiting time pool_delay that each thread processing queue should be executed, which is used to control the balance between delay and throughput;

2) The configuration of the control plane can be delivered to the Envoy proxy of the data plane through the xDS protocol. Combining the Intel® IPP encryption library and the CryptoMB private key provider, using the AVX512 instruction set, service mesh implementations can offload the TLS handshake to handle more connections, reduce latency, and save CPU.

3) Alibaba Cloud Service Grid ASM determines whether the model supports the AVX512 instruction set by judging the machine model, and then decides whether to enable this function.

In the Alibaba Cloud Service Grid ASM product, global configuration is currently provided, and multi-level configuration is being gradually supported. Tested on Alibaba Cloud's G7 type machine, after enabling multi-buffer, TLS performance test was performed on ASM products, and the number of requested
QPS increased by 75 % (this data is public data, sourced from Alibaba service grid products ​Introduction ​​)