Some time ago, I encountered some problems in the requirements related to HTTPS. Today, I have time to sort out the relevant knowledge of HTTPS. I hope it can be helpful to you.

picture

What is HTTPS

HTTPS, that is, HTTP Security, is built on top of the SSL / TSL protocol. Among them, TSL is an upgraded version of the SSL protocol. TLS 1.0 is usually marked as SSL 3.1, TLS 1.1 is SSL 3.2, and TLS 1.2 is SSL 3.3, which can be understood as the same set of agreements. His role is mainly the following three points:

Prevent eavesdropping. All information is transmitted encrypted and cannot be eavesdropped by third parties.

Tamper-proof. With a verification mechanism, once it is tampered with, both parties in the communication will find it immediately.

Prevent impersonation. Equipped with identity certificates to prevent identity impersonation. This article will try to explain clearly what HTTPS is from the perspective of android users.

TLS verification process

TLS verification process

The interaction process is as follows:

1. The client sends a request to the server, and the content of the request includes:

1. Supported protocol versions, such as TLS version 1.0.
2. A client-generated random number that is later used to generate a "session key".
3. Supported encryption methods, such as RSA public key encryption.
4. Supported compression methods.

2. The server responds to the client with the following content:
3. Confirm the encrypted communication protocol version used, such as TLS 1.0 version. If the browser does not match the version supported by the server, the server turns off encrypted communication.
4. A server-generated random number that is later used to generate a "session key".
5. Confirm the encryption method used, such as RSA public key encryption.
6. server certificate.
7. After the client receives the server's response, it first verifies the server certificate. If the certificate is not issued by a trusted authority, or if the domain name in the certificate does not match the actual domain name, or if the certificate has expired, a warning will be displayed to the visitor, who can choose whether to continue the communication.
   If there is no problem with the certificate, the client will take the server's public key from the certificate. Then, send the following three items of information to the server.
8. a random number. The random number is encrypted with the server public key to prevent eavesdropping.
9. Code change notification, indicating that subsequent information will be sent using the encryption method and key agreed upon by both parties.
10. The client handshake end notification indicates that the handshake phase of the client has ended. This item is also the hash value of all the content sent earlier, which is used for verification by the server.
11. After the server receives the third random number pre-master key from the client, it calculates and generates the "session key" used for this session. Then, finally send the following information to the client.
12. Code change notification, indicating that subsequent information will be sent using the encryption method and key agreed upon by both parties.
13. Server handshake end notification, indicating that the server's handshake phase has ended. This item is also the hash value of all the content sent earlier, which is used for client verification.

As for why it is necessary to use three random numbers to generate a "session key", dog250 explains it well:

"Whether it is the client or the server, random numbers are needed, so that the generated key will not be the same every time. Since the certificate in the SSL protocol is static, it is very necessary to introduce a random factor to ensure the negotiated key. randomness of the key.

For the RSA key exchange algorithm, the pre-master-key itself is a random number, plus the randomness in the hello message, and the three random numbers finally derive a symmetric key through a key exporter.

The existence of the pre master is that the SSL protocol does not trust that each host can generate a completely random random number. If the random number is not random, then the pre master secret may be guessed, so it is not appropriate to only use the pre master secret as the key Therefore, a new random factor must be introduced, then the key generated by the client and the server plus the three random numbers of the pre master secret is not easy to be guessed. A pseudo-random may not be random at all, but it is three Pseudo-random is very close to random. Every time you add a degree of freedom, the randomness increases by more than one. "

HTTPS related nouns

There are many concepts involved in HTTPS, such as X509, .pem, .crt, etc. After understanding the interaction process of HTTPS, let’s take a look at these concepts first.

• X509 - This is a certificate standard that mainly defines what content should be included in the certificate. For details, please refer to RFC5280, which is the certificate standard used by SSL.
• PEM - Privacy Enhanced Mail, open to see the text format, start with "-----BEGIN..." and end with "-----END...", the content is BASE64 encoding. View the information of the PEM format certificate:

openssl x509 -in certificate.pem -text -noout
1.

Apache and *NIX servers prefer this encoding.

• DER - Distinguished Encoding Rules, open to see that it is in binary format, unreadable. Check the information of the DER format certificate:

openssl x509 -in certificate.der -inform der -text -noout
1.

Java and Windows servers prefer this encoding format.

The above are certificate standards and two commonly used encoding formats. Let's take a look at what file extensions are available.

• CRT - CRT should be the three letters of certificate. In fact, it still means certificate. It is common in *NIX system. It may be PEM code or DER code. Most of them should be PEM code. I believe you already know how to distinguish it.
• CER - or certificate, or certificate, common in Windows systems, the same, may be PEM encoding, may also be DER encoding, most should be DER encoding.
• KEY - Usually used to store a public key or private key, not an X.509 certificate, the encoding is the same, it may be PEM, or it may be DER. The way to view the KEY:

openssl rsa -in mykey.key -text -noout
1.

If it is in DER format, it should be like this: openssl rsa -in mykey.key -text -noout -inform der

• CSR - Certificate Signing Request, that is, a certificate signing request. This is not a certificate, but an application to obtain a signed certificate from an authoritative certificate authority. Its core content is a public key (of course, some other information is attached). When generating this When applying, a private key will also be generated at the same time, and the private key should be kept by yourself. Friends who have done iOS APP should know how to apply for a developer certificate from Apple. The way to check: (If it is in DER format, it is still the same Add -inform der, not written here)

openssl req -noout -text -in my.csr
1.

• PFX/P12 - predecessor of PKCS#12, for *nix servers, generally CRT and KEY are stored separately in different files, but Windows IIS stores them in a PFX file, (so this file contains the certificate and private key) will this be unsafe? Probably not. PFX usually has an "extraction password". If you want to read the contents, it will ask you to provide the extraction password. PFX uses DER encoding. How to convert PFX to PEM encoding?

openssl pkcs12 -in for-iis.pfx -out for-iis.pem -nodes
1.

At this time, you will be prompted to enter the extraction code. for-iis.pem is a readable text. The command to generate pfx is similar to this: openssl pkcs12 -export -in certificate.crt -inkey privateKey.key -out certificate.pfx -certfile CACert. crt

Among them, CACert.crt is the root certificate of CA (Authoritative Certificate Authority), and if so, it is also brought in through the -certfile parameter. From this point of view, PFX is actually a certificate keystore.

• JKS - Java Key Storage, which is a Java patent and has little to do with OpenSSL. Using a tool called "keytool" in Java, you can convert PFX to JKS. Of course, keytool can also directly generate JKS, but here Not much to say.
• BKS - from BouncyCastleProvider, it also uses TripleDES to protect the Key in the keystore, it can prevent the certificate store from being accidentally modified (the keyentry of Keystore changing 1 bit will cause an error), BKS can interoperate with JKS. The following command can be used to convert crt to bks, that is to say, bks is actually the certificate file mentioned above, and so is jks.

keytool -importcert -trustcacerts -keystore e:\key.bks -file e:\server.crt -storetype BKS -provider org.bouncycastle.jce.provider.BouncyCastleProvider
1.

bks generation reference: generation of bks format certificate used in SSL communication

HTTPS on Java

Using HTTPS on java is very simple. A typical HTTPS method is as follows:

URL url = new URL("https://google.com");  
HttpsURLConnection urlConnection = url.openConnection();  
InputStream in = urlConnection.getInputStream();
1.
2.
3.

At this time, the default SSLSocketFactory is used, which is consistent with the SSLContext used in the following code

private synchronized SSLSocketFactory getDefaultSSLSocketFactory() {  
  try {
    SSLContext sslContext = SSLContext.getInstance("TLS");
    sslContext.init(null, null, null);
    return defaultSslSocketFactory = sslContext.getSocketFactory();
  } catch (GeneralSecurityException e) {
    throw new AssertionError(); // The system has no TLS. Just give up.
  }
}
1.
2.
3.
4.
5.
6.
7.
8.
9.

The most important method in this code is sslContext.init(null, null, null); There are three parameters here, namely:

• KeyManager[], its own certificate, is used to verify whether the server is trusted. If it is a one-way verification, it will be empty, indicating that the server does not need to be verified.
• TrustManager[], verify whether the server certificate is credible. If it is empty, use android's own certificate library. If the method of the server certificate is in the default library, the verification is passed.

Sometimes the CA is also untrustworthy. In order to obtain a higher security update, we need to specify the anchor point of the newcomer ourselves. The following code can be used:

// 取到证书的输入流
InputStream is = new FileInputStream("anchor.crt");  
CertificateFactory cf = CertificateFactory.getInstance("X.509");  
Certificate ca = cf.generateCertificate(is);


// 创建 Keystore 包含我们的证书
String keyStoreType = KeyStore.getDefaultType();  
KeyStore keyStore = KeyStore.getInstance(keyStoreType);  
keyStore.load(null);  
keyStore.setCertificateEntry("anchor", ca);


// 创建一个 TrustManager 仅把 Keystore 中的证书 作为信任的锚点
String algorithm = TrustManagerFactory.getDefaultAlgorithm();  
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(algorithm);  
trustManagerFactory.init(keyStore);  
TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();


// 用 TrustManager 初始化一个 SSLContext
SSLContext sslContext = SSLContext.getInstance("TLS");  
sslContext.init(null, trustManagers, null);  
return sslContext.getSocketFactory();
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.

In this case, only anchor.crt and the certificates issued by it will be trusted.

• SecureRandom, that is, the strategy for generating random numbers, is generally passed directly to new SecureRandom() as a parameter.

Note that one-way authentication is used here, that is, only the client needs to verify the certificate of the server, and the client caches the certificate locally, so the three parameters of the sslContext.init() method here are all empty.

When the default SSLSocketFactory verifies the server's certificate (that is, when TrustManager[] is empty), it will trust more than 100 root certificates built into the device.

Since there is one-way verification, there is also two-way verification, that is, not only the client needs to verify the server, but the server also needs to verify the client. In this case, the client needs to send its own certificate to the server for verification. In this case, it is only necessary to provide the KeyManager of the certificate as the first parameter in init() to SSLContext.