HTTPS is a secure channel. If there is a small green lock like an A-share in front of the browser navigation bar, then it will feel very reassuring.

Lock up your shameful thoughts and foul language with this little lock, and do whatever you want. Thinking about it makes people excited.

But wait, why did Charles catch HTTPS packets?

Simple principle of HTTPS

We hope that the data transmission process is a black box for the user and a black box for the attacker. This is mainly reflected in two aspects.

• Users do not want their sensitive data to be accessed. For example, your own account password, such as the chat data that you send to your girlfriend.
• Developers do not want their data to be obtained by users. For example, if your own signature verification method is cracked, users can do many illegal things.

The HTTP protocol is a question-and-answer protocol, which is transmitted in clear text during the transmission process.

If the attacker intercepts the transmission message between the web browser and the web server, he can read the information directly. For example, the content of the packet is obtained through a proxy method or a LAN sniffing method.

There are many related tools, such as wireshark, tcpdump, dsniff, etc.

But if the content of the transmission is encrypted, then even if you catch all the datagrams, it is of little value.

HTTPS is a protocol for transmitting encrypted data. If we add a TLS/SSL​ layer between TCP and HTTP, it becomes HTTPS.

HTTPS includes a handshake phase and a transfer phase. The handshake phase is the most important negotiation phase.

The goal of the handshake is to securely exchange symmetric keys, and the whole process requires the participation of 3 random numbers. After the Change Cipher Spec, all the encrypted content is transmitted.

This process can be easily captured using the WireShark packet capture tool. There are a lot of articles analyzing the handshake process, so I won't repeat them here.

Of course, the efficiency of HTTPS is very low. A little expansion here.

HTTP3, which is Google's QUIC, in addition to solving the problem of head-of-line blocking, can also be used as an alternative to TCP+TLS+HTTP/2​. HTTP3 is a secure channel by default, using the UDP protocol. Powered by the DH key exchange algorithm, it reduces connection establishment time - 0 RTT round trips in common cases.

This is much faster than the HTTPS handshake.

Charles capture

Although the transmission process of HTTPS is encrypted, if we are the initiator of the request and the device is in our own hands, it is very easy to capture the content of the HTTPS connection.

This is a headache for developers. For example, if I use the AK and SK provided by the cloud platform to directly initiate HTTPS calls, the user can capture these two key passwords. Therefore, ordinary developers cannot directly transmit AK and SK on the network, even if this is functional.

Here I take the HTTPS request of the packet capture browser on the MacOS machine as an example to illustrate the use of Charles.

After starting Charles, we need to set it as the system proxy.

Then, under the Help/ menu, find the Root certificate to install.

安装完毕之后，我们还要信任这个证书。这样，当你的浏览器访问我们的Charles代理时，就可以畅通无阻。

安装到System Keychains中，而且一定要信任它哦。

通常情况下，我访问一个HTTPS连接，抓到的内容都是一团糟。

我们还差最后一步。默认情况下，Charles并没有任何过滤，我么还需要把要抓包的网址，加入HTTPS的代理配置中才可以。

右键找到这个连接，然后选择启动SSL代理即可。

此时，我们再看一下这些连接的内容，就能够变成人眼能够识别的了。

当然，电脑上的代理没有什么意义。我们做代理，一般是想要抓取手机上的应用产生的请求。

但方法是一样的，你只需要把这个Root证书，安装到你的手机中，然后信任它就可以了。

为什么能够抓到数据？

在这个案例中，Charles是作为中间人而存在的。对于Charles来说，对于服务端的请求，是由它发起的。

你可以把它想象成一个浏览器，它发出的请求和返回的内容，对于Charles自身来说自然是可见的。欺骗服务器很容易，重要的欺骗客户端。

Charles通过伪造一个CA证书，来冒充一个服务端。当浏览器或者移动手机访问Charles冒充的服务端时，Charles会携带CA证书返回给客户端。

对于普通的CA证书来说，浏览器和客户端是不信任的。这也是为什么要进行HTTPS抓包，必须安装CA证书的原因---我们需要把这个信任关系建立起来。

这两部分是割裂的，可以说是由两条完全不同的SSL通道。请求报文在全程是加密的，除了一个非常薄弱的交接点。

在通道的粘合处，所有的信息却是明文的。Charles掌控了这个过程，自然就能够把原始信息展示出来。

End

可以看到，Charles是可以抓取到HTTPS的明文信息的。在中间人场景中，它既作为客户端发起请求，也作为服务端接收请求，然后在请求的转发处获取数据。

作为用户，我们千万不能随意信任来历不明的证书，否则你的很多隐私数据将暴露在阳光之下。

作为开发者，也不能把敏感数据直接放在URL或者请求体里，防止用户抓包获取到这些信息，对服务造成破坏。

当然，在CN，隐私可能是个伪命题。就比如xjjdog，虽然我一直在隐藏自己，但还是有很多朋友知道我到底是不是带把的。

这个时候，HTTPS就没什么用。

作者简介：小姐姐味道  (xjjdog)，一个不允许程序员走弯路的公众号。聚焦基础架构和Linux。十年架构，日百亿流量，与你探讨高并发世界，给你不一样的味道。