When people started getting interested in Hadoop a few years ago, WANDisco prepared for the expected surge in business with its big data movement technology. However, the business never fully took off. When the Hadoop bubble finally burst, companies had to find new markets. Now that industrial IoT data collected via 5G networks is becoming mainstream, WANDisco is part of the big data boom on the cloud.

Hadoop is a distributed system infrastructure developed by the Apache Foundation. Users can develop distributed programs without knowing the underlying details of the distribution . Make full use of the power of the cluster for high-speed computing and storage.

Like many in the industry, WANdisco CEO David Richards has been frustrated by the Hadoop era. Fortunately, he managed to steer the company through the Hadoop storm and into more pleasant "waters".

It was a blessing in disguise, Richards said of the whole experience with Hadoop: "We built a product that solved a large-scale data problem, and looked like a bunch of idiots at the time, because no one had such a large-scale data."

As the migration to Hadoop winds down, WANDisco's core technology needs a new marketplace that helps users move large amounts of data across the network in a timely manner. The company's advantage is its ability to enable analysts or data scientists to start working on data the moment it reaches its destination, rather than waiting days, weeks or months to move entire datasets.

Because consensus is used to ensure data is accurate, data doesn't have to be scanned, so machine learning can start running within a second of turning on the product. It can also be multiplexed on any available bandwidth and makes the whole thing twice as efficient.

Despite the failure of Hadoop, the cloud data platform has risen, and the object storage data flowing into the cloud is indeed huge. WANdisco was recently used to move 25 petabytes (1PB=1024TB) of data from a large telco to the cloud, where it can be analyzed using one of the popular cloud data warehouses. It was also involved in another deal to move 1 exabyte (1 exabyte = 1024 petabytes) of data per year to the cloud for a major automotive company.

WANdisco became a 10-year success overnight. The technical theory has never changed that moving petabytes of data over the Internet is very difficult, and making sure the data arrives in the correct order and in a timely manner is also a problem.

5G technology was initially promoted primarily as a consumer technology. New wireless technologies will allow us to download full movies in seconds and will usher in a new era of interactive gaming. 5G will be the new network that will allow us to participate in augmented reality and enjoy the metaverse.

As it turns out, that's not all 5G is going to be able to do. Richards said: "The use cases are not just video or mobile. More industries are focusing on becoming IoT backbone companies with complete solutions of sensor edge cloud.

Data collected from cars is particularly large and needs to be timely. Manufacturers are collecting reliability and performance data on EV components, while insurers are interested in driving data to help price premiums.

WANdisco is a UK-listed company headquartered in the US. In 2022, revenue soars 967%, from about $12 million to $127 million. The company reported deferred revenue of about $110 million, up from less than $10 million last year.

Richards called the figures "ridiculous" and was shocked by the sudden volume of business WANDisco had generated. "What's happening in the company right now is crazy. I've never seen anything like it. That's what we say is going to happen. But when it does happen, you have to be able to support it."

Richards is optimistic that the big data faucet will only open wider. As companies realize the value of their data is far greater than the value of their traditional business, this will create the need for greater data movement to drive larger machine learning workloads.

Meanwhile, WANdisco released Data Activation Platform 2.0, which the company claims will lead to faster movement of larger data sets to the cloud. The new version provides better support for using multiple data transfer agents at the same time; automatic cleaning of the data source environment; enhanced data fidelity verification; enhanced control over data transfer priority; and user interface enhancements.