In recent years, two brothers, High Performance Computing (HPC) and cloud computing, have always appeared in various occasions at the same time, and a new term has emerged — HPC On Cloud. What is HPC On Cloud? Why is there HPC On Cloud? Let’s talk about it together.
Cloud Computing vs HPC
Let’s take a look at what their concepts are.
Cloud computing is a kind of usage mode that distributes according to demand and charges according to usage. Users can put hard disks and CPUs on their computers or company servers on the Internet and make unified dynamic calls. They only need to pay by hour or even minute, instead of buying computers and servers by themselves. For example, cloud computing is similar to tap water at home, so we don’t need to build a water plant in our own home but just need to turn on the tap and distribute it on demand by paying.
HPC is applied the area of large-scale data processing. Take Baidu’s project team as an example, HPC is used in many departments, such as speech recognition, image recognition, LBS, natural language processing and so on. Extending to our life, Alipay face recognition and AlphaGo robot design are all based on the powerful HPC algorithm.
From their concepts, we can see that the starting points of cloud computing and HPC are different. Cloud computing usually uses virtualization technology to enable a single physical machine with excess resources to accomplish multiple tasks independently and in isolation. The essence of cloud computing is the business model of computing resources, while that of HPC is to break through the resource advantages of a single machine and allow more machines to accomplish a task together. In a narrow sense, HPC mainly includes parallel computing and cluster computing, which are used in high-tech simulation, computing and so on. The former emphasizes humanization and the latter pays more attention to performance.
Traditional HPC architects often take a wait-and-see attitude towards cloud computing technology. As we all know, virtualization technology is a significant feature of cloud computing, but architects believe that virtualization can affect the performance of applications. Virtualization improves flexibility, but at the same time reduces the throughput of applications. Enterprises may use cloud architecture for improving hardware utilization, integrating servers, or increasing license utilization, but the workload of virtualized resource operation also brings about the shortcomings of complexity and cost increase.
So, cloud computing and HPC seem to come from two different planets, so why do they always appear in pairs?
HPC On Cloud
Although HPC and cloud computing have different starting points, they are not opposites.
Today, HPC is no longer the HPC that is only suitable for the most advanced core technology enterprises of the government. With the rapid development of Artificial Intelligence (AI) industry, the demand of high performance data analysis and numerical simulation is moving towards more enterprises. In addition to the traditional research market, the HPC business market is in full swing.
Although HPC has excellent performance, it is far inferior to cloud computing in terms of user interface and flexibility. And compared with HPC, cloud computing advances in providing virtualization and flexibility of storage and network, making cloud servers truly equivalent to a physical server with moderate performance. In some general environments, traditional HPC architectures have become inadequate, and in some private scenarios that require on-demand allocation, HPC and cloud computing have begun to converge. People expect to rely on the flexible application mode of cloud computing to solve the system management problems of public HPC platform, and “HPC On Cloud” has become an important measure expected by the industry.
On October 18–20, the HPC China 2018 conference was held in Qingdao. At the conference, Scott Tease, senior director of HPC and AI Technologies of Lenovo Data Center Group (DCG), pointed out in an interview that, “HPC, as the other side of cloud computing, complements the shortcomings of the cloud computing economy, and HPC itself is a technology that can be widely used. If combined with AI, the application will be wider.”
HPC architects are beginning to be willing to sacrifice 5% of application performance in order to achieve the flexibility and flexibility that virtualization brings. You should know that, for a long time, HPC architects have been adamant that performance is everything. Now, their attitudes have changed dramatically. The main reasons for this change are as follows.
First of all, system management, high bandwidth, low latency, massive data processing and so on are the problems that need to be solved by current cloud computing and HPC. Some cloud computing platforms allow higher costs to improve high performance, while HPC also needs to improve its scalability to adapt to the expansion of data.
Second, cloud computing business has covered the needs of HPC, supporting heterogeneous computing. Many cloud service providers have launched HPC products, mainly for in-depth learning applications, but also includes the exploration of scientific computing.
Third, it is possible to share the computing power of HPC through cloud computing now.
In addition, from the dimension of deployment scale, the bottom of cloud computing is also a supercomputer — through the data center operating system, the super large-scale physical servers in different regions are aggregated into a supercomputer, unified scheduling and use.
Obviously, HPC On Cloud is already the trend of the times.
According to IDC data, the global HPC market will reach 15.2 billion US dollars in 2019, and the growth rate of the Chinese market is obvious. HPC will begin to be applied in large data, artificial intelligence and other fields, gradually shifting from scientific research to commercialization. As cloud computing technology can achieve high-performance networking and large-scale GPU cluster services, the speed of technology updating is accelerating, the relationship between the two is bound to be closer, and the future market space is very broad.
With the advent of 5G era, the explosive amount of information is bound to put forward higher requirements for data centers, computers and other aspects. After all, the performance of physical servers is a firm cornerstone for the realization of various intelligent applications on the Internet. For whether cloud computing or HPC, the corresponding data center is the focus of construction, and the comprehensive upgrading of hardware equipment is imminent. For high-performance cloud data center interconnection, Gigalight provides a series of excellent products and solutions to make HPC On Cloud a higher level. For more details, please visit our official website (www.gigalight.com).
