By Yan Hao, OceanData Solution Director, Huawei Storage product line
In recent years, driven by independent innovation and multi-cloud transformation, the financial industry has launched the transformation of traditional databases. In the choice of target database, distributed database has a high voice, but the actual landing situation is not ideal. According to the "Financial industry database Supply Chain Security Development Report" statistics, the use of distributed databases in the banking industry accounted for only 17.5%, in the securities and insurance industry is even less than 4%, and the scale application in the financial core scenario is slow.
What causes this phenomenon? It is true that distributed database paints a beautiful blueprint from the architecture, but there are three technical reasons for the transformation of distributed database in the financial industry to remain at the superficial level:
First, the reliability does not meet the requirements of the financial industry. The importance of financial business is extraordinary, business interruption, data loss will cause immeasurable losses. Most of the distributed database adopts the integrated storage architecture, the server not only provides computing resources, but also is responsible for data storage. Because the server is prone to failure, service interruption and data loss occur at the same time after the fault, which is worse. At present, distributed databases generally use multiple data copies to improve reliability, but new problems arise: multiple copies must maintain strict consistency, so the database performance is bound to be affected; If the data is not strictly consistent, data loss may occur.
Second, the performance gap with the traditional database is too large. At present, the single-machine performance of distributed databases is poor, so it is necessary to split a database into multiple databases in the past and rely on concurrency advantages to improve performance. But the traditional business data amount is large, the structure of the library table is complex, it is difficult to achieve perfect transformation. In addition, during peak hours, excessive access to a local database is likely to cause congestion, which causes the entire system to break down.
Third, the renovation cost is too high, operation and maintenance management is difficult. The transformation of sub-library and sub-table is a systematic project, which involves simultaneous modification of applications, which is time-consuming and laborious. The number of servers and hard drives has soared, and retrofitting costs are high. In addition, unlike professional storage that can provide good hard disk health management capabilities, the risk of server failure often needs to be borne by operations personnel. What is more frightening is that such failures often occur in succession, and you never know which disk will become the "black swan" that triggers the butterfly effect.
For the above reasons, distributed databases have always been difficult to break through in the financial industry. So, what's the solution? Throughout the development of the IT industry, the industry often solves functional problems through software development; For stability and performance problems, it is often broken through hardware technology innovation. The current problem of distributed database is ultimately caused by the lack of reliability of server hardware under the integrated storage and computing architecture. Only rely on multiple copies, sub-library sub-table of these software-level solutions, can not completely solve the problem, improve the hardware capability is the key. Therefore, the reliability of distributed database can be fundamentally solved by upgrading distributed database to storage and computing separation architecture and storing data by more reliable professional storage.
First of all, under the storage separation architecture, the data is stored in highly reliable professional storage, even if the server is damaged data will not be lost, so there is no need for the existence of multiple copies, and the data synchronization between copies naturally no longer exists, which can well solve the performance problem of distributed databases. Secondly, the improvement of storage resource utilization also greatly reduces the number of hard disks, and then the health management of hard disks through professional storage can better eliminate system risks. Thirdly, under the storage separation architecture, the computing and storage resources are untied, and the capacity and computing power are expanded on demand respectively, which can reduce the number of servers from the source, thus reducing the procurement and management costs.
It is always a difficult problem to use distributed database. In essence, the database can make up for the overall reliability and performance loss of the system through the distributed deployment of data, but the cost is high. In fact, by improving the performance of a single database and reducing the system risk after the failure of a single database, it can avoid the sub-database and sub-table. For example, Oracle database ensures data layer availability through Storage separation architecture, allowing servers to access shared data, and realizes shared access to cache data through Automatic Storage Management (ASM). Multiple servers can access a database at the same time. The problem of single library performance and continuity is solved. Coincidentally, Huawei OceanData distributed database storage solution adopts self-developed sky-high database acceleration engine, which also realizes cache sharing between database servers under the storage and computing separation architecture, and is supported by OceanStor Dorado high-performance all-flash storage. Multiple distributed database instances can read and write a library at the same time and handle their own transactions, which greatly improves the overall database performance. Through the storage and computing separation architecture, distributed database can achieve high performance, high reliability, and greatly reduce the use threshold and cost of enterprises.
Huawei OceanData distributed database storage solution is actively adapted to various application scenarios. In the core application scenario, Huawei has created a joint solution between GaussDB and OceanStor Dorado to ensure that data is fully replicated to the redundancy site using active-active storage to ensure that the performance of the redundancy site is not affected. In this way, GaussDB implements dual-cluster Dr For the redundancy site and the redundancy site, meeting the requirements of core financial services. The core business of a large bank is commercialized in a certain country; In the Internet application scenario, Huawei uses the storage separation architecture and self-developed container storage solution to realize the rapid recovery of container applications after server failures, storage failures, and site failures, and works with MySQL database to create excellent practices and achieve landing in a city in the southwest.
Huawei OceanData distributed database storage solution, with the power of professional storage, help distributed database transformation more worry, more assured! Facing the future, Huawei Storage will continue to deepen innovation cooperation with industry partners, promote innovation and upgrading of data inventory calculation separation, comprehensively improve performance, reliability and manageability, and jointly promote the large-scale application of distributed databases in the financial industry.
Disclaimer: The contents and opinions of this article represent the author's own views, for readers to clash ideas and technical communication reference, not as the official basis of Huawei's products and technologies. For details about the products and technologies of Huawei, visit the product and technology introduction page or contact Huawei personnel.
