Introduction

with the rapid development of big data, cloud computing, artificial intelligence and other technologies, the demand for data storage has exploded. High performance, high reliability and scalability have become important goals pursued by modern storage systems. In this context, SPDK(Storage Performance Development Kit) emerged as a revolutionary Storage Performance Development Kit. This article will introduce the concept, advantages and future challenges of SPDK in detail.

I. INTRODUCTION to SPDK

1.1 What is SPDK? Why do I need SPDK?

SPDK, known as Storage Performance Development Kit, is a set of tools and libraries for writing high-Performance and scalable user-mode Storage applications. It achieves high performance through a series of key technologies, including moving all necessary drivers to user space, avoiding system calls and enabling zero copy access to applications, polling hardware to complete I/O operations, reduce the total latency and latency difference; Avoid all locks in the I/O path, but rely on message delivery.

1.2 key technologies of SPDK

as SSDs, especially NVMe-based SSDs, become more and more popular in data centers, the performance and efficiency of storage systems become crucial. NVMe devices provide higher throughput and lower latency than traditional SAS or SATA hard disks, which makes them thousands of times faster in IOPS (input/output operations per second) than mechanical hard disks, 5 to 10 times faster than the previous SATA SSD. However, traditional storage software stacks often become bottlenecks when processing these high-performance devices, because the time consumption of software in IO transactions is too much compared with the performance of NVMe devices.

SPDK is designed to solve this problem. By implementing drivers and I/O processing in user space, it avoids performance bottlenecks in traditional storage systems. Features of SPDK include:

user space driver: SPDK moves drivers from kernel space to user space, avoiding data copy and context switching between kernel and user states, thus improving the efficiency of I/O operations.

Asynchronous I/O processing: SPDK uses an asynchronous I/O model to process I/O requests, allowing applications to perform other tasks while waiting for I/O completion, improving the concurrent processing capability of the system.

Lock-free design: SPDK uses a lock-free design to manage shared resources, reducing competition and deadlocks in concurrent operations, and improving system stability and scalability.

Block stack library: SPDK provides a complete block stack library, including file systems, volume managers, and block device drivers, simplifying the development of storage applications.

Network Storage Support: SPDK supports multiple network storage protocols such as NVMe-oF, iSCSI, and vhost, allowing block storage services to be provided over the network.

Hot Plug and failover: SPDK supports online hot plug and failover of storage devices, improving the reliability and availability of the system.

Performance monitoring and analysis tools: SPDK provides performance monitoring and analysis tools to help developers diagnose and optimize the performance of storage applications.

1.3 application scenarios of SPDK

SPDK supports multiple application scenarios, such as NVMe-oF, iSCSI, and vhost servers. These servers can provide disks over the network or other processes, with efficient CPU efficiency. SPDK targets can be used as examples of how to achieve high-performance storage targets or as a basis for production and deployment.

iSCSI Target: iSCSI Target is a server that provides block storage over the network. It supports the standard iSCSI protocol and allows clients to access remote storage over the TCP/IP network. iSCSI Target is applicable to scenarios where storage resources need to be accessed through IP networks, such as cloud computing, virtualization, and remote backup.

NVMe over Fabrics (NVMe-oF) Target: NVMe-oF Target is a server that provides NVMe storage over the network. It supports NVMe-oF protocol and allows clients to access NVMe storage over IP networks. NVMe-oF Target is applicable to scenarios that require high-performance and low-latency storage resources, such as high-performance computing, big data processing, and cloud computing.

vhost Target: vhost Target is a server used to provide virtualized storage over the network. It supports vhost and virtio protocols and allows virtual machines and containers to directly access storage resources. vhost Target is applicable to scenarios where storage resources need to be virtualized, such as cloud computing and virtualized environments.

SPDK Target: SPDK Target is a unified application that combines iSCSI, NVMe-oF, and vhost functions. It provides a high-performance, scalable storage solution that is suitable for scenarios where multiple storage protocols and virtualization technologies are supported at the same time.

II. Advantages of SPDK

2.1 Performance advantages

One of the core advantages of SPDK is its significant performance improvement. By migrating drivers from kernel space to user space, SPDK effectively avoids performance bottlenecks in traditional storage systems. Implementing drivers in user space allows SPDK to interact directly with hardware, avoiding the overhead of Data copy and context switching between user space and kernel space. This design not only reduces the number of system calls, but also speeds up I/O operations.

SPDK uses polling mode instead of dependent interrupt to complete I/O operations, which reduces the difference between total latency and latency. In traditional storage systems, interrupt processing may introduce additional latency, while SPDK monitors device status in real time through polling mechanism, achieving faster response time and lower latency.

In addition, the lock-free design and parallel processing mechanism of SPDK are the key to its performance advantages. The lock-free design eliminates lock competition in concurrent operations, thus reducing performance bottlenecks. The parallel processing mechanism allows SPDK to process multiple I/O requests simultaneously, greatly improving the concurrency of the system and enabling it to efficiently handle high loads in large-scale parallel processing scenarios.

2.2 reliability and scalability

SPDK's block stack library provides a unified storage device interface, which simplifies storage management and makes it easier for developers to interact with different types of storage devices. SPDK supports multiple storage device types, such as NVMe and virtio, which not only improves system compatibility, but also enhances its scalability. Developers can easily integrate and use different types of storage devices in the SPDK environment to adapt to the changing hardware environment.

The SPDK design also considers high availability. It supports hot plug and failover, which means that the system can continue to operate even when the storage device fails, ensuring data security and system continuity. These features improve the reliability and availability of the SPDK system, enabling it to meet the strict requirements of data centers and enterprise-level applications.

2.3 ease of use and development efficiency

another significant advantage of SPDK is its ease of use and development efficiency. SPDK provides a wide range of APIs and libraries. These tools and libraries provide developers with convenient interfaces to quickly build high-performance storage applications, without having to understand the complexity of underlying hardware and kernel drivers. This abstraction layer simplifies the development process, reduces the development difficulty, and enables developers to focus more on the implementation of application functions.

References

[1] SPDK official document Chinese version (August 2019 edition)

[2] Wang Hailiang. SPDK official document Chinese version (August 2019 edition) translation.

[3] Storage Performance Development Kit (SPDK) [EB/OL]. (2020-04-20)[2023-03-28]. https://spdk.io/

[4] SPDK: Storage Performance Development Kit [EB/OL]. (2020-04-20)[2023-03-28]. https://github.com/spdk/spdk