01 Data storage

blocks, files, and objects are three storage formats for storing, organizing, and presenting data in different ways. Each of these methods has its own characteristics and limitations. It cannot be absolutely said that one storage format is superior to the other. Instead, it should be appropriately selected according to the use scenario to achieve performance, security, reliability, balance between user experience and cost.

02 block storage, file storage, object storage

block Storage

block storage is a technology used to store data into blocks. These blocks are then stored as separate parts, each part has a unique identifier, so the storage system can store smaller data in the most appropriate location. Developers tend to use block storage for computing scenarios that require fast, efficient, and reliable data transmission.

Block storage is used as a bare disk for servers or cloud hosts. Access to block storage needs to be implemented through a block driver. Generally, it needs to be formatted into a file system before it can be used, traditional SAN and DAS use block storage.

File storage

file storage, also known as file-level or file-based storage, is a hierarchical storage method for organizing and storing data. In other words, data is stored in files, files are organized in folders, and folders are organized under the hierarchy of directories and subdirectories.

Object Storage

object Storage, commonly known as object-based storage, is a data storage architecture for processing large amounts of unstructured data. These data cannot be easily organized into traditional relational databases with rows and columns or do not meet their requirements. Specific examples include PDF, photos, audio, video, Web pages, and other types of media and Web content (text or non-text).

03 advantages and disadvantages of three types of storage

block Storage

advantages

quick Search: most block storage technologies can search for data by retrieving instructions in memory, thus making data search faster.

Save storage space: if data is stored in a block storage system, the system will divide the data blocks into smaller blocks, thus reducing the storage space of the entire system to save storage costs.

Easy to manage: block storage technology can read and write data through instructions in memory, so that data can be classified, sorted, and filtered as needed, this makes it easier to manage and search data.

Better Reliability: block storage technology can use smaller hard disk drives, thus reducing the failure rate of hard disks, reducing the maintenance of hard disks and more frequent data read and write operations.

Disadvantages

using SAN architecture for networking, you need to purchase additional optical fiber channel cards and optical fiber switches, which is costly.

It is difficult to share data between hosts of different hosts and different operating systems.

File storage

advantages

simple: file storage is the simplest, most familiar, and most direct way to organize files and folders on a computer hard drive or NAS device. Simply name the files, mark them with metadata, and store them in folders under the Directory and subdirectory hierarchy. You do not need to write applications or code to access data.

File Sharing: file storage is an ideal choice for centralizing and sharing files within a local area network (LAN). Any computer with appropriate permissions on the network can easily access files stored on NAS devices.

Common protocols: file storage uses common file-level protocols, such as Server Message Block (SMB), Universal Internet File System (CIFS), or Network File System (NFS). If you use a Windows or Linux operating system, standard protocols such as SMB/CIFS and NFS allow you to read and write files to Windows or Linux-based servers over a local area network (LAN).

Data protection: storing files on a separate storage device connected to a LAN can provide a certain degree of data protection in the event of a network computer failure. Cloud-based file storage provides additional data protection and disaster recovery capabilities by copying data files to data centers distributed across multiple geographic locations.

Cost-effective: file storage using NAS devices helps transfer files from expensive computing hardware to cheaper LAN-connected storage devices. In addition, if you choose to order cloud file storage services, you can eliminate the local hardware upgrade costs and related continuous maintenance and operation costs.

Disadvantages

slow data transmission and read/write

object Storage

advantages

scalability: unlimited scaling is probably the most significant advantage of object-based data storage. Objects, that is, any number of independent data units, can be stored in a flat data environment in storage devices such as servers. You only need to add more devices/servers to the object storage cluster in parallel to perform additional processing and support the higher throughput required for large files such as videos or images.

Reduce complexity: the complexity of a hierarchical file system that contains folders and directories can be solved by object storage. Because you do not need to browse folders, directories, or complex hierarchies, this reduces the possibility of performance latency and improves the efficiency of data retrieval. This improves performance, especially when managing large amounts of data.

Availability: you can configure an object storage system to replicate content. If a disk in the cluster fails, you can use the copy disk to ensure that the system continues to run without interruption or performance degradation. Data can be replicated within nodes and clusters as well as between distributed data centers for additional backups in off-site or across geographic areas.

Disadvantages

to modify an object, you must write the object completely at one time.

It cannot be used well with traditional databases.

04 application scenario

block Storage: Databases, ETL, high-performance computing, operating system storage, and startup volumes

file storage: files are shared across multiple compute instances.

Object Storage: large, scalable, and persistent storage, disaster recovery, and archiving of different objects, such as images, audio, and video.

05 how to use

block Storage

for example, if we have a separate disk or cloud disk, run the fdisk command to check it:

1 >fdisk -l

2> disk/dev/vdb:500GB, 322122547200 bytes, 629145600 fans

at this time, the disk has not been mounted. We use the mount command to mount the disk to the/home/apps Directory of the system so that it can be used.

1 > mount /dev/vdb /home/apps

2 >df -h

3 File system capacity used available used% mount point

4 /dev/vdb 500G 0G 500G 0% /home/apps

file storage

after the disk is mounted, you need to install the file system first if you want to use it. We use mkfs to format it:

1 > mkfs -t ext4 /dev/vdb

after formatting, we use the mount command to mount the file system. After mounting, we can operate on the file system.

1 > cd /home/apps/

2 > mkdir a B c d

3 > ls

4 a B c d

file storage organizes files in a tree-like manner and requires recursive traversal every time. Therefore, if the file directory is deep, the search speed is slow.

Object Storage

object storage is to store data in a bucket as an object and give it a unique identifier for access. It has a directory structure, but the actual storage method is flat, therefore, the search efficiency is very high. Object Storage usually provides a set of RESTFul APIs for developers to use. You can use PUT to upload objects and GET to obtain objects.

Upload:

1 PUT /ObjectName HTTP/1.1

2 Content-Length:ContentLength

3 Content-Type: ContentType

4 Host: 127.0.0.1Date: GMT

5 Date

6 Transfer-Encoding: chunked

get:

1GET /ObjectName HTTP/1.1

2 Host: 127.0.0.1

3 Date: GMT Date

summary

block Storage: storage reads and writes are fast, which is not conducive to sharing.

File storage: storage reads and writes are slow for sharing

object Storage: fast storage read/write for sharing

the article is reprinted from: Huaxin intelligent public account