Case Study: Accelerating File Transfers from Tier 1 Storage to Tier 2 Object Storage

This user case study focuses on a top tier Media company which has grown from a small digital distribution company in Los Angeles to a global media services, distribution, and software solutions company, aggregating and delivering film and television digital assets on behalf of major studios, broadcasters and streaming providers to more than 600 platforms around the world.

This media company initially tested the Enterprise File Fabric’s M-Stream feature to restore data from Tier 2 Object Storage to an EMC Isilon storage system and it measured 5-6 X throughput increase compared to their existing file transfer client Apps.

Test Setup

    • Enterprise File Fabric 1803.04 16GB RAM, 12CPU, 10GB Network deployed on VMware vSphere 6.5.
    • Tier 2 Storage: Object Storage – 12 Petabyte Raw, Data encoded using Erasure Coding (ISA-L) on repurposed Dell hardware.
    • Tier 1 Storage: Dell EMC Isilon. Isilon Write Block size 512KB, Read 128 KB.
    • Test File Size 120GB to 1TB.

Results

    • Throughput: Copying files to/from Tier 2 (Object Storage) and Tier 1 (Isilon) storage: 700-770MBps.
    • Single stream of data from Object Storage to Isilon 100 – 150MBps (Using existing client Apps).

Results Summary

The Enterprise File Fabric aggregated concurrent connections from the Object Storage to the Isilon resulting in a 5-6X throughput increase on a single large file.

Typically, throughput increases of up to 10X are observed in data transfers between object storage (nearline) tiers of data and file systems.

In any data transfer, factors such as CPU cores, network line speed, and number of HDD spindles will determine the max transfer rate.

In this particular instance the Isilon proved to be the limiter on achieving better results.

This case study is available as a PDF.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather
The following two tabs change content below.