Mounting the File Fabric with Rclone in Linux

RClone Mount File FabricAs per per our prior blog post  Rclone v1.54 now supports the Enterprise File Fabric (EFF) v2006.03 for copying, moving, syncing and listing of files/objects.

Did you know Rclone also supports mapping drives in Windows or mounting in OSX/Linux?

Mounting/mapping drives can be useful for end users navigating storage from GUI based tools like Windows Explorer and Mac Finder.  The File Fabric already ships easy to install drive-integrated tools for transparent  end-user access Mac/Linux/Windows, so what use case would prompt a company to use an Rclone mount with the File Fabric instead?

Well, sometimes a system use case requires headless access to a drive or mount point which can be the case for Linux or Windows  server installations.

Mounting Rclone with the File Fabric on Linux

First setup and install Rclone per the instructions in the article.

Next, ensure a mount point is configured on Linux.

For this example I will mount to /net/rclone.

mkdir -p /net/rclone 

rclone mount --vfs-cache-mode full --write-back-cache --vfs-write-back 10s --vfs-cache-max-age 300s --vfs-cache-max-size 5000M --vfs-cache-poll-interval 60s --cache-dir /tmp sme: /net/rclone

This will run the mount in the foreground.  You will need to open another terminal to test reading and writing from the mount point.  The benefit of running in foreground mode is that any errors will appear in the terminal running the mount command.  This allows for proper testing of the options passed in to run rclone and adjustments.   Once the proper testing has been completed, the addition flag –daemon can be added to the end of the mount command to allow the mount to run in background.

When running with the –daemon option, the mount should be stopped with:

fusermount -u /net/rclone

Rclone mount suggested options

We suggest the following Rclone mount options to provide an optimal experience.

–vfs-cache-mode full Cache is required for writing data to the Enterprise File Fabric
–write-back-cache (Optional) Increases performance by allowing writes to be acknowledged to applications after successful write to cache.
–vfs-write-back 10s Waits 10 seconds after a file is closed in cache before uploading to EFF.  This minimizes the chance of starting an upload of a file which is still being actively write to.
–vfs-cache-max-age 300s This sets a 300 second max age of items in the cache.  Adjust as needed, longer cache times result in larger cache, but possibly increased performance.
–vfs-cache-max-size 5000M (Optional) Sets the max size of the cache to prevent cache from filling disk.  Cache needs to be larger than the largest file the mount will need to work with.
–vfs-cache-poll-interval 60s How often rclone checks the cache for stale files.
–cache-dir /tmp (Optional) Sets the cache location.  /tmp is suggested, as /tmp is cleared on reboots in many distributions
sme: This is the name of the rclone remote.  To mount a specific directory use [remote]:dir1/subdir2/
/net/rclone Mount point
–daemon Run in background

Additional notes:

Rclone has a number of flags and some may give enhanced performance with regards to tiering a lot of small files.  The –transfers flag can allow more parallel data movement, the default is 4, but moving lots of small files may necessitate the changing of this value to 10 or 20.

Rclone mount in use with the Enterprise File Fabric should be limited to moving data to/from a remote linux workstation.  Rclone mount should not be used to copy/move data between remote providers as all data will be pulled through the workstation.  Large data movements within and between providers should be done utilizing the Enterprise File Fabric File Manager or one of the Cloud tool packages, as these tools utilize the File Fabric API with the built in file transfer acceleration feature,  M-Stream.

In our next post on Rclone  and the File Fabric we will outline how the File Fabric and RCcone can be use with Windows.

Facebooktwitterredditpinterestlinkedinmailby feather
The following two tabs change content below.

Douglas Soltesz

Director Product Solutions at Storage Made Easy
Doug's focus is in Object and Cloud Storage APIs, Data Governance, Virtualization, and Containerization.