Article

Table of Contents
Theme:
Was this article helpful?

0  out of  2 found this helpful

Try Vultr Today with

$50 Free on Us!

Want to contribute?

You could earn up to $600 by adding new articles.

How to use Rclone with Vultr Object Storage

Author: David Finster

Last Updated: Thu, Nov 3, 2022
Vultr Object Storage

Introduction

Rclone is a command-line program to manage files on cloud storage, with cloud equivalents to Unix commands like rsync, cp, mv, mount, ls, and more. This guide describes how to configure rclone for Vultr Object Storage.

Prerequisites

The rclone commands are cross-platform. This guide shows examples for macOS in zsh, which are compatible with Linux in bash. Windows users will make minor command adjustments for PowerShell or CMD.

Configuration

Locate your rclone configuration file location. This example path below comes from a macOS installation.

$ rclone config file

Configuration file doesn't exist, but rclone will use this path:

/Users/example/.config/rclone/rclone.conf

Create a configuration file.

$ nano /Users/example/.config/rclone/rclone.conf

Paste the following to the file. Replace access_key_id and secret_access_key with your values.

[vultr-ewr1]

type = s3

provider = Vultr

env_auth = false

access_key_id = YOUR_ACCESS_KEY

secret_access_key = YOUR_SECRET_ACCESS_KEY

region =

endpoint = https://ewr1.vultrobjects.com

location_constraint =

acl = private

server_side_encryption =

storage_class =

The acl = line sets the default file permission.

  • private : Owner gets full control. No one else has access. This permission is the default.

  • public-read : Owner gets full control. Everyone has read access.

In general, use private unless you share files via URL. Read more about s3 permissions in the rclone documentation.

Testing

Create a test folder and change to it.

$ mkdir tmp

$ cd tmp

Assuming the tmp folder is empty, create three test files.

$ echo "hello rclone" > test1.txt

$ echo "hello rclone" > test2.txt

$ echo "hello rclone" > test3.txt

Copy one file to the bucket.

$ rclone copy test1.txt vultr-ewr1:testbucket

List the bucket. Notice that test1.txt exists in the bucket and is 13 bytes.

$ rclone ls vultr-ewr1:testbucket

   13 test1.txt

Delete test1.txt from the local system.

$ rm test1.txt

List the bucket. Notice that test1.txt still exists.

$ rclone ls vultr-ewr1:testbucket

   13 test1.txt

Sync the entire folder to the bucket.

$ rclone sync . vultr-ewr1:testbucket

List the bucket. Notice that rclone synced the bucket with the local system by removing test1.txt and adding the other two files.

$ rclone ls vultr-ewr1:testbucket

   13 test2.txt

   13 test3.txt

Sync Two Buckets

In addition to cloning local files to object Storage, rclone can work with two different object storage buckets. Assuming you had two buckets in the vultr-ewr1 object store shown previously, you'd do this:

$ rclone sync vultr-ewr1:my_bucket1 vultr-ewr1:my_bucket2

If you have multiple stores with different keys, declare them in rclone.conf with different labels. You can also use different permissions in the acl= line of each. Here are two different object stores in the New Jersey location with different permissions.

[vultr-ewr1-private]

type = s3

provider = Vultr

env_auth = false

access_key_id = SAMPLE_KEY1

secret_access_key = SAMPLE_SECRET_KEY1

region =

endpoint = https://ewr1.vultrobjects.com

location_constraint =

acl = private

server_side_encryption =

storage_class =



[vultr-ewr1-public]

type = s3

provider = Vultr

env_auth = false

access_key_id = SAMPLE_KEY2

secret_access_key = SAMPLE_SECRET_KEY2

region =

endpoint = https://ewr1.vultrobjects.com

location_constraint =

acl = public-read

server_side_encryption =

storage_class =

To sync bucket1-private with bucket2-public, use rclone like this:

$ rclone sync vultr-ewr1-private:my_pivate_bucket vultr-ewr1-public:my_public_bucket

Sync Two Different Locations

You can also sync buckets between locations. For example, here is a sample configuration with New Jersey (ewr1) and Amsterdam (ams1).

[ewr1]

type = s3

provider = Vultr

env_auth = false

access_key_id = SAMPLE_EWR1_ACCESS_KEY

secret_access_key = SAMPLE_EWR1_SECRET_KEY

region =

endpoint = https://ewr1.vultrobjects.com

location_constraint =

acl = private

server_side_encryption =

storage_class =



[ams1]

type = s3

provider = Vultr

env_auth = false

access_key_id = SAMPLE_AMS1_ACCESS_KEY

secret_access_key = SAMPLE_AMS1_SECRET_KEY

region =

endpoint = https://ams1.vultrobjects.com

location_constraint =

acl = public-read

server_side_encryption =

storage_class =

To sync files between the locations, you'd use rclone like this:

$ rclone sync ewr1:my_new_jersey_bucket ams1:my_amsterdam_bucket

You'll find a full list of our locations and hostnames in our Object Storage documentation.

Troubleshooting

If you encounter HTTP error codes 429, 503, or 504, you may have hit the Vultr Object Storage rate limit. If you encounter this issue, we recommend the following command line options.

More Information

Please refer to the rclone documentation for more information.

Want to contribute?

You could earn up to $600 by adding new articles.