Difference between revisions of "Rclone"

From SME Server
Jump to navigationJump to search
m (→‎Useage: change TAG from CODE to PRE, to better visual, easy to copy)
 
(3 intermediate revisions by 2 users not shown)
Line 33: Line 33:
 
Current cloud providers supported are:
 
Current cloud providers supported are:
  
    Amazon Drive
+
 
    Amazon S3
+
* Alibaba Cloud (Aliyun) Object Storage System (OSS)
    Backblaze B2
+
* Amazon Drive   (See note)
    Box
+
* Amazon S3
    Ceph
+
* Backblaze B2
    DigitalOcean Spaces
+
* Box
    Dreamhost
+
* Ceph
    Dropbox
+
* DigitalOcean Spaces
    FTP
+
* Dreamhost
    Google Cloud Storage
+
* Dropbox
    Google Drive
+
* FTP
    HTTP
+
* Google Cloud Storage
    Hubic
+
* Google Drive
    Memset Memstore
+
* HTTP
    Microsoft Azure Blob Storage
+
* Hubic
    Microsoft OneDrive
+
* Jottacloud
    Minio
+
* IBM COS S3
    Nextloud
+
* Koofr
    OVH
+
* Memset Memstore
    Openstack Swift
+
* Mega
    Oracle Cloud Storage
+
* Microsoft Azure Blob Storage
    Ownloud
+
* Microsoft OneDrive
    pCloud
+
* Minio
    put.io
+
* Nextcloud
    QingStor
+
* OVH
    Rackspace Cloud Files
+
* OpenDrive
    SFTP
+
* Openstack Swift
    Wasabi
+
* Oracle Cloud Storage
    WebDAV
+
* ownCloud
    Yandex Disk
+
* pCloud
    The local filesystem
+
* put.io
 
+
* QingStor
 +
* Rackspace Cloud Files
 +
* Scaleway
 +
* SFTP
 +
* Wasabi
 +
* WebDAV
 +
* Yandex Disk
 +
* The local filesystem
 
I have only tested it against Dropbox.
 
I have only tested it against Dropbox.
  
Line 73: Line 80:
 
<pre>
 
<pre>
 
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
 
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
 
 
unzip rclone-current-linux-amd64.zip
 
unzip rclone-current-linux-amd64.zip
 
 
cd rclone-*-linux-amd64
 
cd rclone-*-linux-amd64
 
 
sudo cp rclone /usr/bin/
 
sudo cp rclone /usr/bin/
 
 
sudo chown root:root /usr/bin/rclone
 
sudo chown root:root /usr/bin/rclone
 
 
sudo chmod 755 /usr/bin/rclone
 
sudo chmod 755 /usr/bin/rclone
 
 
</pre>
 
</pre>
  
Line 102: Line 103:
 
<pre>
 
<pre>
 
!/bin/sh
 
!/bin/sh
 
 
signal-event pre-backup
 
signal-event pre-backup
 
 
cd /
 
cd /
 
 
rclone mkdir smeserver-dropbox:backup
 
rclone mkdir smeserver-dropbox:backup
 
 
rclone mkdir smeserver-dropbox:backup/smeserver
 
rclone mkdir smeserver-dropbox:backup/smeserver
 
 
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom
 
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom
 
 
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom
 
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom
 
 
rclone mkdir smeserver-dropbox:backup/smeserver/home
 
rclone mkdir smeserver-dropbox:backup/smeserver/home
 
 
rclone mkdir smeserver-dropbox:backup/smeserver/ssh
 
rclone mkdir smeserver-dropbox:backup/smeserver/ssh
 
 
rclone mkdir smeserver-dropbox:backup/smeserver/root
 
rclone mkdir smeserver-dropbox:backup/smeserver/root
 
 
rclone copy -v  / smeserver-dropbox:backup/smeserver --files-from backup-files-list-rclone
 
rclone copy -v  / smeserver-dropbox:backup/smeserver --files-from backup-files-list-rclone
 
 
rclone copy -v  "etc/e-smith/templates-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom/"
 
rclone copy -v  "etc/e-smith/templates-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom/"
 
 
rclone copy -v  "etc/e-smith/templates-user-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom/"
 
rclone copy -v  "etc/e-smith/templates-user-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom/"
 
 
rclone copy -v  etc/ssh smeserver-dropbox:backup/smeserver/ssh
 
rclone copy -v  etc/ssh smeserver-dropbox:backup/smeserver/ssh
 
 
rclone copy -v  root smeserver-dropbox:backup/smeserver/root
 
rclone copy -v  root smeserver-dropbox:backup/smeserver/root
 
 
rclone copy -v  home/e-smith/ smeserver-dropbox:backup/smeserver/home/ --exclude "tmp/**"
 
rclone copy -v  home/e-smith/ smeserver-dropbox:backup/smeserver/home/ --exclude "tmp/**"
 
 
</pre>
 
</pre>
  
Line 172: Line 157:
  
 
<!-- list of category you want to see this page in -->
 
<!-- list of category you want to see this page in -->
[[Category: Contrib]]  
+
[[Category: Howto]]  
 
[[Category: rclone]]  
 
[[Category: rclone]]  
 
[[Category: backup]]
 
[[Category: backup]]
 
<!-- Please keep there the template revision  number as is -->
 
<!-- Please keep there the template revision  number as is -->

Latest revision as of 07:11, 7 November 2021




rclone
NeedImage.svg
rclone logo
Maintainerbrianr
Urlhttps://wiki.contribs.org
LicenceGPL
Category

cli

Tags clicentosSMEinstall


Warning.png Warning:
Only for testing purpose, please report in bugzilla


Maintainer

Brian Read

Version

Requires command level knowledge to install and run.

Description

Rclone is a program written in Go to allow local files to be saved to one of many cloud servers. It tries to emulate rsync.

Current cloud providers supported are:


  • Alibaba Cloud (Aliyun) Object Storage System (OSS)
  • Amazon Drive (See note)
  • Amazon S3
  • Backblaze B2
  • Box
  • Ceph
  • DigitalOcean Spaces
  • Dreamhost
  • Dropbox
  • FTP
  • Google Cloud Storage
  • Google Drive
  • HTTP
  • Hubic
  • Jottacloud
  • IBM COS S3
  • Koofr
  • Memset Memstore
  • Mega
  • Microsoft Azure Blob Storage
  • Microsoft OneDrive
  • Minio
  • Nextcloud
  • OVH
  • OpenDrive
  • Openstack Swift
  • Oracle Cloud Storage
  • ownCloud
  • pCloud
  • put.io
  • QingStor
  • Rackspace Cloud Files
  • Scaleway
  • SFTP
  • Wasabi
  • WebDAV
  • Yandex Disk
  • The local filesystem

I have only tested it against Dropbox.

Installation

Rclone is made available as a statically linked executible and can be installed as follows: (https://rclone.org/install/)

curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
unzip rclone-current-linux-amd64.zip
cd rclone-*-linux-amd64
sudo cp rclone /usr/bin/
sudo chown root:root /usr/bin/rclone
sudo chmod 755 /usr/bin/rclone

Note that this loads the 64 bit version, other versions are available.

Then you can run the configuration process:

rclone config

Choose your cloud provider and follow the instructions for a headless server.

For Dropbox I had to run rclone on a desktop with a browser, giving permission for rclone to be allowed to connect to Dropbox and then copy and paste a key string across to the smeserver.

Useage

Here is the shell script that I developed and tested to see if it would work:

!/bin/sh
signal-event pre-backup
cd /
rclone mkdir smeserver-dropbox:backup
rclone mkdir smeserver-dropbox:backup/smeserver
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom
rclone mkdir smeserver-dropbox:backup/smeserver/home
rclone mkdir smeserver-dropbox:backup/smeserver/ssh
rclone mkdir smeserver-dropbox:backup/smeserver/root
rclone copy -v  / smeserver-dropbox:backup/smeserver --files-from backup-files-list-rclone
rclone copy -v  "etc/e-smith/templates-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom/"
rclone copy -v  "etc/e-smith/templates-user-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom/"
rclone copy -v  etc/ssh smeserver-dropbox:backup/smeserver/ssh
rclone copy -v  root smeserver-dropbox:backup/smeserver/root
rclone copy -v  home/e-smith/ smeserver-dropbox:backup/smeserver/home/ --exclude "tmp/**"

Incremental Backup

holk has documented a use of rclone that will give an incremental backup for up to 7 days (or whatever you might want)

See here:

https://forums.contribs.org/index.php/topic,53562.msg278216.html#msg278216

Encrypted

Rclone allows the remote data to be encrypted by specifying an intermediate remote which does the encrypting.

See here.

I have not tried this, but it looks pretty easy to add in and does not involve the SMEServer software.

Restore

I assume that you can restore by just copying the data back from the cloud provider and then doing a signal-event post-upgrade; signal-event reboot;

Uninstall

Just delete /usr/bin/rclone and the config file /root/.config/rclone/rclone.conf

Bugs

Please raise bugs under the SME-Contribs section in bugzilla and select the smeserver-rclone component or use this link


Below is an overview of the current issues for this contrib:

No open bugs found.

Changelog

Only released version in smecontrib are listed here.