Descriptions of rclone often carry the strapline "Rclone syncs your files to cloud storage".[8] Those prior to 2020 include the alternative "Rsync for Cloud Storage".[9]
Rclone is well known for itsrclone sync andrclone mount commands.[10] It provides further management functions analogous to those ordinarily used for files on local disks, but which tolerate some intermittent and unreliable service. Rclone is commonly used withmedia servers such asPlex,[11]Emby orJellyfin[12] to stream content direct from consumerfile storage services.[11]
Nick Craig-Wood was inspired byrsync.[14][9] Concerns about the noise and power costs arising from home computer servers prompted him to embrace cloud storage and he began developing rclone as open source software in 2012 under the nameswiftsync.[15][16][3]
Rclone was promoted to stable version 1.00 in July 2014.[6]
In May 2017,Amazon Drive barred new users of rclone and other upload utilities, citing security concerns.[17] Amazon Drive had been advertised as offering unlimited storage for £55 per year. Amazon's AWS S3 service continues to support new rclone users.
The original rclone logo was updated in September 2018.[18]
In March 2020, Nick Craig-Wood resigned from Memset Ltd, a cloud hosting company he founded, to focus on open source software.[19][16][20][21]
Amazon's AWS April 2020 public sector blog explained how theFred Hutch Cancer Research Center were using rclone in their Motuz tool to migrate very large biomedical research datasets in and out of AWS S3 object stores.[22]
In November 2020, rclone was updated to correct a weakness in the way it generated passwords. Passwords for encrypted remotes can be generated randomly by rclone or supplied by the user. In all versions of rclone from 1.49.0 to 1.53.2 the seed value for generated passwords was based on the number of seconds elapsed in the day, and therefore not truly random.CVE-2020-28924 recommended users upgrade to the latest version of rclone and check the passwords protecting their encrypted remotes.[23]
Release 1.55 of rclone in March 2021 included features sponsored byCERN and their CS3MESH4EOSC project.[24] The work wasEU funded to promote vendor-neutralapplication programming interfaces and protocols for synchronisation and sharing of academic data on cloud storage.[25]
Rclone supports the following services as backends. There are others, built on standard protocols such asWebDAV or S3, that work.[8] WebDAV backends do not support rclone functionality dependent on server side checksum or modtime.[26]
Remotes are usually defined interactively from these backends, local disk, or memory (as S3), withrclone config. Rclone can further wrap those remotes with one or more ofalias,chunk,compress,crypt orunion, remotes.
Once defined, the remotes are referenced by other rclone commands interchangeably with the local drive. Remote names are followed by a colon to distinguish them from local drives. For example, a remoteexample_remote containing a folder, or pseudofolder,myfolder is referred to within a command as a pathexample_remote:/myfolder.[36]
Rclone commands directly apply to remotes, or mount them for file access or streaming. With appropriate cache options the mount can be addressed as if a conventional,block level disk. Commands are provided to serve remotes overSFTP,HTTP,WebDAV,FTP andDLNA.[8] Commands can have sub-commands and flags. Filters determine which files on a remote that rclone commands are applied to.[37]
rclone rc passes commands or new parameters to existing rclone sessions and has an experimental web browser interface.[38]
Rclone's crypt implements encryption of filesat rest in cloud storage. It layers an encrypted remote over a pre-existing, cloud or other remote. Crypt is commonly[11] used to encrypt / decrypt media, for streaming, on consumer storage services such asGoogle Drive.
Rclone's configuration file contains the crypt password. The password can be lightly obfuscated, or the whole rclone.conf file can be encrypted.[39]
Crypt can either encrypt file content and name, or additionally full paths. In the latter case there is a potential clash with encryption for cloud backends, such asMicrosoft OneDrive, having limited path lengths.[40] Crypt remotes do not encrypt object modification time or size. The encryption mechanism for content, name and path is available, for scrutiny, on the rclone website. Key derivation is withscrypt.[39]
These examples describe paths and file names but object keys behave similarly.
To recursively copy files from directoryremote_stuff, at the remotexmpl, to directorystuff in the home folder:-
$rclonecopy-v-Pxmpl:/remote_stuff~/stuff
-v enables logging and-P, progress information. By default rclone checks the file integrity (hash) after copy; can retry each file up to three times if the operation is interrupted; uses up to four parallel transfer threads, and does not apply bandwidth throttling.
Running the above command again copies any new or changed files at the remote to the local folder but, like default rsync behaviour, will not delete from the local directory, files which have been removed from the remote.
To additionally delete files from the local folder which have been removed from the remote - more like the behaviour of rsync with a--delete flag:-
$rclonesyncxmpl:/remote_stuff~/stuff
And to delete files from the source after they have been transferred to the local directory - more like the behaviour of rsync with a--remove-source-file flag:-
$rclonemovexmpl:/remote_stuff~/stuff
To mount the remote directory at a mountpoint in the pre-existing, emptystuff directory in the home directory (the ampersand at the end makes the mount command run as a background process):-
$rclonemountxmpl:/remote_stuff~/stuff&
Default rclone syntax can be modified. Alternative transfer, filter, conflict and backend specific flags are available. Performance choices include number of concurrent transfer threads; chunk size; bandwidth limit profiling, and cache aggression.[37]
This section maylendundue weight to the idea that rclone is used for cybercrime, when most of the time it is used for innocuous purposes. Pleasehelp improve it by rewriting it tocreate a more balanced presentation. Discuss andresolve this issue before removing this message.(January 2023)
May 2020 reports stated rclone had been used byhackers to exploitDiebold NixdorfATMs with ProLockransomware.[76][77] TheFBI issued a Flash Alert MI-000125-MW on May 4, 2020, in relation to the compromise.[78] They issued a further, related alert 20200901–001 in September 2020. Attackers had exfiltrated / encrypted data from organisations involved in healthcare, construction, finance, and legal services. Multiple US government agencies, and industrial entities were affected. Researchers established the hackers spent about a month exploring the breached networks, using rclone to archive stolen data to cloud storage, before encrypting the target system.[78][79][80] Reported targets includedLaSalle County, and the city ofNovi Sad.[81][82]
The FBI warned January 2021, in Private Industry Notification 20210106–001, of extortion activity using Egregor ransomware and rclone. Organisations worldwide had been threatened with public release of exfiltrated data. In some cases rclone had been disguised under the namesvchost.[83] BooksellerBarnes & Noble, US retailerKmart, games developerUbisoft and theVancouver metro system have been reported as victims.[84]
A June 2021,Microsoft Security IntelligenceTwitter post identified use of rclone in BazaCall cyber attacks.[88] The attackers sent emails encouraging recipients to contact a fakecall centre to cancel a paid for service. The call centre team then instructed victims to download a hostile file that installed malware on the target network, ultimately allowing use of rclone for covert extraction of potentially sensitive data.
In a 2021,Star Wars Day blog article,Managed Security Service Provider Red Canary announcedRclone Wars, anallusion toClone Wars. The post notes illicit use of other legitimate file transfer utilities in exfiltrate and extort schemes but focuses onMEGAsync, MEGAcmd and rclone. To identify use of renamed rclone executables on compromised devices the authors suggest monitoring for distinctive rclone top level commands and command line flag strings such asremote: and\\.[89]
Rsync transfers files with other computers that have rsync installed.[90] It operates at theblock, rather than file, level and has adelta algorithm so that it only needs to transfer changes in files. Rsync preserves file attributes and permissions. Rclone has a wider range of content management capabilities, and types of backend it can address, but only works at a whole file / object level.[91][3] It does not currently preserve permissions and attributes.[92] Rclone is designed to have some tolerance of intermittent and unreliable connections or remote services. Its transfers are optimised for high latency networks. Rclone decides which of those whole files / objects to transfer after obtaining checksums, to compare, from the remote server. Where checksums are not available, rclone can use object size and timestamp.[93]
Rsync is single threaded.[94] Rclone is multi threaded with a user definable number of simultaneous transfers.[95][96]
Rclone can pipe data between two completely remote locations, sometimes without local download. During an rsync transfer, one side must be a local drive.[90][96]
Rclone ignores trailingslashes. Rsync requires their correct use.[97] Rclone filters require the use of** to refer to the contents of a directory. Rsync does not.[98]
In 2016, a poster onHacker News summarised rclone's relationship to rsync as:-(rclone) exists to give you rsync to things that aren't rsync. If you want to rsync to things that are rsync, use rsync.[101][102]
^abcdefg"Rclone".rclone.Archived from the original on June 17, 2020. RetrievedJuly 29, 2020.
^ab"rclone/rclone". July 31, 2020.Archived from the original on June 30, 2020. RetrievedJuly 29, 2020 – via GitHub.
^Tozzi, Christopher (September 25, 2020)."How to Access S3 Buckets from Windows or Linux".Itpro Today: It News, How-Tos, Trends, Case Studies, Career Tips, More.Archived from the original on October 29, 2020. RetrievedSeptember 27, 2020.