Repository files navigation

⚠️ This project is in beta stage.

If you are not sure whether this project fits your need, then it does not. Youare probably looking forOneDrive Online or sync and FUSE implementationslikerclone.

This project may be helpful for 🐧real nerds 🐧 who enjoywacky block device stacking, intend to leverage block level encryption or theirexisting BTRFS backup infrastructure, or explore fresh new bugs in BTRFS zonedmode, with the cost ofeverything.

System requirements:

• Linux >= 5.19 is required for io-uring withIORING_SETUP_SQE128 support.

• Kernel driverublk_drv and zoned block device support should be enabled.Most distributions like Arch Linux and NixOS unstable meet these requirementsby default. You can check your system by:

  $zgrep -E'CONFIG_BLK_DEV_UBLK|CONFIG_BLK_DEV_ZONED' /proc/config.gzCONFIG_BLK_DEV_ZONED=yCONFIG_BLK_DEV_UBLK=m

  If you see the same result, your kernel is probably supported.

• You may need to runsudo modprobe ublk_drv manually to load the driverfirst. This is not required for running orb in the shipped systemd service orvia NixOS module, which does this automatically.

This project is packaged in Nix flake. Here's the simplified output graph:

├───nixosModules│   ├───default: Alias to `orb`.│   └───orb: The NixOS module.└───packages    ├───x86_64-linux    │   ├───default: Alias to `orb`.    │   ├───orb: The main program with systemd units.    │   ├───cryptsetup-format-zoned: workaround script for cryptsetup-luksFormat on zoned devices.    │   └───ublk-chown-unprivileged: The optional utility for unprivileged ublk.    [..more Linux platforms are supported..]

# Example flake.nix for demostration. Please edit your own one to add changes.{inputs.nixpkgs.url="github:NixOS/nixpkgs/nixos-unstable";inputs.orb.url="github:oxalica/orb";outputs={nixpkgs,orb, ...}:{nixosConfigurations.your-system=nixpkgs.lib.nixosSystem{system="x86_64-linux";modules=withnixosModules;[orb.nixosModules.orb./path/to/your/configuration.nix];};};}

{ ...}:{services.orb.instances={# The instance name. It coresponds to the systemd service# `orb@my-device.service`. By default it will not be automatically started."my-device".settings={# Required device id. It's recommended to start at 80.# This creates block device `/dev/ublkb80`.ublk.id=80;# Other settings and their defaults can be seen in# ./contrib/config-onedrive.example.tomldevice={dev_size="1TiB";zone_size="256MiB";min_chunk_size="1MiB";max_chunk_size="256MiB";};backend.onedrive.remote_dir="/orb";};};# If you want to mount the block device, you can create systemd mounts.# This is an example.systemd.mounts=[{type="btrfs";# Fill in your filesystem UUID after mkfs.what="/dev/disk/by-uuid/11111111-2222-3333-4444-555555555555";where="/mnt/my-mount-point";# Do not forget dependencies.requires=["orb@my-device.service"];after=["orb@my-device.service"];# It's recommended to set `noatime` and `compress` to reduce write# frequency and amplification.options="noatime,compress=zstd:7";}];}

You need following dependencies to be installed with your package manager:

• Rust >= 1.76
• pkg-config
• openssl

Build command:cargo build --release

contrib/orb@.example.serviceis the example template systemd service to install.The instance configurations locate at/etc/orb/<name>.toml, whose format isdocumented in./contrib/config-onedrive.example.toml.Once configured and logined (see the next section), runsystemctl start orb@<name>.service to start the service.

The service configuration does not contain the login credential. It must beinteractively setup for the first time, and then the service will rotate thecredentials automatically unless the user revokes the permission, or after along offline time (seems to be >1month, but is determined by Microsoft).

1. First, you need to know this project (orb) is an third party program whichaccess your files on Microsoft OneDrive on behalf of you, to provide blockdevice interface as a service. Your files and/or data on your MicrosoftOneDrive may be lost due to program bugs or other reasons. We provide nowarranties. By following the login steps below, you understood and want touse orb at your own risk.

2. We cannot provide an "official App/Client ID" without risking impersonatedbecause this project is open sourced and free to distribute. So you need toregister your own App on MicrosoftAzure.

   In the registration page,

   • In "Supported account types" section, select "Personal Microsoft accountsonly". Other accounts are currently unsupported.
   • In "Redirect URI (optional)" section, select "Public client/native(mobile & desktop)", and enter the following URI:

     http://localhost

     It must be this exactly (it'shttp nothttps), or you may fail thenext step.

   Then click "Register", it will jump to the registered App information pageif success. In "Essential" section, copy the UUID in the "Application(client) ID" field. This is the Client ID to be used in the next step.Note that one App can be used in multiple accounts, for multiple times. Youdo not need to register more than one App in almost any cases.

3. Login with this command with root permission with arguments filled:

   #orb login --systemd<instance> --client-id<the-client-uuid-from-the-last-step>

   <instance> is the instance name of your systemd service (for example, yousetup/etc/orb/foo.toml, thenfoo is the instance name) or in NixOSmodule settingservices.orb.instances.<instance>.

   It will prompt a URL, and you need to open it in your browser and followingthe interactive login steps to login into your Microsoft account withOneDrive.

   The credential will be saved under/var/lib/orb/<instance>, owned byroot, and cannot be accessed by non-root users. It will be rotated by theservice, and please never copy or save it outside the local machine. If youneed to login to the same account on two machines, login twice.

   ⚠️You must not serve the same remote directory simultaneously in multipleinstances (or machines), or it will cause data race and your data will becorrupted. orb will try its best to detect and prevent such racing serving.

4. On success, the web page will redirect to a mostly empty page with only one line:

   Successfully logined. This page can be closed.

   The command should exit normally with credential saved. Now you are readyto start the orb service.

Once your logined and started the service successfully, you are ready to use it.Usually you need to create an filesystem on the emulated block device, and thisis almost the same as the setup for your fresh hard disks, with a fewexceptions:

• The emulated device is under/dev/ublkb<ID> whereID is specified inyour configurationublk.id.

• The device is azoned device(aka. ZBC/ZBD/ZNS, host managed SMR disks) due to API restrictions andperformance reasons. Only a few filesystems and/or device mappers support it,eg. dm-crypt, F2FS and BTRFS.

• It has a high latency and low throughput depend on your network. Doingactive works on it should be avoided. It can be used, for example, forbackup purpose.

• ⚠️ Since the block device is emulated, you must ensure toumount thefilesystem on it before shutting down the backing device service(orb@<instance>.service), or you will lose your last written data. Thiscould be enforced by systemd mounts with aBindsTo= dependency.

Due to the limitation of OneDrive API, permanently deletion cannot be done viaAPI. You may need to regularily "Empty recycle bin" onOneDriveonline to free the capacity occupied.

⚠️ You MUST not "Restore" any files under the directory managed by theorb service (backend.onedrive.remote_dir). Otherwise, it may break filesystemconsistency and your data may be lost.

#./contrib/cryptsetup-format-zoned.sh /dev/ublkb<ID># Use a a password.OR#./contrib/cryptsetup-format-zoned.sh /dev/ublkb<ID> /path/to/key/file# Use a key file.

$nix shell github:oxalica/orb#cryptsetup-format-zoned -c sudo cryptsetup-format-zoned /dev/ublkb<ID>

#cryptsetup luksOpen /dev/ublkb<ID> my-device-unencrypted#cryptsetup close my-device-unencrypted

{ ...}:{environment.etc."crypttab".text=''    mydecrypteddev /dev/ublkb<ID> /path/to/key/file noauto  '';systemd.services."systemd-cryptsetup@mydecrypteddev"={# Inform Nix that this is an overriding units for auto-generated ones.overrideStrategy="asDropin";# Specify dependencies to the orb service.bindsTo=["orb@my-instance.service"];after=["orb@my-instance.service"];};}

#mkfs.btrfs /dev/ublkb<ID> -O block-group-tree

sudo mount -t btrfs -o noatime,compress=zstd:7 /dev/ublkb<ID> /mnt/my-mount-point

The sub-packageorb-ublk andublk-chown-unprivileged (directory/orb-ublk,/ublk-chown-unprivileged and the whole sub-tree of them)are licensed under either ofApache License, Version2.0 orMIT license atyour option.

The main package (all other files in the repository except content of/orb-ublk and/or/ublk-chown-unprivileged directory) is licensed underGNU General Public License v3.0 or (at your option) laterversions.