sio
packagemodule
This package is not in the latest version of its module.
Details
Validgo.mod file
The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
- Redistributable license
Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
- Tagged version
Modules with tagged versions give importers more predictable builds.
Stable versionWhen a project reaches major version v1 it is considered stable.
- Learn more about best practices
Repository
Links
Open Source Insights
README¶
Secure IO
Go implementation of the Data At Rest Encryption (DARE) format.
Introduction
It is a common problem to store data securely - especially on untrusted remote storage.One solution to this problem is cryptography. Before data is stored it is encryptedto ensure that the data is confidential. Unfortunately encrypting data is not enough toprevent more sophisticated attacks. Anyone who has access to the stored data can try tomanipulate the data - even if the data is encrypted.
To prevent these kinds of attacks the data must be encrypted in a tamper-resistant way.This means an attacker should not be able to:
- Read the stored data - this is achieved by modern encryption algorithms.
- Modify the data by changing parts of the encrypted data.
- Rearrange or reorder parts of the encrypted data.
Authenticated encryption schemes (AE) - like AES-GCM or ChaCha20-Poly1305 - encrypt andauthenticate data. Any modification to the encrypted data (ciphertext) is detected whiledecrypting the data. But even an AE scheme alone is not sufficiently enough to prevent allkinds of data manipulation.
All modern AE schemes produce an authentication tag which is verified after the ciphertextis decrypted. If a large amount of data is decrypted it is not always possible to bufferall decrypted data until the authentication tag is verified. Returning unauthenticateddata has the same issues like encrypting data without authentication.
Splitting the data into small chunks fixes the problem of deferred authentication checksbut introduces a new one. The chunks can be reordered - e.g. exchanging chunk 1 and 2 -because every chunk is encrypted separately. Therefore the order of the chunks must beencoded somehow into the chunks itself to be able to detect rearranging any number ofchunks.
This project specifies aformat foren/decrypting an arbitrary data stream and gives somerecommendationsabout how to use and implement data at rest encryption (DARE). Additionally this projectprovides a reference implementation in Go.
Applications
DARE is designed with simplicity and efficiency in mind. It combines modern AE schemeswith a very simple reorder protection mechanism to build a tamper-resistant encryptionscheme. DARE can be used to encrypt files, backups and even large object storage systems.
Its main properties are:
- Security and high performance by relying on modern AEAD ciphers
- Small overhead - encryption increases the amount of data by ~0.05%
- Support for long data streams - up to 256 TB under the same key
- Random access - arbitrary sequences / ranges can be decrypted independently
Install:go get -u github.com/minio/sio
DARE andgithub.com/minio/sio are finalized and can be used in production.
We also provide a CLI tool to en/decrypt arbitrary data streams directly fromyour command line:
Install ncrypt:go get -u github.com/minio/sio/cmd/ncrypt && ncrypt -h
Performance
| Cipher | 8 KB | 64 KB | 512 KB | 1 MB |
|---|---|---|---|---|
| AES_256_GCM | 90 MB/s | 1.96 GB/s | 2.64 GB/s | 2.83 GB/s |
| CHACHA20_POLY1305 | 97 MB/s | 1.23 GB/s | 1.54 GB/s | 1.57 GB/s |
On i7-6500U 2 x 2.5 GHz | Linux 4.10.0-32-generic | Go 1.8.3 | AES-NI & AVX2
Documentation¶
Overview¶
Package sio implements the DARE format. It provides an API for secureen/decrypting IO operations using io.Reader and io.Writer.
Index¶
- Constants
- func Decrypt(dst io.Writer, src io.Reader, config Config) (n int64, err error)
- func DecryptBuffer(dst, src []byte, config Config) (output []byte, err error)
- func DecryptReader(src io.Reader, config Config) (io.Reader, error)
- func DecryptReaderAt(src io.ReaderAt, config Config) (io.ReaderAt, error)
- func DecryptWriter(dst io.Writer, config Config) (io.WriteCloser, error)
- func DecryptedSize(size uint64) (uint64, error)
- func Encrypt(dst io.Writer, src io.Reader, config Config) (n int64, err error)
- func EncryptReader(src io.Reader, config Config) (io.Reader, error)
- func EncryptWriter(dst io.Writer, config Config) (io.WriteCloser, error)
- func EncryptedSize(size uint64) (uint64, error)
- type Config
- type Error
Examples¶
Constants¶
const (// Version20 specifies version 2.0Version20byte = 0x20// Version10 specifies version 1.0Version10byte = 0x10)
const (// AES_256_GCM specifies the cipher suite AES-GCM with 256 bit keys.AES_256_GCMbyte =iota// CHACHA20_POLY1305 specifies the cipher suite ChaCha20Poly1305 with 256 bit keys.CHACHA20_POLY1305)
Variables¶
This section is empty.
Functions¶
funcDecrypt¶
Decrypt reads from src until it encounters an io.EOF and decrypts all receiveddata. The decrypted data is written to dst. It returns the number of bytesdecrypted and the first error encountered while decrypting, if any.
Decrypt returns the number of bytes written to dst. Decrypt only writes data todst if the data was decrypted successfully. It returns an error of type sio.Errorif decryption fails.
Example¶
// the master key used to derive encryption keysmasterkey, err := hex.DecodeString("000102030405060708090A0B0C0D0E0FF0E0D0C0B0A090807060504030201000") // use your own key hereif err != nil {fmt.Printf("Cannot decode hex key: %v", err) // add error handlingreturn}// the nonce used to derive the encryption keynonce, err := hex.DecodeString("0000000000000000000000000000000000000000000000000000000000000001") // use your generated nonce hereif err != nil {fmt.Printf("Cannot decode hex key: %v", err) // add error handlingreturn}// derive the encryption key from the master key and the noncevar key [32]bytekdf := hkdf.New(sha256.New, masterkey, nonce, nil)if _, err = io.ReadFull(kdf, key[:]); err != nil {fmt.Printf("Failed to derive encryption key: %v", err) // add error handlingreturn}input := os.Stdin // customize for your needs - the encrypted dataoutput := os.Stdout // customize from your needs - the decrypted outputif _, err = Decrypt(output, input, Config{Key: key[:]}); err != nil {if _, ok := err.(Error); ok {fmt.Printf("Malformed encrypted data: %v", err) // add error handling - here we know that the data is malformed/not authentic.return}fmt.Printf("Failed to decrypt data: %v", err) // add error handlingreturn}Output:funcDecryptBuffer¶added inv0.2.1
DecryptBuffer decrypts all received data in src.The decrypted data is appended to dst.If the number of output bytes is unknown,making a dst with capacity of len(src) is reasonable.
DecryptBuffer only returns data to if the data was decrypted successfully.It returns an error of type sio.Error if decryption fails.
funcDecryptReader¶
DecryptReader wraps the given src and returns an io.Reader which decryptsall received data. DecryptReader returns an error if the provided decryptionconfiguration is invalid. The returned io.Reader returns an error oftype sio.Error if the decryption fails.
funcDecryptReaderAt¶added inv0.2.1
DecryptReaderAt wraps the given src and returns an io.ReaderAt which decryptsall received data. DecryptReaderAt returns an error if the provided decryptionconfiguration is invalid. The returned io.ReaderAt returns an error oftype sio.Error if the decryption fails.
funcDecryptWriter¶
DecryptWriter wraps the given dst and returns an io.WriteCloser whichdecrypts all data written to it. DecryptWriter returns an error if theprovided decryption configuration is invalid.
The returned io.WriteCloser must be closed successfully to finalize thedecryption process. The returned io.WriteCloser returns an error oftype sio.Error if the decryption fails.
funcDecryptedSize¶
DecryptedSize computes the size of a decrypted data streamfrom the encrypted stream size. It is the inverse of EncryptedSize().
DecryptedSize returns an error if the provided size is to largeor if the provided size is an invalid encrypted stream size.
funcEncrypt¶
Encrypt reads from src until it encounters an io.EOF and encrypts all receiveddata. The encrypted data is written to dst. It returns the number of bytesencrypted and the first error encountered while encrypting, if any.
Encrypt returns the number of bytes written to dst.
Example¶
// the master key used to derive encryption keys// this key must be keep secretmasterkey, err := hex.DecodeString("000102030405060708090A0B0C0D0E0FF0E0D0C0B0A090807060504030201000") // use your own key hereif err != nil {fmt.Printf("Cannot decode hex key: %v", err) // add error handlingreturn}// generate a random nonce to derive an encryption key from the master key// this nonce must be saved to be able to decrypt the data again - it is not// required to keep it secretvar nonce [32]byteif _, err = io.ReadFull(rand.Reader, nonce[:]); err != nil {fmt.Printf("Failed to read random data: %v", err) // add error handlingreturn}// derive an encryption key from the master key and the noncevar key [32]bytekdf := hkdf.New(sha256.New, masterkey, nonce[:], nil)if _, err = io.ReadFull(kdf, key[:]); err != nil {fmt.Printf("Failed to derive encryption key: %v", err) // add error handlingreturn}input := os.Stdin // customize for your needs - the plaintextoutput := os.Stdout // customize from your needs - the decrypted outputif _, err = Encrypt(output, input, Config{Key: key[:]}); err != nil {fmt.Printf("Failed to encrypt data: %v", err) // add error handlingreturn}Output:funcEncryptReader¶
EncryptReader wraps the given src and returns an io.Reader which encryptsall received data. EncryptReader returns an error if the provided encryptionconfiguration is invalid.
Example¶
// the master key used to derive encryption keys// this key must be keep secretmasterkey, err := hex.DecodeString("000102030405060708090A0B0C0D0E0FF0E0D0C0B0A090807060504030201000") // use your own key hereif err != nil {fmt.Printf("Cannot decode hex key: %v", err) // add error handlingreturn}// generate a random nonce to derive an encryption key from the master key// this nonce must be saved to be able to decrypt the data again - it is not// required to keep it secretvar nonce [32]byteif _, err = io.ReadFull(rand.Reader, nonce[:]); err != nil {fmt.Printf("Failed to read random data: %v", err) // add error handlingreturn}// derive an encryption key from the master key and the noncevar key [32]bytekdf := hkdf.New(sha256.New, masterkey, nonce[:], nil)if _, err = io.ReadFull(kdf, key[:]); err != nil {fmt.Printf("Failed to derive encryption key: %v", err) // add error handlingreturn}input := os.Stdin // customize for your needs - the plaintext inputencrypted, err := EncryptReader(input, Config{Key: key[:]})if err != nil {fmt.Printf("Failed to encrypted reader: %v", err) // add error handlingreturn}// the encrypted io.Reader can be used like every other reader - e.g. for copyingif _, err := io.Copy(os.Stdout, encrypted); err != nil {fmt.Printf("Failed to copy data: %v", err) // add error handlingreturn}Output:funcEncryptWriter¶
EncryptWriter wraps the given dst and returns an io.WriteCloser whichencrypts all data written to it. EncryptWriter returns an error if theprovided decryption configuration is invalid.
The returned io.WriteCloser must be closed successfully to finalize theencryption process.
Example¶
// the master key used to derive encryption keys// this key must be keep secretmasterkey, err := hex.DecodeString("000102030405060708090A0B0C0D0E0FF0E0D0C0B0A090807060504030201000") // use your own key hereif err != nil {fmt.Printf("Cannot decode hex key: %v", err) // add error handlingreturn}// generate a random nonce to derive an encryption key from the master key// this nonce must be saved to be able to decrypt the data again - it is not// required to keep it secretvar nonce [32]byteif _, err = io.ReadFull(rand.Reader, nonce[:]); err != nil {fmt.Printf("Failed to read random data: %v", err) // add error handlingreturn}// derive an encryption key from the master key and the noncevar key [32]bytekdf := hkdf.New(sha256.New, masterkey, nonce[:], nil)if _, err = io.ReadFull(kdf, key[:]); err != nil {fmt.Printf("Failed to derive encryption key: %v", err) // add error handlingreturn}output := os.Stdout // customize for your needs - the encrypted outputencrypted, err := EncryptWriter(output, Config{Key: key[:]})if err != nil {fmt.Printf("Failed to encrypted writer: %v", err) // add error handlingreturn}// the encrypted io.Writer can be used now but it MUST be closed at the end to// finalize the encryption.if _, err = io.Copy(encrypted, os.Stdin); err != nil {fmt.Printf("Failed to copy data: %v", err) // add error handlingreturn}if err = encrypted.Close(); err != nil {fmt.Printf("Failed to finalize encryption: %v", err) // add error handlingreturn}Output:funcEncryptedSize¶
EncryptedSize computes the size of an encrypted data streamfrom the plaintext size. It is the inverse of DecryptedSize().
EncryptedSize returns an error if the provided size is to large.
Types¶
typeConfig¶
type Config struct {// The minimal supported version of the format. If// not set the default value - Version10 - is used.MinVersionbyte// The highest supported version of the format. If// not set the default value - Version20 - is used.MaxVersionbyte// A list of supported cipher suites. If not set the// default value is used.CipherSuites []byte// The secret encryption key. It must be 32 bytes long.Key []byte// The first expected sequence number. It should only// be set manually when decrypting a range within a// stream.SequenceNumberuint32// The RNG used to generate random values. If not set// the default value (crypto/rand.Reader) is used.Randio.Reader// Nonce will override the nonce if set non-nil.// V2 will use all 12 bytes, V1 first 8 bytes.Nonce *[12]byte// The size of the encrypted payload in bytes. The// default value is 64KB. It should be used to restrict// the size of encrypted packages. The payload size// must be between 1 and 64 KB.//// This field is specific for version 1.0 and is// deprecated.PayloadSizeint}Config contains the format configuration. The only fieldwhich must always be set manually is the secret key.
Source Files
Directories