Repository files navigation

SOPS is an editor of encrypted files that supports YAML, JSON, ENV, INI and BINARYformats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, age, and PGP.(demo)

Binaries and packages of the latest stable release are available athttps://github.com/getsops/sops/releases.

For the adventurous, unstable features are available in the main branch, which you can install from source:

$ mkdir -p$GOPATH/src/github.com/getsops/sops/$ git clone https://github.com/getsops/sops.git$GOPATH/src/github.com/getsops/sops/$cd$GOPATH/src/github.com/getsops/sops/$ make install

(requires Go >= 1.19)

If you don't have Go installed, set it up with:

$ {apt,yum,brew} install golang$echo'export GOPATH=~/go'>>~/.bashrc$source~/.bashrc$ mkdir$GOPATH

Or whatever variation of the above fits your system and shell.

To useSOPS as a library, take a look at thedecrypt package.

For a quick presentation of SOPS, check out this Youtube tutorial:

If you're using AWS KMS, create one or multiple master keys in the IAM consoleand export them, comma separated, in theSOPS_KMS_ARN env variable. It isrecommended to use at least two master keys in different regions.

export SOPS_KMS_ARN="arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e,arn:aws:kms:ap-southeast-1:656532927350:key/9006a8aa-0fa6-4c14-930e-a2dfb916de1d"

SOPS usesaws-sdk-go-v2 to communicate with AWS KMS. It will automaticallyread the credentials from the~/.aws/credentials file which can be created with theaws configure command.

An example of the~/.aws/credentials file is shown below:

$ cat~/.aws/credentials[default]aws_access_key_id = AKI.....aws_secret_access_key = mw......

In addition to the~/.aws/credentials file, you can also use theAWS_ACCESS_KEY_ID andAWS_SECRET_ACCESS_KEYenvironment variables to specify your credentials:

export AWS_ACCESS_KEY_ID="AKI......"export AWS_SECRET_ACCESS_KEY="mw......"

For more information and additional environment variables, seespecifying credentials.

If you want to use PGP, export the fingerprints of the public keys, commaseparated, in theSOPS_PGP_FP env variable.

export SOPS_PGP_FP="85D77543B3D624B63CEA9E6DBC17301B491B3F21,E60892BB9BD89A69F759A1A0A3D652173B763E8F"

Note: you can use both PGP and KMS simultaneously.

Then simply callsops edit with a file path as argument. It will handle theencryption/decryption transparently and open the cleartext file in an editor

$ sops edit mynewtestfile.yamlmynewtestfile.yaml doesn't exist, creating it.please wait while an encryption key is being generated and stored in a secure fashionfile written to mynewtestfile.yaml

Editing will happen in whatever$SOPS_EDITOR or$EDITOR is set to, or, if it'snot set, in vim, nano, or vi.Keep in mind that SOPS will wait for the editor to exit, and then try to reencryptthe file. Some GUI editors (atom, sublime) spawn a child process and then exitimmediately. They usually have an option to wait for the main editor window to beclosed before exiting. See#127 formore information.

The resulting encrypted file looks like this:

myapp1:ENC[AES256_GCM,data:Tr7o=,iv:1=,aad:No=,tag:k=]app2:db:user:ENC[AES256_GCM,data:CwE4O1s=,iv:2k=,aad:o=,tag:w==]password:ENC[AES256_GCM,data:p673w==,iv:YY=,aad:UQ=,tag:A=]# private key for secret operations in app2key:|-        ENC[AES256_GCM,data:Ea3kL5O5U8=,iv:DM=,aad:FKA=,tag:EA==]an_array:    -ENC[AES256_GCM,data:v8jQ=,iv:HBE=,aad:21c=,tag:gA==]    -ENC[AES256_GCM,data:X10=,iv:o8=,aad:CQ=,tag:Hw==]    -ENC[AES256_GCM,data:KN=,iv:160=,aad:fI4=,tag:tNw==]sops:kms:        -created_at:1441570389.775376enc:CiC....Pm1Hmarn:arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e        -created_at:1441570391.925734enc:Ci...awNxarn:arn:aws:kms:ap-southeast-1:656532927350:key/9006a8aa-0fa6-4c14-930e-a2dfb916de1dpgp:        -fp:85D77543B3D624B63CEA9E6DBC17301B491B3F21created_at:1441570391.930042enc:|              -----BEGIN PGP MESSAGE-----              hQIMA0t4uZHfl9qgAQ//UvGAwGePyHuf2/zayWcloGaDs0MzI+zw6CmXvMRNPUsA              ...=oJgS              -----END PGP MESSAGE-----

A copy of the encryption/decryption key is stored securely in each KMS and PGPblock. As long as one of the KMS or PGP method is still usable, you will be ableto access your data.

To decrypt a file in acat fashion, use the-d flag:

$ sops decrypt mynewtestfile.yaml

SOPS encrypted files contain the necessary information to decrypt their content.All a user of SOPS needs is valid AWS credentials and the necessarypermissions on KMS keys.

Given that, the only command a SOPS user needs is:

$ sops edit<file>

<file> will be opened, decrypted, passed to a text editor (vim by default),encrypted if modified, and saved back to its original location. All of thesesteps, apart from the actual editing, are transparent to the user.

The order in which available decryption methods are tried can be specified with--decryption-order option orSOPS_DECRYPTION_ORDER environment variableas a comma separated list. The default order isage,pgp. Offline methods aretried first and then the remaining ones.

If you want to testSOPS without having to do a bunch of setup, you can usethe example files and pgp key provided with the repository:

$ git clone https://github.com/getsops/sops.git$ cd sops$ gpg --import pgp/sops_functional_tests_key.asc$ sops edit example.yaml

This last step will decryptexample.yaml using the test private key.

If you want to encrypt with specific GnuPG subkeys, it does not suffice to provide theexact key ID of the subkey to SOPS, since GnuPG might useanother subkey insteadto encrypt the file key with. To force GnuPG to use a specific subkey, you need toappend! to the key's fingerprint.

creation_rules:    -pgp:>-        85D77543B3D624B63CEA9E6DBC17301B491B3F21!,        E60892BB9BD89A69F759A1A0A3D652173B763E8F!

Please note that this is only passed on correctly to GnuPG since SOPS 3.9.3.

age is a simple, modern, and secure tool forencrypting files. It's recommended to use age over PGP, if possible.

You can encrypt a file for one or more age recipients (comma separated) usingthe--age option or theSOPS_AGE_RECIPIENTS environment variable:

$ sops encrypt --age age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw test.yaml> test.enc.yaml

When decrypting a file with the corresponding identity, SOPS will look for atext file namekeys.txt located in asops subdirectory of your userconfiguration directory. On Linux, this would be$XDG_CONFIG_HOME/sops/age/keys.txt.If$XDG_CONFIG_HOME is not set$HOME/.config/sops/age/keys.txt is used instead.On macOS, this would be$HOME/Library/Application Support/sops/age/keys.txt. OnWindows, this would be%AppData%\sops\age\keys.txt. You can specify the locationof this file manually by setting the environment variableSOPS_AGE_KEY_FILE.Alternatively, you can provide the key(s) directly by setting theSOPS_AGE_KEYenvironment variable.

The contents of this key file should be a list of age X25519 identities, oneper line. Lines beginning with# are considered comments and ignored. Eachidentity will be tried in sequence until one is able to decrypt the data.

Encrypting with SSH keys via age is also supported by SOPS. You can use SSH public keys("ssh-ed25519 AAAA...", "ssh-rsa AAAA...") as age recipients when encrypting a file.When decrypting a file, SOPS will look for~/.ssh/id_ed25519 and falls back to~/.ssh/id_rsa. You can specify the location of the private key manually by settingthe environment variableuseSOPS_AGE_SSH_PRIVATE_KEY_FILE.

Note that onlyssh-rsa andssh-ed25519 are supported.

A list of age recipients can be added to the.sops.yaml:

creation_rules:    -age:>-        age1s3cqcks5genc6ru8chl0hkkd04zmxvczsvdxq99ekffe4gmvjpzsedk23c,        age1qe5lxzzeppw5k79vxn3872272sgy224g2nzqlzy3uljs84say3yqgvd0sw

It is also possible to useupdatekeys, when adding or removing age recipients. For example:

$ sops updatekeys secret.enc.yaml2022/02/09 16:32:02 Syncing keysfor file /iac/solution1/secret.enc.yamlThe following changes will be made to the file's groups:Group 1    age1s3cqcks5genc6ru8chl0hkkd04zmxvczsvdxq99ekffe4gmvjpzsedk23c+++ age1qe5lxzzeppw5k79vxn3872272sgy224g2nzqlzy3uljs84say3yqgvd0swIs this okay? (y/n):y2022/02/09 16:32:04 File /iac/solution1/secret.enc.yaml synced with new keys

GCP KMS usesApplication Default Credentials.If you already logged in using

$ gcloud auth login

you can enable application default credentials using the sdk:

$ gcloud auth application-default login

Encrypting/decrypting with GCP KMS requires a KMS ResourceID. You can use thecloud console the get the ResourceID or you can create one using the gcloudsdk:

$ gcloud kms keyrings create sops --location global$ gcloud kms keys create sops-key --location global --keyring sops --purpose encryption$ gcloud kms keys list --location global --keyring sops# you should seeNAME                                                                   PURPOSE          PRIMARY_STATEprojects/my-project/locations/global/keyRings/sops/cryptoKeys/sops-key ENCRYPT_DECRYPT  ENABLED

Now you can encrypt a file using:

$ sops encrypt --gcp-kms projects/my-project/locations/global/keyRings/sops/cryptoKeys/sops-key test.yaml > test.enc.yaml

And decrypt it using:

$ sops decrypt test.enc.yaml

The Azure Key Vault integration uses thedefault credential chainwhich tries several authentication methods, in this order:

1. Environment credentials
   1. Service Principal with Client Secret
   2. Service Principal with Certificate
   3. User with username and password
   4. Configuration for multi-tenant applications
2. Workload Identity credentials
3. Managed Identity credentials
4. Azure CLI credentials

For example, you can use a Service Principal with the following environment variables:

AZURE_TENANT_IDAZURE_CLIENT_IDAZURE_CLIENT_SECRET

You can create a Service Principal using the CLI like this:

$ az ad sp create-for-rbac -n my-keyvault-sp{"appId":"<some-uuid>","displayName":"my-keyvault-sp","name":"http://my-keyvault-sp","password":"<random-string>","tenant":"<tenant-uuid>"}

The appId is the client ID, and the password is the client secret.

Encrypting/decrypting with Azure Key Vault requires the resource identifier fora key. This has the following form:

https://${VAULT_URL}/keys/${KEY_NAME}/${KEY_VERSION}

To create a Key Vault and assign your service principal permissions on itfrom the commandline:

# Create a resource group if you do not have one:$ az group create --name sops-rg --location westeurope# Key Vault names are globally unique, so generate one:$ keyvault_name=sops-$(uuidgen| tr -d -| head -c 16)# Create a Vault, a key, and give the service principal access:$ az keyvault create --name$keyvault_name --resource-group sops-rg --location westeurope$ az keyvault key create --name sops-key --vault-name$keyvault_name --protection software --ops encrypt decrypt$ az keyvault set-policy --name$keyvault_name --resource-group sops-rg --spn$AZURE_CLIENT_ID \    --key-permissions encrypt decrypt# Read the key id:$ az keyvault key show --name sops-key --vault-name$keyvault_name --query key.kidhttps://sops.vault.azure.net/keys/sops-key/some-string

Now you can encrypt a file using:

$ sops encrypt --azure-kv https://sops.vault.azure.net/keys/sops-key/some-string test.yaml > test.enc.yaml

And decrypt it using:

$ sops decrypt test.enc.yaml

When usingsops in scripts or from other programs, there are often situations where you do not want to writeencrypted or decrypted data to disk. The best way to avoid this is to pass data to SOPS via stdin, and to letSOPS write data to stdout. By default, the encrypt and decrypt operations write data to stdout already. To passdata via stdin, you need to not provide an input filename. For encryption, you also must provide the--filename-override option with the file's filename. The filename will be used to determine the input and outputtypes, and to select the correct creation rule.

The simplest way to decrypt data from stdin is as follows:

$ cat encrypted-data| sops decrypt> decrypted-data

By default,sops determines the input and output format from the provided filename. Since in this case,no filename is provided,sops will use the binary store which expects JSON input and outputs binary dataon decryption. This is often not what you want.

To avoid this, you can either provide a filename with--filename-override, or explicitly controlthe input and output formats by passing--input-type and--output-type as appropriate:

$ cat encrypted-data| sops decrypt --filename-override filename.yaml> decrypted-data$ cat encrypted-data| sops decrypt --input-type yaml --output-type yaml> decrypted-data

In both cases,sops will assume that the data you provide is in YAML format, and will encode the decrypteddata in YAML as well. The second form allows to use different formats for input and output.

To encrypt, it is important to note that SOPS also uses the filename to look up the correct creation rule from.sops.yaml. Therefore, you must provide the--filename-override parameter which allows you to tellSOPS which filename to use to match creation rules:

$echo'foo: bar'| sops encrypt --filename-override path/filename.sops.yaml> encrypted-data

SOPS will find a matching creation rule forpath/filename.sops.yaml in.sops.yaml and use that one toencrypt the data from stdin. This filename will also be used to determine the input and output store. As always,the input store type can be adjusted by passing--input-type, and the output store type by passing--output-type:

$echo foo=bar| sops encrypt --filename-override path/filename.sops.yaml --input-type dotenv> encrypted-data

We assume you have an instance (or more) of Vault running and you have privileged access to it. For instructions on how to deploy a secure instance of Vault, refer to Hashicorp's official documentation.

To easily deploy Vault locally: (DO NOT DO THIS FOR PRODUCTION!!!)

$ docker run -d -p8200:8200 vault:1.2.0 server -dev -dev-root-token-id=toor

$# Substitute this with the address Vault is running on$export VAULT_ADDR=http://127.0.0.1:8200$# this may not be necessary in case you previously used `vault login` for production use$export VAULT_TOKEN=toor$# to check if Vault started and is configured correctly$ vault statusKey             Value---             -----Seal Type       shamirInitializedtrueSealedfalseTotal Shares    1Threshold       1Version         1.2.0Cluster Name    vault-cluster-618cc902Cluster ID      e532e461-e8f0-1352-8a41-fc7c11096908HA Enabledfalse$# It is required to enable a transit engine if not already done (It is suggested to create a transit engine specifically for SOPS, in which it is possible to have multiple keys with various permission levels)$ vault secretsenable -path=sops transitSuccess! Enabled the transit secrets engine at: sops/$# Then create one or more keys$ vault write sops/keys/firstkey type=rsa-4096Success! Data written to: sops/keys/firstkey$ vault write sops/keys/secondkey type=rsa-2048Success! Data written to: sops/keys/secondkey$ vault write sops/keys/thirdkey type=chacha20-poly1305Success! Data written to: sops/keys/thirdkey$ sops encrypt --hc-vault-transit$VAULT_ADDR/v1/sops/keys/firstkey vault_example.yml$ cat<<EOF > .sops.yamlcreation_rules:    - path_regex: \.dev\.yaml$      hc_vault_transit_uri: "$VAULT_ADDR/v1/sops/keys/secondkey"    - path_regex: \.prod\.yaml$      hc_vault_transit_uri: "$VAULT_ADDR/v1/sops/keys/thirdkey"EOF$ sops encrypt --verbose prod/raw.yaml> prod/encrypted.yaml

When creating new files,sops uses the PGP, KMS and GCP KMS defined in thecommand line arguments--kms,--pgp,--gcp-kms or--azure-kv, or fromthe environment variablesSOPS_KMS_ARN,SOPS_PGP_FP,SOPS_GCP_KMS_IDS,SOPS_AZURE_KEYVAULT_URLS. That information is stored in the file under thesops section, such that decrypting files does not require providing thoseparameters again.

Master PGP and KMS keys can be added and removed from asops file in one ofthree ways:

1. By using a.sops.yaml file and theupdatekeys command.
2. By using command line flags.
3. By editing the file directly.

The SOPS team recommends theupdatekeys approach.

Theupdatekeys command uses the.sops.yamlconfiguration file to update (add or remove) the corresponding secrets in theencrypted file. Note that the example below uses theBlock Scalar yaml construct to build a spaceseparated list.

creation_rules:    -pgp:>-        85D77543B3D624B63CEA9E6DBC17301B491B3F21,        FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4

$ sops updatekeys test.enc.yaml

SOPS will prompt you with the changes to be made. This interactivity can bedisabled by supplying the-y flag.

Therotate command generates a new data encryption key and reencrypt all valueswith the new key. At the same time, the command line flag--add-kms,--add-pgp,--add-gcp-kms,--add-azure-kv,--rm-kms,--rm-pgp,--rm-gcp-kmsand--rm-azure-kv can be used to add and remove keys from a file. These flags usethe comma separated syntax as the--kms,--pgp,--gcp-kms and--azure-kvarguments when creating new files.

Useupdatekeys if you want to add a key without rotating the data key.

# add a new pgp key to the file and rotate the data key$ sops rotate -i --add-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 example.yaml# remove a pgp key from the file and rotate the data key$ sops rotate -i --rm-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 example.yaml

Alternatively, invokingsops edit with the flag-s will display the master keyswhile editing. This method can be used to add or removekms orpgp keys under thesops section.

For example, to add a KMS master key to a file, add the following entry whileediting:

sops:kms:        -arn:arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e

And, similarly, to add a PGP master key, we add its fingerprint:

sops:pgp:        -fp:85D77543B3D624B63CEA9E6DBC17301B491B3F21

When the file is saved, SOPS will update its metadata and encrypt the data keywith the freshly added master keys. The removed entries are simply deleted fromthe file.

When removing keys, it is recommended to rotate the data key using-r,otherwise, owners of the removed key may have add access to the data key in thepast.

If you want to use a specific profile, you can do so with aws_profile:

sops:kms:        -arn:arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27eaws_profile:foo

If no AWS profile is set, default credentials will be used.

Similarly the --aws-profile flag can be set with the command line with any of the KMS commands.

SOPS has the ability to use KMS in multiple AWS accounts by assuming roles ineach account. Being able to assume roles is a nice feature of AWS that allowsadministrators to establish trust relationships between accounts, typically fromthe most secure account to the least secure one. In our use-case, we use rolesto indicate that a user of the Master AWS account is allowed to make use of KMSmaster keys in development and staging AWS accounts. Using roles, a single filecan be encrypted with KMS keys in multiple accounts, thus increasing reliabilityand ease of use.

You can use keys in various accounts by tying each KMS master key to a role thatthe user is allowed to assume in each account. TheIAM rolesdocumentation has full details on how this needs to be configured on AWS's side.

From the point of view of SOPS, you only need to specify the role a KMS keymust assume alongside its ARN, as follows:

sops:kms:        -arn:arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27erole:arn:aws:iam::927034868273:role/sops-dev-xyz

The role must have permission to call Encrypt and Decrypt using KMS. An examplepolicy is shown below.

{"Sid":"Allow use of the key","Effect":"Allow","Action": ["kms:Encrypt","kms:Decrypt","kms:ReEncrypt*","kms:GenerateDataKey*","kms:DescribeKey"  ],"Resource":"*","Principal": {"AWS": ["arn:aws:iam::927034868273:role/sops-dev-xyz"    ]  }}

You can specify a role in the--kms flag andSOPS_KMS_ARN variable byappending it to the ARN of the master key, separated by a+ sign:

<KMS ARN>+<ROLE ARN>arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500+arn:aws:iam::927034868273:role/sops-dev-xyz

SOPS has the ability to useAWS KMS key policy and encryption contextto refine the access control of a given KMS master key.

When creating a new file, you can specify the encryption context in the--encryption-context flag by comma separated list of key-value pairs:

$ sops edit --encryption-context Environment:production,Role:web-server test.dev.yaml

The format of the Encrypt Context string is<EncryptionContext Key>:<EncryptionContext Value>,<EncryptionContext Key>:<EncryptionContext Value>,...

The encryption context will be stored in the file metadata and doesnot need to be provided at decryption.

Encryption contexts can be used in conjunction with KMS Key Policies to defineroles that can only access a given context. An example policy is shown below:

{"Effect":"Allow","Principal": {"AWS":"arn:aws:iam::111122223333:role/RoleForExampleApp"  },"Action":"kms:Decrypt","Resource":"*","Condition": {"StringEquals": {"kms:EncryptionContext:AppName":"ExampleApp","kms:EncryptionContext:FilePath":"/var/opt/secrets/"    }  }}

It is recommended to renew the data key on a regular basis.sops supports keyrotation via therotate command. Invoking it on an existing file causessopsto reencrypt the file with a new data key, which is then encrypted with the variousKMS and PGP master keys defined in the file.

Add the-i option to write the rotated file back, instead of printing it tostdout.

$ sops rotate example.yaml

It is often tedious to specify the--kms--gcp-kms--pgp and--age parameters for creationof all new files. If your secrets are stored under a specific directory, like agit repository, you can create a.sops.yaml configuration file at the rootdirectory to define which keys are used for which filename.

Let's take an example:

• file namedsomething.dev.yaml should use one set of KMS A, PGP and age
• file namedsomething.prod.yaml should use another set of KMS B, PGP and age
• other files use a third set of KMS C and PGP
• all live undermysecretrepo/something.{dev,prod,gcp}.yaml

Under those circumstances, a file placed atmysecretrepo/.sops.yamlcan manage the three sets of configurations for the three types of files:

# creation rules are evaluated sequentially, the first match winscreation_rules:# upon creation of a file that matches the pattern *.dev.yaml,# KMS set A as well as PGP and age is used    -path_regex:\.dev\.yaml$kms:'arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500,arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e+arn:aws:iam::361527076523:role/hiera-sops-prod'pgp:'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4'age:'age129h70qwx39k7h5x6l9hg566nwm53527zvamre8vep9e3plsm44uqgy8gla'# prod files use KMS set B in the PROD IAM, PGP and age    -path_regex:\.prod\.yaml$kms:'arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e+arn:aws:iam::361527076523:role/hiera-sops-prod,arn:aws:kms:eu-central-1:361527076523:key/cb1fab90-8d17-42a1-a9d8-334968904f94+arn:aws:iam::361527076523:role/hiera-sops-prod'pgp:'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4'age:'age129h70qwx39k7h5x6l9hg566nwm53527zvamre8vep9e3plsm44uqgy8gla'hc_vault_uris:"http://localhost:8200/v1/sops/keys/thirdkey"# gcp files using GCP KMS    -path_regex:\.gcp\.yaml$gcp_kms:projects/mygcproject/locations/global/keyRings/mykeyring/cryptoKeys/thekey# Finally, if the rules above have not matched, this one is a# catchall that will encrypt the file using KMS set C as well as PGP# The absence of a path_regex means it will match everything    -kms:'arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500,arn:aws:kms:us-west-2:142069644989:key/846cfb17-373d-49b9-8baf-f36b04512e47,arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e'pgp:'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4'

When creating any file undermysecretrepo, whether at the root or undera subdirectory, SOPS will recursively look for a.sops.yaml file. If one isfound, the filename of the file being created is compared with the filenameregexes of the configuration file. The first regex that matches is selected,and its KMS and PGP keys are used to encrypt the file. It should be noted thatthe looking up of.sops.yaml is from the working directory (CWD) instead ofthe directory of the encrypting file (seeIssue 242).

Thepath_regex checks the path of the encrypting file relative to the.sops.yaml config file. Here is another example:

• files located under directorydevelopment should use one set of KMS A
• files located under directoryproduction should use another set of KMS B
• other files use a third set of KMS C

creation_rules:# upon creation of a file under development,# KMS set A is used    -path_regex:.*/development/.*kms:'arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500,arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e+arn:aws:iam::361527076523:role/hiera-sops-prod'pgp:'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4'# prod files use KMS set B in the PROD IAM    -path_regex:.*/production/.*kms:'arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e+arn:aws:iam::361527076523:role/hiera-sops-prod,arn:aws:kms:eu-central-1:361527076523:key/cb1fab90-8d17-42a1-a9d8-334968904f94+arn:aws:iam::361527076523:role/hiera-sops-prod'pgp:'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4'# other files use KMS set C    -kms:'arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500,arn:aws:kms:us-west-2:142069644989:key/846cfb17-373d-49b9-8baf-f36b04512e47,arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e'pgp:'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4'

Creating a new file with the right keys is now as simple as

$ sops edit<newfile>.prod.yaml

Note that the configuration file is ignored when KMS or PGP parameters arepassed on the SOPS command line or in environment variables.

SOPS checks for theSOPS_GPG_EXEC environment variable. If specified,it will attempt to use the executable set there instead of the defaultofgpg.

Example: place the following in your~/.bashrc

SOPS_GPG_EXEC ='your_gpg_client_wrapper'

By default, SOPS uses the key serverkeys.openpgp.org to retrieve the GPGkeys that are not present in the local keyring.This is no longer configurable. You can learn more about why from this write-up:SKS Keyserver Network Under Attack.

By default, SOPS encrypts the data key for a file with each of the master keys,such that if any of the master keys is available, the file can be decrypted.However, it is sometimes desirable to require access to multiple master keysin order to decrypt files. This can be achieved with key groups.

When using key groups in SOPS, data keys are split into parts such that keys frommultiple groups are required to decrypt a file. SOPS uses Shamir's Secret Sharingto split the data key such that each key group has a fragment, each key in thekey group can decrypt that fragment, and a configurable number of fragments (threshold)are needed to decrypt and piece together the complete data key. When decrypting afile using multiple key groups, SOPS goes through key groups in order, and ineach group, tries to recover the fragment of the data key using a master key fromthat group. Once the fragment is recovered, SOPS moves on to the next group,until enough fragments have been recovered to obtain the complete data key.

By default, the threshold is set to the number of key groups. For example, ifyou have three key groups configured in your SOPS file and you don't overridethe default threshold, then one master key from each of the three groups willbe required to decrypt the file.

Management of key groups is done with thesops groups command.

For example, you can add a new key group with 3 PGP keys and 3 KMS keys to thefilemy_file.yaml:

$ sops groups add --file my_file.yaml --pgp fingerprint1 --pgp fingerprint2 --pgp fingerprint3 --kms arn1 --kms arn2 --kms arn3

Or you can delete the 1st group (group number 0, as groups are zero-indexed)frommy_file.yaml:

$ sops groups delete --file my_file.yaml 0

Key groups can also be specified in the.sops.yaml config file,like so:

creation_rules:    -path_regex:.*keygroups.*key_groups:# First key group          -pgp:                -fingerprint1                -fingerprint2kms:                -arn:arn1role:role1context:foo:bar                -arn:arn2aws_profile:myprofile# Second key group          -pgp:                -fingerprint3                -fingerprint4kms:                -arn:arn3                -arn:arn4# Third key group          -pgp:                -fingerprint5

Given this configuration, we can create a new encrypted file like we normallywould, and optionally provide the--shamir-secret-sharing-threshold command lineflag if we want to override the default threshold. SOPS will then split the datakey into three parts (from the number of key groups) and encrypt each fragment withthe master keys found in each group.

For example:

$ sops edit --shamir-secret-sharing-threshold 2 example.json

Alternatively, you can configure the Shamir threshold for each creation rule in the.sops.yaml configwithshamir_threshold:

creation_rules:    -path_regex:.*keygroups.*shamir_threshold:2key_groups:# First key group          -pgp:                -fingerprint1                -fingerprint2kms:                -arn:arn1role:role1context:foo:bar                -arn:arn2aws_profile:myprofile# Second key group          -pgp:                -fingerprint3                -fingerprint4kms:                -arn:arn3                -arn:arn4# Third key group          -pgp:                -fingerprint5

And then runsops edit example.json.

The threshold (shamir_threshold) is set to 2, so this configuration will requiremaster keys from two of the three different key groups in order to decrypt the file.You can then decrypt the file the same way as with any other SOPS file:

$ sops decrypt example.json

There are situations where you might want to run SOPS on a machine thatdoesn't have direct access to encryption keys such as PGP keys. Thesops keyservice allows you to forward a socket so that SOPS can access encryptionkeys stored on a remote machine. This is similar to GPG Agent, but moreportable.

SOPS uses a client-server approach to encrypting and decrypting the datakey. By default, SOPS runs a local key service in-process. SOPS uses a keyservice client to send an encrypt or decrypt request to a key service, whichthen performs the operation. The requests are sent using gRPC and ProtocolBuffers. The requests contain an identifier for the key they should performthe operation with, and the plaintext or encrypted data key. The requests donot contain any cryptographic keys, public or private.

WARNING: the key service connection currently does not use any sort ofauthentication or encryption. Therefore, it is recommended that you make surethe connection is authenticated and encrypted in some other way, for examplethrough an SSH tunnel.

Whenever we try to encrypt or decrypt a data key, SOPS will try to do so firstwith the local key service (unless it's disabled), and if that fails, it willtry all other remote key services until one succeeds.

You can start a key service server by runningsops keyservice.

You can specify the key services thesops binary uses with--keyservice.This flag can be specified more than once, so you can use multiple keyservices. The local key service can be disabled withenable-local-keyservice=false.

For example, to decrypt a file using both the local key service and the keyservice exposed on the unix socket located in/tmp/sops.sock, you can run:

$ sops decrypt --keyservice unix:///tmp/sops.sock file.yaml`

And if you only want to use the key service exposed on the unix socket locatedin/tmp/sops.sock and not the local key service, you can run:

$ sops decrypt --enable-local-keyservice=false --keyservice unix:///tmp/sops.sock file.yaml

Sometimes, users want to be able to tell what files were accessed by whom in anenvironment they control. For this reason, SOPS can generate audit logs torecord activity on encrypted files. When enabled, SOPS will write a log entryinto a pre-configured PostgreSQL database when a file is decrypted. The logincludes a timestamp, the username SOPS is running as, and the file that wasdecrypted.

In order to enable auditing, you must first create the database and credentialsusing the schema found inaudit/schema.sql. This schema defines thetables that store the audit events and a role namedsops that only haspermission to add entries to the audit event tables. The default password forthe rolesops issops. You should change this password.

Once you have created the database, you have to tell SOPS how to connect to it.Because we don't want users of SOPS to be able to control auditing, the auditconfiguration file location is not configurable, and must be at/etc/sops/audit.yaml. This file should have strict permissions suchthat only the root user can modify it.

For example, to enable auditing to a PostgreSQL database namedsops runningon localhost, using the usersops and the passwordsops,/etc/sops/audit.yaml should have the following contents:

backends:postgres:        -connection_string:"postgres://sops:sops@localhost/sops?sslmode=verify-full"

You can find more information on theconnection_string format in thePostgreSQL docs.

Under thepostgres map entry in the above YAML is a list, so one canprovide more than one backend, and SOPS will log to all of them:

backends:postgres:        -connection_string:"postgres://sops:sops@localhost/sops?sslmode=verify-full"        -connection_string:"postgres://sops:sops@remotehost/sops?sslmode=verify-full"

By default SOPS just dumps all the output to the standard output. We can use the--output flag followed by a filename to save the output to the file specified.Beware using both--in-place and--output flags will result in an error.

In addition to writing secrets to standard output and to files on disk, SOPShas two commands for passing decrypted secrets to a new process:exec-envandexec-file. These commands will place all output into the environment ofa child process and into a temporary file, respectively. For example, if aprogram looks for credentials in its environment,exec-env can be used toensure that the decrypted contents are available only to this process and neverwritten to disk.

# print secrets to stdout to confirm values$ sops decrypt out.json{"database_password":"jf48t9wfw094gf4nhdf023r","AWS_ACCESS_KEY_ID":"AKIAIOSFODNN7EXAMPLE","AWS_SECRET_KEY":"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"}# decrypt out.json and run a command# the command prints the environment variable and runs a script that uses it$ sops exec-env out.json'echo secret: $database_password; ./database-import'secret: jf48t9wfw094gf4nhdf023r# launch a shell with the secrets available in its environment$ sops exec-env out.json'sh'sh-3.2#echo$database_passwordjf48t9wfw094gf4nhdf023r# the secret is not accessible anywhere elsesh-3.2$exit$echo your password:$database_passwordyour password:

If you want process signals to be sent to the command, for example if you arerunningexec-env to launch a server and your server handles SIGTERM, then the--same-process flag can be used to instructsops to start your command inthe same process instead of a child process. This uses theexecve system calland is supported on Unix-like systems.

If the command you want to run only operates on files, you can useexec-fileinstead. By default, SOPS will use a FIFO to pass the contents of thedecrypted file to the new program. Using a FIFO, secrets are only passed inmemory which has two benefits: the plaintext secrets never touch the disk, andthe child process can only read the secrets once. In contexts where this won'twork, eg platforms like Windows where FIFOs unavailable or secret files that needto be available to the child process longer term, the--no-fifo flag can beused to instruct SOPS to use a traditional temporary file that will get cleanedup once the process is finished executing.exec-file behaves similar tofind(1) in that{} is used as a placeholder in the command which will besubstituted with the temporary file path (whether a FIFO or an actual file).

# operating on the same file as before, but as a file this time$ sops exec-file out.json'echo your temporary file: {}; cat {}'your temporary file: /tmp/.sops894650499/tmp-file{"database_password":"jf48t9wfw094gf4nhdf023r","AWS_ACCESS_KEY_ID":"AKIAIOSFODNN7EXAMPLE","AWS_SECRET_KEY":"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"}# launch a shell with a variable TMPFILE pointing to the temporary file$ sops exec-file --no-fifo out.json'TMPFILE={} sh'sh-3.2$echo$TMPFILE/tmp/.sops506055069/tmp-file291138648sh-3.2$ cat$TMPFILE{"database_password":"jf48t9wfw094gf4nhdf023r","AWS_ACCESS_KEY_ID":"AKIAIOSFODNN7EXAMPLE","AWS_SECRET_KEY":"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"}sh-3.2$ ./program --config$TMPFILEsh-3.2$exit# try to open the temporary file from earlier$ cat /tmp/.sops506055069/tmp-file291138648cat: /tmp/.sops506055069/tmp-file291138648: No such file or directory

Additionally, on unix-like platforms, bothexec-env andexec-filesupport dropping privileges before executing the new program via the--user <username> flag. This is particularly useful in cases where theencrypted file is only readable by root, but the target program does notneed root privileges to function. This flag should be used where possiblefor added security.

To overwrite the default file name (tmp-file) inexec-file use the--filename <filename> parameter.

# the encrypted file can't be read by the current user$ cat out.jsoncat: out.json: Permission denied# execute sops as root, decrypt secrets, then drop privileges$ sudo sops exec-env --user nobody out.json'sh'sh-3.2$echo$database_passwordjf48t9wfw094gf4nhdf023r# dropped privileges, still can't load the original filesh-3.2$ iduid=4294967294(nobody) gid=4294967294(nobody) groups=4294967294(nobody)sh-3.2$ cat out.jsoncat: out.json: Permission denied

sops publish $file publishes a file to a pre-configured destination (this lives in the SOPSconfig file). Additionally, support re-encryption rules that work just like the creation rules.

This command requires a.sops.yaml configuration file. Below is an example:

destination_rules:    -s3_bucket:"sops-secrets"path_regex:s3/*recreation_rule:pgp:F69E4901EDBAD2D1753F8C67A64535C4163FB307    -gcs_bucket:"sops-secrets"path_regex:gcs/*recreation_rule:pgp:F69E4901EDBAD2D1753F8C67A64535C4163FB307    -vault_path:"sops/"vault_kv_mount_name:"secret/"# defaultvault_kv_version:2# defaultpath_regex:vault/*omit_extensions:true

The above configuration will place all files unders3/* into the S3 bucketsops-secrets,all files undergcs/* into the GCS bucketsops-secrets, and the contents of all files undervault/* into Vault's KV store under the pathsecrets/sops/. For the files that will bepublished to S3 and GCS, it will decrypt them and re-encrypt them using theF69E4901EDBAD2D1753F8C67A64535C4163FB307 pgp key.

You would deploy a file to S3 with a command like:sops publish s3/app.yaml

To publish all files in selected directory recursively, you need to specify--recursive flag.

If you don't want file extension to appear in destination secret path, use--omit-extensionsflag oromit_extensions: true in the destination rule in.sops.yaml.

There are a few settings for Vault that you can place in your destination rules. The firstisvault_path, which is required. The others are optional, and they arevault_address,vault_kv_mount_name,vault_kv_version.

SOPS uses the official Vault API provided by Hashicorp, which makes use ofenvironmentvariables forconfiguring the client.

vault_kv_mount_name is used if your Vault KV is mounted somewhere other thansecret/.vault_kv_version supports1 and2, with2 being the default.

If the destination secret path already exists in Vault and contains the same data as the sourcefile, it will be skipped.

Below is an example of publishing to Vault (using token auth with a local dev instance of Vault).

$export VAULT_TOKEN=...$export VAULT_ADDR='http://127.0.0.1:8200'$ sops decrypt vault/test.yamlexample_string: barexample_number: 42example_map:    key: value$ sops publish vault/test.yamluploading /home/user/sops_directory/vault/test.yaml to http://127.0.0.1:8200/v1/secret/data/sops/test.yaml? (y/n): y$ vault kv get secret/sops/test.yaml====== Metadata ======Key              Value---              -----created_time     2019-07-11T03:32:17.074792017Zdeletion_time    n/adestroyedfalseversion          3========= Data =========Key               Value---               -----example_map       map[key:value]example_number    42example_string    bar

SOPS uses the file extension to decide which encryption method to use on the filecontent.YAML,JSON,ENV, andINI files are treated as trees of data, and key/values areextracted from the files to only encrypt the leaf values. The tree structure is alsoused to check the integrity of the file.

Therefore, if a file is encrypted using a specific format, it needs to be decryptedin the same format. The easiest way to achieve this is to conserve the original fileextension after encrypting a file. For example:

$ sops encrypt -i myfile.json$ sops decrypt myfile.json

If you want to change the extension of the file once encrypted, you need to providesops with the--input-type flag upon decryption. For example:

$ sops encrypt myfile.json> myfile.json.enc$ sops decrypt --input-type json myfile.json.enc

When operating on stdin, use the--input-type and--output-type flags as follows:

$ cat myfile.json| sops decrypt --input-type json --output-type json

SOPS indentsJSON files by default using onetab. However, you can changethis default behaviour to usespaces by either using the additional--indent=2 CLI option orby configuring.sops.yaml with the code below.

The special value0 disables indentation, and-1 uses a single tab.

stores:json:indent:2json_binary:indent:2

SOPS indentsYAML files by default using 4 spaces. However, you can changethis default behaviour by either using the additional--indent=2 CLI option orby configuring.sops.yaml with:

stores:yaml:indent:2

SOPS only supports a subset ofYAML's many types. Encrypting YAML files thatcontain strings, numbers and booleans will work fine, but files that contain anchorswill not work, because the anchors redefine the structure of the file at load time.

This file will not work in SOPS:

bill-to:&id001street:|        123 Tornado Alley        Suite 16city:East Centervillestate:KSship-to:*id001

SOPS uses the path to a value as additional data in the AEAD encryption, and thusdynamic paths generated by anchors break the authentication step.

JSON and TEXT file types do not support anchors and thus have no such limitation.

YAML supports having more than one "document" in a single file, whileformats likeJSON do not. SOPS is able to handle both. This means thefollowing multi-document will be encrypted as expected:

---data: foo---data: bar

Note that thesops metadata, i.e. the hash, etc, is computed for the physicalfile rather than each internal "document".

YAML andJSON top-level arrays are not supported, because SOPSneeds a top-levelsops key to store its metadata.

This file will not work in SOPS:

---  -some  -array  -elements

But this one will work because thesops key can be added at the same level as thedata key.

data:    -some    -array    -elements

Similarly, withJSON arrays, this document will not work:

["some","array","elements"]

But this one will work just fine:

{"data": ["some","array","elements"  ]}

Take a look into theexamples folder for detailed use cases of SOPS in a CI environment. The section below describes specific tips for common use cases.

The command below creates a new file with a data key encrypted by KMS and PGP.

$ sops edit --kms"arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500" --pgp C9CAB0AF1165060DB58D6D6B2653B624D620786D /path/to/new/file.yaml

Similar to the previous command, we tell SOPS to use one KMS and one PGP key.The path points to an existing cleartext file, so we givesops the flag-e toencrypt the file, and redirect the output to a destination file.

$export SOPS_KMS_ARN="arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500"$export SOPS_PGP_FP="C9CAB0AF1165060DB58D6D6B2653B624D620786D"$ sops encrypt /path/to/existing/file.yaml> /path/to/new/encrypted/file.yaml

Decrypt the file with-d.

$ sops decrypt /path/to/new/encrypted/file.yaml

Rather than redirecting the output of-e or-d,sops can replace theoriginal file after encrypting or decrypting it.

# file.yaml is in cleartext$ sops encrypt -i /path/to/existing/file.yaml# file.yaml is now encrypted$ sops decrypt -i /path/to/existing/file.yaml# file.yaml is back in cleartext

SOPS primary use case is encrypting YAML and JSON configuration files, but italso has the ability to manage binary files. When encrypting a binary, SOPS willread the data as bytes, encrypt it, store the encrypted base64 undertree['data'] and write the result as JSON.

Note that the base64 encoding of encrypted data can actually make the encryptedfile larger than the cleartext one.

In-place encryption/decryption also works on binary files.

$ dd if=/dev/urandom of=/tmp/somerandom bs=1024count=512512+0 recordsin512+0 records out524288 bytes (524 kB) copied, 0.0466158 s, 11.2 MB/s$ sha512sum /tmp/somerandom9589bb20280e9d381f7a192000498c994e921b3cdb11d2ef5a986578dc2239a340b25ef30691bac72bdb14028270828dad7e8bd31e274af9828c40d216e60cbe /tmp/somerandom$ sops encrypt -i /tmp/somerandompleasewaitwhile a data encryption key is being generated and stored securely$ sops decrypt -i /tmp/somerandom$ sha512sum /tmp/somerandom9589bb20280e9d381f7a192000498c994e921b3cdb11d2ef5a986578dc2239a340b25ef30691bac72bdb14028270828dad7e8bd31e274af9828c40d216e60cbe /tmp/somerandom

SOPS can extract a specific part of a YAML or JSON document, by provided thepath in the--extract command line flag. This is useful to extract specificvalues, like keys, without needing an extra parser.

$ sops decrypt --extract'["app2"]["key"]'~/git/svc/sops/example.yaml-----BEGIN RSA PRIVATE KEY-----MIIBPAIBAAJBAPTMNIyHuZtpLYc7VsHQtwOkWYobkUblmHWRmbXzlAX6K8tMf3WfImcbNkqAKnELzFAPSBeEMhrBN0PyOC9lYlMCAwEAAQJBALXD4sjuBn1E7Y9aGiMzbJEBuZJ4wbhYxomVoQKfaCu+kH80uLFZKoSz85/ySauWE8LgZcMLIBoiXNhDKfQLvHECIQD6tCG9NMFWor69kgbX8vK5Y+QL+kRq+9HK6yZ9a+hsLQIhAPn4Ie6HGTjwfHSTXWZpGSan7NwTkIu4U5q2SlLjcZh/AiEA78NYRRBwGwAYNUqzutGBqyXKUl4uErb0xAEyVV7e8J0CIQC8VBY8f8yg+Y7Kxbw4zDYGyb3KkXL10YorpeuZR4LuQQIgbKGPkMM4w5blyE1tqGN0T7sJwEx+EUOgacRNqM2ljVA=-----END RSA PRIVATE KEY-----

The tree path syntax uses regular python dictionary syntax, without thevariable name. Extract keys by naming them, and array elements by numberingthem.

$ sops decrypt --extract'["an_array"][1]'~/git/svc/sops/example.yamlsecretuser2

SOPS can set a specific part of a YAML or JSON document, by providingthe path and value in theset command. This is useful to set specificvalues, like keys, without needing an editor.

$ sopsset~/git/svc/sops/example.yaml'["app2"]["key"]''"app2keystringvalue"'

The tree path syntax uses regular python dictionary syntax, without thevariable name. Set to keys by naming them, and array elements bynumbering them.

$ sopsset~/git/svc/sops/example.yaml'["an_array"][1]''"secretuser2"'

The value must be formatted as json.

$ sopsset~/git/svc/sops/example.yaml'["an_array"][1]''{"uid1":null,"uid2":1000,"uid3":["bob"]}'

Symmetrically, SOPS can unset a specific part of a YAML or JSON document, by providingthe path in theunset command. This is useful to unset specific values, like keys, withoutneeding an editor.

$ sopsunset~/git/svc/sops/example.yaml'["app2"]["key"]'

The tree path syntax uses regular python dictionary syntax, without thevariable name. Set to keys by naming them, and array elements bynumbering them.

$ sopsunset~/git/svc/sops/example.yaml'["an_array"][1]'

You most likely want to store encrypted files in a version controlled repository.SOPS can be used with git to decrypt files when showing diffs between versions.This is very handy for reviewing changes or visualizing history.

To configure SOPS to decrypt files during diff, create a.gitattributes fileat the root of your repository that contains a filter and a command.

*.yaml diff=sopsdiffer

Here we only care about YAML files.sopsdiffer is an arbitrary name that we mapto a SOPS command in the git configuration file of the repository.

$ git config diff.sopsdiffer.textconv"sops decrypt"$ grep -A 1 sopsdiffer .git/config[diff"sopsdiffer"]    textconv ="sops decrypt"

With this in place, calls togit diff will decrypt both previous and currentversions of the target file prior to displaying the diff. And it even works withgit client interfaces, because they call git diff under the hood!

Note: this only works on YAML and JSON files, not on BINARY files.

By default, SOPS encrypts all the values of a YAML or JSON file and leaves thekeys in cleartext. In some instances, you may want to exclude some values frombeing encrypted. This can be accomplished by adding the suffix_unencryptedto any key of a file. When set, all values underneath the key that set the_unencrypted suffix will be left in cleartext.

Note that, while in cleartext, unencrypted content is still added to thechecksum of the file, and thus cannot be modified outside of SOPS withoutbreaking the file integrity check.This behavior can be modified using--mac-only-encrypted flag or.sops.yamlconfig file which makes SOPS compute a MAC only over values it encrypted andnot all values.

The unencrypted suffix can be set to a different value using the--unencrypted-suffix option.

Conversely, you can opt in to only encrypt some values in a YAML or JSON file,by adding a chosen suffix to those keys and passing it to the--encrypted-suffix option.

A third method is to use the--encrypted-regex which will only encrypt values underkeys that match the supplied regular expression. For example, this command:

$ sops encrypt --encrypted-regex'^(data|stringData)$' k8s-secrets.yaml

will encrypt the values under thedata andstringData keys in a YAML filecontaining kubernetes secrets. It will not encrypt other values that help you tonavigate the file, likemetadata which contains the secrets' names.

Conversely, you can opt in to only leave certain keys without encrypting by using the--unencrypted-regex option, which will leave the values unencrypted of those keysthat match the supplied regular expression. For example, this command:

$ sops encrypt --unencrypted-regex'^(description|metadata)$' k8s-secrets.yaml

will not encrypt the values under thedescription andmetadata keys in a YAML filecontaining kubernetes secrets, while encrypting everything else.

For YAML files, another method is to use--encrypted-comment-regex which willonly encrypt comments and values which have a preceding comment matching the suppliedregular expression.

Conversely, you can opt in to only left certain keys without encrypting by using the--unencrypted-comment-regex option, which will leave the values and commentsunencrypted when they have a preeceding comment, or a trailing comment on the same line,that matches the supplied regular expression.

You can also specify these options in the.sops.yaml config file.

Note: these six options--unencrypted-suffix,--encrypted-suffix,--encrypted-regex,--unencrypted-regex,--encrypted-comment-regex, and--unencrypted-comment-regex aremutually exclusive and cannot all be used in the same file.

When SOPS creates a file, it generates a random 256 bit data key and asks eachKMS and PGP master key to encrypt the data key. The encrypted version of the datakey is stored in thesops metadata undersops.kms andsops.pgp.

For KMS:

sops:kms:        -enc:CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAQB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAyGdRODuYMHbA8Ozj8CARCAO7opMolPJUmBXd39Zlp0L2H9fzMKidHm1vvaF6nNFq0ClRY7FlIZmTm4JfnOebPseffiXFn9tG8cq7oienc_ts:1439568549.245995arn:arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e

For PGP:

sops:pgp:        -fp:85D77543B3D624B63CEA9E6DBC17301B491B3F21created_at:1441570391.930042enc:|              -----BEGIN PGP MESSAGE-----              Version: GnuPG v1              hQIMA0t4uZHfl9qgAQ//UvGAwGePyHuf2/zayWcloGaDs0MzI+zw6CmXvMRNPUsA              pAgRKczJmDu4+XzN+cxX5Iq9xEWIbny9B5rOjwTXT3qcUYZ4Gkzbq4MWkjuPp/Iv              qO4MJaYzoH5YxC4YORQ2LvzhA2YGsCzYnljmatGEUNg01yJ6r5mwFwDxl4Nc80Cn              RwnHuGExK8j1jYJZu/juK1qRbuBOAuruIPPWVdFB845PA7waacG1IdUW3ZtBkOy3              O0BIfG2ekRg0Nik6sTOhDUA+l2bewCcECI8FYCEjwHm9Sg5cxmP2V5m1mby+uKAm              kewaoOyjbmV1Mh3iI1b/AQMr+/6ZE9MT2KnsoWosYamFyjxV5r1ZZM7cWKnOT+tu              KOvGhTV1TeOfVpajNTNwtV/Oyh3mMLQ0F0HgCTqomQVqw5+sj7OWAASuD3CU/dyo              pcmY5Qe0TNL1JsMNEH8LJDqSh+E0hsUxdY1ouVsg3ysf6mdM8ciWb3WRGxih1Vmf              unfLy8Ly3V7ZIC8EHV8aLJqh32jIZV4i2zXIoO4ZBKrudKcECY1C2+zb/TziVAL8              qyPe47q8gi1rIyEv5uirLZjgpP+JkDUgoMnzlX334FZ9pWtQMYW4Y67urAI4xUq6              /q1zBAeHoeeeQK+YKDB7Ak/Y22YsiqQbNp2n4CKSKAE4erZLWVtDvSp+49SWmS/S              XgGi+13MaXIp0ecPKyNTBjF+NOw/I3muyKr8EbDHrd2XgIT06QXqjYLsCb1TZ0zm              xgXsOTY3b+ONQ2zjhcovanDp7/k77B+gFitLYKg4BLZsl7gJB12T8MQnpfSmRT4=              =oJgS              -----END PGP MESSAGE-----

SOPS then opens a text editor on the newly created file. The user adds data to thefile and saves it when done.

Upon save, SOPS browses the entire file as a key/value tree. Every time SOPSencounters a leaf value (a value that does not have children), it encrypts thevalue with AES256_GCM using the data key and a 256 bit random initializationvector.

Each file uses a single data key to encrypt all values of a document, but eachvalue receives a unique initialization vector and has unique authentication data.

Additional data is used to guarantee the integrity of the encrypted dataand of the tree structure: when encrypting the tree, key names are concatenatedinto a byte string that is used as AEAD additional data (aad) when encryptingvalues. We expect that keys do not carry sensitive information, andkeeping them in cleartext allows for better diff and overall readability.

Any valid KMS or PGP master key can later decrypt the data key and access thedata.

Multiple master keys allow for sharing encrypted files without sharing masterkeys, and provide a disaster recovery solution. The recommended way to use SOPSis to have two KMS master keys in different regions and one PGP public key withthe private key stored offline. If, by any chance, both KMS master keys arelost, you can always recover the encrypted data using the PGP private key.

In addition to authenticating branches of the tree using keys as additionaldata, SOPS computes a MAC on all the values to ensure that no value has beenadded or removed fraudulently. The MAC is stored encrypted with AES_GCM andthe data key under tree ->sops ->mac.This behavior can be modified using--mac-only-encrypted flag or.sops.yamlconfig file which makes SOPS compute a MAC only over values it encrypted andnot all values.

📝A note from the maintainers

This section was written by the original authors of SOPS while they wereworking at Mozilla. It is kept here for historical reasons and to providetechnical background on the project. It is not necessarily representativeof the views of the current maintainers, nor are they currently affiliatedwith Mozilla.

Automating the distribution of secrets and credentials to components of aninfrastructure is a hard problem. We know how to encrypt secrets and share thembetween humans, but extending that trust to systems is difficult. Particularlywhen these systems follow devops principles and are created and destroyedwithout human intervention. The issue boils down to establishing the initialtrust of a system that just joined the infrastructure, and providing it accessto the secrets it needs to configure itself.

In many infrastructures, even highly dynamic ones, the initial trust isestablished by a human. An example is seen in Puppet by the way certificates areissued: when a new system attempts to join a Puppetmaster, an administratormust, by default, manually approve the issuance of the certificate the systemneeds. This is cumbersome, and many puppetmasters are configured to auto-signnew certificates to work around that issue. This is obviously not recommendedand far from ideal.

AWS provides a more flexible approach to trusting new systems. It uses apowerful mechanism of roles and identities. In AWS, it is possible to verifythat a new system has been granted a specific role at creation, and it ispossible to map that role to specific resources. Instead of trusting new systemsdirectly, the administrator trusts the AWS permission model and its automationinfrastructure. As long as AWS keys are safe, and the AWS API is secure, we canassume that trust is maintained and systems are who they say they are.

Using the AWS trust model, we can create fine grained access controls toAmazon's Key Management Service (KMS). KMS is a service that encrypts anddecrypts data with AES_GCM, using keys that are never visible to users of theservice. Each KMS master key has a set of role-based access controls, andindividual roles are permitted to encrypt or decrypt using the master key. KMShelps solve the problem of distributing keys, by shifting it into an accesscontrol problem that can be solved using AWS's trust model.

When Mozilla's Services Operations team started revisiting the issue ofdistributing secrets to EC2 instances, we set a goal to store these secretsencrypted until the very last moment, when they need to be decrypted on targetsystems. Not unlike many other organizations that operate sufficiently complexautomation, we found this to be a hard problem with a number of prerequisites:

1. Secrets must be stored in YAML files for easy integration into hiera
2. Secrets must be stored in GIT, and when a new CloudFormation stack isbuilt, the current HEAD is pinned to the stack. (This allows secrets tobe changed in GIT without impacting the current stack that mayautoscale).
3. Entries must be encrypted separately. Encrypting entire files as blobs makesgit conflict resolution almost impossible. Encrypting each entryseparately is much easier to manage.
4. Secrets must always be encrypted on disk (admin laptop, upstreamgit repo, jenkins and S3) and only be decrypted on the targetsystems

SOPS can be used to encrypt YAML, JSON and BINARY files. In BINARY mode, thecontent of the file is treated as a blob, the same way PGP would encrypt anentire file. In YAML and JSON modes, however, the content of the file ismanipulated as a tree where keys are stored in cleartext, and values areencrypted. hiera-eyaml does something similar, and over the years we learnedto appreciate its benefits, namely:

• diffs are meaningful. If a single value of a file is modified, only thatvalue will show up in the diff. The diff is still limited to only showingencrypted data, but that information is already more granular thatindicating that an entire file has changed.
• conflicts are easier to resolve. If multiple users are working on thesame encrypted files, as long as they don't modify the same values,changes are easy to merge. This is an improvement over the PGPencryption approach where unsolvable conflicts often happen whenmultiple users work on the same file.

OpenPGP gets a lot of bad press for being an outdated crypto protocol, and whiletrue, what really made us look for alternatives is the difficulty of managing anddistributing keys to systems. With KMS, we manage permissions to an API, not keys,and that's a lot easier to do.

But PGP is not dead yet, and we still rely on it heavily as a backup solution:all our files are encrypted with KMS and with one PGP public key, with itsprivate key stored securely for emergency decryption in the event that we loseall our KMS master keys.

SOPS can be used without KMS entirely, the same way you would use an encryptedPGP file: by referencing the pubkeys of each individual who has access to the file.It can easily be done by providing SOPS with a comma-separated list of public keyswhen creating a new file:

$ sops edit --pgp"E60892BB9BD89A69F759A1A0A3D652173B763E8F,84050F1D61AF7C230A12217687DF65059EF093D3,85D77543B3D624B63CEA9E6DBC17301B491B3F21" mynewfile.yaml

The security of the data stored using SOPS is as strong as the weakestcryptographic mechanism. Values are encrypted using AES256_GCM which is thestrongest symmetric encryption algorithm known today. Data keys are encryptedin either KMS, which also uses AES256_GCM, or PGP which uses either RSA orECDSA keys.

Going from the most likely to the least likely, the threats are as follows:

An attacker with access to an AWS console can grant itself access to one ofthe KMS master keys used to encrypt asops data key. This threat should bemitigated by protecting AWS accesses with strong controls, such as multi-factorauthentication, and also by performing regular audits of permissions grantedto AWS users.

PGP keys are routinely mishandled, either because owners copy them frommachine to machine, or because the key is left forgotten on an unused machinean attacker gains access to. When using PGP encryption, SOPS users should takespecial care of PGP private keys, and store them on smart cards or offlineas often as possible.

SOPS doesn't apply any restriction on the size or type of PGP keys. A weak PGPkeys, for example 512 bits RSA, could be factorized by an attacker to gainaccess to the private key and decrypt the data key. Users of SOPS should relyon strong keys, such as 2048+ bits RSA keys, or 256+ bits ECDSA keys.

A vulnerability in AES256_GCM could potentially leak the data key or the KMSmaster key used by a SOPS encrypted file. While no such vulnerability existstoday, we recommend that users keep their encrypted files reasonably private.

SOPS will remain backward compatible on the major version, meaning that allimprovements brought to the 1.X and 2.X branches (current) will maintain thefile format introduced in1.0.

Please report any security issues privately usingGitHub's advisory form.

Mozilla Public License Version 2.0

SOPS was initially launched as a project at Mozilla in 2015 and has beengraciously donated to the CNCF as a Sandbox project in 2023, now under thestewardship of anew group of maintainers.

The original authors of the project were:

• Adrian Utrilla @autrilla
• Julien Vehent @jvehent

Furthermore, the project has been carried for a long time by AJ Bahnken @ajvb,and had not been possible without the contributions of numerouscontributors.

SOPS was inspired byhiera-eyaml,credstash,sneaker,password store and too many years managingPGP encrypted files by hand...

We are aCloud Native Computing Foundationsandbox project.