

TheNetworkOptions
package acts as a mediator between ways of configuringnetwork transport mechanisms (SSL/TLS, SSH, proxies, etc.) and Julia packagesthat provide access to transport mechanisms. This allows the a common interfaceto configuring things like TLS and SSH host verification and proxies viaenvironment variables (currently) and other configuration mechanisms (in thefuture), while packages that need to configure these mechanisms can simplyaskNetworkOptions
what to do in specific situations without worrying abouthow that configuration is expressed.
ca_roots()::Union{Nothing, String}
Theca_roots()
function tells the caller where, if anywhere, to find a file ordirectory of PEM-encoded certificate authority roots. By default, on systemslike Windows and macOS where the built-in TLS engines know how to verify hostsusing the system's built-in certificate verification mechanism, this functionwill returnnothing
. On classic UNIX systems (excluding macOS), rootcertificates are typically stored in a file in/etc
: the common places for thecurrent UNIX system will be searched and if one of these paths exists, it willbe returned; if none of these typical root certificate paths exist, then thepath to the set of root certificates that are bundled with Julia is returned.
The default value returned byca_roots()
may be overridden by setting theJULIA_SSL_CA_ROOTS_PATH
,SSL_CERT_DIR
, orSSL_CERT_FILE
environmentvariables, in which case this function will always return the value of the firstof these variables that is set (whether the path exists or not). IfJULIA_SSL_CA_ROOTS_PATH
is set to the empty string, then the other variablesare ignored (as if unset); if the other variables are set to the empty string,they behave is if they are not set.
Theca_roots_path()
function is similar to theca_roots()
function exceptthat it always returns a path to a file or directory of PEM-encoded certificateauthority roots. When called on a system like Windows or macOS, where systemroot certificates are not stored in the file system, it will currently returnthe path to the set of root certificates that are bundled with Julia. (In thefuture, this function may instead extract the root certificates from the systemand save them to a file whose path would be returned.)
If it is possible to configure a library that uses TLS to use the systemcertificates that is generally preferable: i.e. it is better to useca_roots()
which returnsnothing
to indicate that the system certs should beused. Theca_roots_path()
function should only be used when configuringlibraries whichrequire a path to a file or directory for root certificates.
The default value returned byca_roots_path()
may be overridden by setting theJULIA_SSL_CA_ROOTS_PATH
,SSL_CERT_DIR
, orSSL_CERT_FILE
environmentvariables, in which case this function will always return the value of the firstof these variables that is set (whether the path exists or not). IfJULIA_SSL_CA_ROOTS_PATH
is set to the empty string, then the other variablesare ignored (as if unset); if the other variables are set to the empty string,they behave is if they are not set.
Thessh_dir()
function returns the location of the directory where thessh
program keeps/looks for configuration files. By default this is~/.ssh
butthis can be overridden by setting the environment variableSSH_DIR
.
Thessh_key_name()
function returns the base name of key files that SSH shoulduse for when establishing a connection. There is usually no reason that thisfunction should be called directly and libraries should generally use thessh_key_path
andssh_pub_key_path
functions to get full paths. If theenvironment variableSSH_KEY_NAME
is set then this function returns that;otherwise it returnsid_rsa
by default.
Thessh_key_path()
function returns the path of the SSH private key file thatshould be used for SSH connections. If theSSH_KEY_PATH
environment variableis set then it will return that value. Otherwise it defaults to returning
joinpath(ssh_dir(),ssh_key_name())
This default value in turn depends on theSSH_DIR
andSSH_KEY_NAME
environment variables.
ssh_pub_key_path()::String
Thessh_pub_key_path()
function returns the path of the SSH public key filethat should be used for SSH connections. If theSSH_PUB_KEY_PATH
environmentvariable is set then it will return that value. If that isn't set butSSH_KEY_PATH
is set, it will return that path with the.pub
suffix appended.If neither is set, it defaults to returning
joinpath(ssh_dir(),ssh_key_name()*".pub")
This default value in turn depends on theSSH_DIR
andSSH_KEY_NAME
environment variables.
Thessh_key_pass()
function returns the value of the environment variableSSH_KEY_PASS
if it is set ornothing
if it is not set. In the future, thismay be able to find a password by other means, such as secure system storage, sopackages that need a password to decrypt an SSH private key should use this APIinstead of directly checking the environment variable so that they gain suchcapabilities automatically when they are added.
ssh_known_hosts_files()::Vector{String}
Thessh_known_hosts_files()
function returns a vector of paths of SSH knownhosts files that should be used when establishing the identities of remoteservers for SSH connections. By default this function returns
[joinpath(ssh_dir(),"known_hosts"), bundled_known_hosts]
wherebundled_known_hosts
is the path of a copy of a known hosts file that isbundled with this package (containing known hosts keys forgithub.com
andgitlab.com
). If the environment variableSSH_KNOWN_HOSTS_FILES
is set,however, then its value is split into paths on the:
character (or on;
onWindows) and this vector of paths is returned instead. If any component of thisvector is empty, it is expanded to the default known hosts paths.
Packages that usessh_known_hosts_files()
should ideally look for matchingentries by comparing the host name and key types, considering the first entry inany of the files which matches to be the definitive identity of the host. If thecaller cannot compare the key type (e.g. because it has been hashes) then itmust approximate the above algorithm by looking for all matching entries for ahost in each file: if a file has any entries for a host then one of them mustmatch; the caller should only continue to search further known hosts files ifthere are no entries for the host in question in an earlier file.
ssh_known_hosts_file()::String
Thessh_known_hosts_file()
function returns a single path of an SSH knownhosts file that should be used when establishing the identities of remoteservers for SSH connections. It returns the first path returned byssh_known_hosts_files
that actually exists. Callers who can look in more thanone known hosts file should usessh_known_hosts_files
instead and look forhost matches in all the files returned as described in that function's docs.
verify_host(url::AbstractString, [transport::AbstractString])::Bool
Theverify_host
function tells the caller whether the identity of a hostshould be verified when communicating over secure transports like TLS or SSH.Theurl
argument may be:
- a proper URL staring with
proto://
- an
ssh
-style bare host name or host name prefixed withuser@
- an
scp
-style host as above, followed by:
and a path location
In each case the host name part is parsed out and the decision about whether toverify or not is made based solely on the host name, not anything else about theinput URL. In particular, the protocol of the URL does not matter (more below).
Thetransport
argument indicates the kind of transport that the query isabout. The currently known values areSSL
(aliasTLS
) andSSH
. If thetransport is omitted, the query will returntrue
only if the host name shouldnot be verified regardless of transport.
The host name is matched against the host patterns in the relevant environmentvariables depending on whethertransport
is supplied and what its value is:
JULIA_NO_VERIFY_HOSTS
— hosts that should not be verified for any transportJULIA_SSL_NO_VERIFY_HOSTS
— hosts that should not be verified for SSL/TLSJULIA_SSH_NO_VERIFY_HOSTS
— hosts that should not be verified for SSHJULIA_ALWAYS_VERIFY_HOSTS
— hosts that should always be verified
The values of each of these variables is a comma-separated list of host namepatterns with the following syntax — each pattern is split on.
into parts andeach part must one of:
- A literal domain name component consisting of one or more ASCII letter,digit, hyphen or underscore (technically not part of a legal host name, butsometimes used). A literal domain name component matches only itself.
- A
**
, which matches zero or more domain name components. - A
*
, which match any one domain name component.
When matching a host name against a pattern list in one of these variables, thehost name is split on.
into components and that sequence of words is matchedagainst the pattern: a literal pattern matches exactly one host name componentwith that value; a*
pattern matches exactly one host name component with anyvalue; a**
pattern matches any number of host name components. For example:
**
matches any host name**.org
matches any host name in the.org
top-level domainexample.com
matches only the exact host nameexample.com
*.example.com
matchesapi.example.com
but notexample.com
orv1.api.example.com
**.example.com
matches any domain underexample.com
, includingexample.com
itself,api.example.com
andv1.api.example.com
Suppose you want to not verify any hosts undersafe.example.com
for allprotocols, skip SSL host verification for justssl.example.com
, and skip SSHhost verification forssh.example.com
and its immediate first levelsubdomains. Then you could set the following environment variable values:
export JULIA_NO_VERIFY_HOSTS="**.safe.example.com"export JULIA_SSL_NO_VERIFY_HOSTS="ssl.example.com"export JULIA_SSH_NO_VERIFY_HOSTS="ssh.example.com,*.ssh.example.com"
With this configuration:
example.com
would be verified for all protocolssafe.example.com
,api.safe.example.com
,v1.api.safe.example.com
and soon would be unverified for all transportsssl.example.com
would be unverified for SSL/TLS transportsub.ssl.example.com
would be verified for all transports, including SSL/TLSssh.example.com
andsub.ssh.example.com
would be unverified for SSH onlysub.sub.ssh.example.com
would be verified for all transports
Note that the protocol ofurl
need not match the transport mechanism beingqueried: the protocol of the URL is entirely discarded. The reason for this isthat the typical usage of this utility function is to configure a library toenable or disable specific features like TLS host verification based on a URL.If the URL does not actually use the TLS transport mechanism, then it doesn'tmatter if verification for that transport is enabled or not. Moreover, differentprotocols can use the same transport: for example,https
andftps
protocolsboth use TLS andssh
,scp
andsftp
protocols all use SSH.
A common scenario that occur behind firewalls is for all connections to externalsystems to go through a transparent man-in-the-middle proxy: any SSL/TLSconnection to a host underexample.com
would be internal and should have avalid certificate but any connection outside ofexample.com
would go throughthe proxy, which uses a self-signed certificate. For such a scenario the bestsolution would be to deploy a CA root certificate to all clients, but if that'snot possible, then configuring clients to verify hosts underexample.com
butnot verify other SSL/TLS connections would be a viable solution. In fact, aslong as the man-in-the-middle proxy verifies all upstream TLS connections, thisis still secure (although not private from the proxy, of course). Such aconfiguration can be accomplished with the following exports:
export JULIA_ALWAYS_VERIFY_HOSTS="**.example.com"export JULIA_SSL_NO_VERIFY_HOSTS="**"
This configuration causes all domains underexample.com
to always be verifiedfor all protocols, including SSL/TLS, while skipping host verification for SSL/TLSconnections to all other hosts.