Cloud providers have made a killing from neatly-packaged managed services for years. Whether it be databases or message brokers, developers like ourselves don't seem to have a problem paying a bit extra to have things taken care of. But wait, aren't we typically thelast people to opt for less optimization and less control? Why is this the time we decide otherwise? If I had to make a guess, I'd wager it's partly because server-side DevOps kind of sucks.

As a developer, configuring or debugging a VPS is usually work which is unaccounted for, and it isn't particularly rewarding. At best, your application will probably end up running the same as your local environment. How could we make this inevitable part of our jobs better? Well, we could automate it.

Paramiko andSCP are two Python libraries we can use together to automate tasks we'd want to run on a remote host such as restarting services, making updates, or grabbing log files. We're going to take a look at what scripting with these libraries looks like. Fair warning: there's a sizable amount of code in this tutorial, which tends to make me excited enough to coerce others into my manic code-tutorial episodes. If you start to feel lost, the full repo can be found here:

$ git clone https://github.com/hackersandslackers/paramiko-tutorial.git$cd paramiko-tutorial$ make install$ make run

Paramiko leans heavily on the "in-the-weeds" side of Python libraries. If you're looking for something easy that can just get the job done,pyinfra is supposedly a great (and easy) alternative.

Setting up SSH Keys

To authenticate an SSH connection, we need to set up aprivate RSA SSH key. We can generate a key using the following command:

$ssh-keygen-t rsa

This will prompt us to provide a name for our key. Name it whatever you like:

>> Generating a public/private rsa key pair.>> Enter the fileinwhich you wish to save they key(i.e., /home/username/.ssh/id_rsa):

Next, you'll be prompted to provide a password (recommended, but up to you). This will result in the creation of two keys: aprivate key (which I'll refer to assshkey) and apublic key which takes the naming scheme ofsshkey.pub:

>> Enter passphrase(emptyforno passphrase):>> Enter same passphrase again:>> Your identification has been savedin /Users/username/.ssh/sshkey>> Your public key has been savedin /Users/username/.ssh/sshkey.pub>> The key fingerprint is:SHA256:UFYfdydftf6764576787648574 user@My-MacBook-Pro.local>> The key's randomart image is:+---[RSA 3072]----+|o*-HHFTEf.       ||++o=++.+E.       ||OoO + ..         ||.= + o           ||+ . o . S        ||.. o o .         ||o .+ O .         ||.+               ||..o.             |+----[SHA256]-----+

Consider theprivate key to be sacred; this key and everything about it should remain on your machine (the example I posted above is fake). The correspondingpublic key is what we put on remote hosts in our possession to authenticate a connection.

The easiest way to do this is by usingssh-copy-id, which is a command that exists for this exact purpose:

$ssh-copy-id-i ~/.ssh/mykey user@example.com

In the above example,~/.ssh/mykey is the file path of thepublic key we created.user@example.com should of course be replaced with the address of your remote host, whereuser is the username of the preferred user to connect with.

Verifying our SSH Key

It's always best to check our work. Go ahead and SSH into your remote host; if you didn't set a password when creating your key, you may be surprised to find that you're no longer prompted for a password. That's a good sign.

Check your host's.ssh directory for the public key we created:

$ssh user@example.com$cd ~/.ssh$ls

We're looking for keys that begin with the following header:

-----BEGIN RSA PRIVATE KEY-----...-----END RSA PRIVATE KEY-----

Feel free to do the same on your VPS.

Starting our Script

Let's install our libraries. Fire up whichever virtual environment you prefer and let em rip:

$pip3installparamiko scp

Just one more thing before we write some meaningful Python code! Create a config file to hold the variables we'll need to connect to our host. Here are the barebones of what we need to get into our server:

• Host: The IP address or URL of the remote host we're trying to access.
• Username: This is the username you use to SSH into your server.
• Passphrase (optional): If you specified a passphrase when you created your ssh key, specify that here. Remember that your SSH key passphrase isnot the same as your user's password.
• SSH Key: The file path of the key we created earlier. On OSX, these live in your system's~/.ssh folder. SSH key we're targeting must have an accompanying key with a.pub file extension. This is our public key; if you were following along earlier, this should have already been generated for you.

If you're trying to upload or download files from your remote host, you'll need to include two more variables:

• Remote Path: The path to the remote directory we're looking to target for file transfers. We can either upload things to this folder or download the contents of it.
• Local Path: Same idea as above, but the reverse. For our convenience, the local path we'll be using is simply/data, and contains pictures of cute fox gifs.

Now we have everything we need to make a respectableconfig.py file:

"""Remote host configuration."""fromosimportgetenv,pathfromdotenvimportload_dotenv# Load environment variables from .envBASE_DIR=path.abspath(path.dirname(__file__))load_dotenv(path.join(BASE_DIR,".env"))# SSH Connection VariablesENVIRONMENT=getenv("ENVIRONMENT")SSH_REMOTE_HOST=getenv("SSH_REMOTE_HOST")SSH_USERNAME=getenv("SSH_USERNAME")SSH_PASSWORD=getenv("SSH_PASSWORD")SSH_KEY_FILEPATH=getenv("SSH_KEY_FILEPATH")SCP_DESTINATION_FOLDER=getenv("SCP_DESTINATION_FOLDER")# Local file directoryLOCAL_FILE_DIRECTORY=f"{BASE_DIR}/files"

Creating an SSH Client

We're going to create a class calledRemoteClient to handle the interactions we'll be having with our remote host. Before we get too fancy, let's just start things off by instantiating theRemoteClient class with the variables we created inconfig.py :

"""Client to handle connections and actions executed against a remote host."""classRemoteClient:"""Client to interact with a remote host via SSH & SCP."""def__init__(self,host:str,user:str,password:str,ssh_key_filepath:str,remote_path:str,):self.host=hostself.user=userself.password=passwordself.ssh_key_filepath=ssh_key_filepathself.remote_path=remote_pathself.client=Noneself._upload_ssh_key()

You'll notice I added a few things to our constructor, besides the config values we pass in:

• self.client:self.client will ultimately serve as the connection objection in our class, similar to how you have dealt with terminology likeconn in database libraries. Our connection will beNone until we explicitly connect to our remote host.
• self._upload_ssh_key() isn't a variable, but rather a function to be run automatically whenever our client is instantiated. Calling_upload_ssh_key() is telling ourRemoteClient object to check for local ssh keys immediately upon creation so that we can try to pass them to our remote host. Otherwise, we wouldn't be able to establish a connection at all.

Uploading SSH Keys to a Remote Host

We've reached the section of this exercise where we need to knock out some devastatingly inglorious boilerplate code. This is typically where emotionally inferior individuals succumb to the sheer dull obscurity of understanding SSH keys and maintaining connections. Make no mistake: authenticating and managing connections toanything programmatically is overwhelmingly dull... unless your tour guide happens to be an enchanting wordsmith, serving as your loving protector through perilous obscurity. Some people call this post atutorial. I intend to call itart.

RemoteClient will start with two private methods:_get_ssh_key() and_upload_ssh_key(). The former will fetch a locally stored public key, and if successful, the latter will deliver this public key to our remote host as an olive branch of access. Once a locally created public key exists on a remote machine, that machine will then forever trust us with our requests to connect to it: no passwords required. We'll be including proper logging along the way, just in case we run into any trouble:

"""Client to handle connections and actions executed against a remote host."""fromosimportsystemfromparamikoimportSSHClient,AutoAddPolicy,RSAKeyfromparamiko.auth_handlerimportAuthenticationException,SSHExceptionfromscpimportSCPClient,SCPExceptionfrom.logimportloggerclassRemoteClient:"""Client to interact with a remote host via SSH & SCP."""...def_get_ssh_key(self):"""Fetch locally stored SSH key."""try:self.ssh_key=RSAKey.from_private_key_file(self.ssh_key_filepath)LOGGER.info(f"Found SSH key at self{self.ssh_key_filepath}")returnself.ssh_keyexceptSSHExceptionase:LOGGER.error(f"SSHException while getting SSH key:{e}")exceptExceptionase:LOGGER.error(f"Unexpected error while getting SSH key:{e}")def_upload_ssh_key(self):try:system(f"ssh-copy-id -i{self.ssh_key_filepath}.pub{self.user}@{self.host}>/dev/null 2>&1")LOGGER.info(f"{self.ssh_key_filepath} uploaded to{self.host}")exceptFileNotFoundErrorase:LOGGER.error(f"FileNotFoundError while uploading SSH key:{e}")exceptExceptionase:LOGGER.error(f"Unexpected error while uploading SSH key:{e}")

_get_ssh_key() is quite simple: it verifies that an SSH key exists at the path we specified in our config to be used for connecting to our host. If the file does in fact exist, we happily set ourself.ssh_key variable, so this key can be uploaded and used by our client from here forward. Paramiko provides us with a submodule calledRSAKey to easily handle all things RSA key related, likeparsing a private key file into a usable connection authentication. That's what we get here:

RSAKey.from_private_key_file(self.ssh_key_filepath)

If our RSA key were incomprehensible nonsense instead of a real key, Paramiko'sSSHException would have caught this and raised an exception early on explaining just that. Properly utilizing a library's error handling takes a lot of the guesswork out of "what went wrong," especially in cases like where there's potential for numerous unknowns in a niche space neither of us mess with often.

_upload_ssh_key() is where we get to jam our SSH key down the throat of our remote server while shouting, "LOOK! YOU CAN TRUST ME FOREVER NOW!" To accomplish this, I go a bit "old school" by passing bash commands via Python'sos.system. Unless somebody makes me aware of a cleaner approach in the comments, I'll assume this is the most badass way to handle passing keys to a remote server.

The standard non-Python way of passing keys to a host looks like this:

ssh-copy-id-i ~/.ssh/mykey user@host

This is precisely what we accomplish in our function in Python, which looks like this:

system(f"ssh-copy-id -i{self.ssh_key_filepath}\{self.user}@{self.host}>/dev/null 2>&1")

I suppose you won't let me slip that/dev/null 2>&1 bit by you? Fine. If youmust know, here's some guy on StackOverflow explaining it better than I can:

> is for redirect/dev/null is a black hole where any data sent, will be discarded.2 is the file descriptor for Standard Error.> is for redirect.& is the symbol for file descriptor (without it, the following1 would be considered a filename).1 is the file descriptor for Standard O.

So we're basically telling our remote server we're giving it something, and it's all like "where do I put this thing," to which we reply "nowhere in physical in space, as this is not anobject, but rather aneternal symbol of our friendship. Our remote host is then flooded with gratitude and emotion, because yes, computersdo have emotions, but wecan't be bothered by that right now.

Connecting to our Client

We'll add a method to our client calledconnect() to handle connecting to our host:

...classRemoteClient:"""Client to interact with a remote host via SSH & SCP."""...@propertydefconnection(self):"""Open connection to remote host. """try:client=SSHClient()client.load_system_host_keys()client.set_missing_host_key_policy(AutoAddPolicy())client.connect(self.host,username=self.user,password=self.password,key_filename=self.ssh_key_filepath,timeout=5000,)returnclientexceptAuthenticationExceptionase:LOGGER.error(f"AuthenticationException occurred; did you remember to generate an SSH key?{e}")exceptExceptionase:LOGGER.error(f"Unexpected error occurred while connecting to host:{e}")

Let's break this down:

• client = SSHClient() sets the stage for creating an object representing our SSH client. The following lines will configure this object to make it more useful.
• load_system_host_keys() instructs our client to look for all the hosts we've connected to in the past by looking at our system'sknown_hosts file and finding the SSH keys our host is expecting. We've never connected to our host in the past, so we need to specify our SSH key explicitly.
• set_missing_host_key_policy() tells Paramiko what to do in the event of an unknown key pair. This is expecting a "policy" built-in to Paramiko, to which we're going to specificAutoAddPolicy(). Setting our policy to "auto-add" means that if we attempt to connect to an unrecognized host, Paramiko will automatically add the missing key locally.
• connect() isSSHClient's most important method (as you might imagine). We're finally able to pass ourhost ,user , andSSH key to achieve what we've all been waiting for: a glorious SSH connection into our server! Theconnect() method allows for a ton of flexibility via a vast array of optional keyword arguments too. I happen to pass a few here: settinglook_for_keys toTrue gives Paramiko permission to look around in our~/.ssh folder to discover SSH keys on its own, and settingtimeout will automatically close connections we'll probably forget to close. We could even pass variables for things likeport andpassword , if we had elected to connect to our host this way.

Disconnecting

We should close connections to our remote host whenever we're done using them. Failing to do so might not necessarily bedisastrous, but I've had a few instances where enough hanging connections would eventually max out inbound traffic on port 22. Regardless of whether your use case might consider a reboot to be a disaster or mild inconvenience, let's just close our damn connections like adults as though we were wiping our butts after pooping. No matter your connection hygiene, I advocate for setting atimeout variable (as we saw earlier). Anyway. voila:

classRemoteClient:...defdisconnect(self):"""Close ssh connection."""ifself.client:self.client.close()ifself.scp:self.scp.close()

Fun fact: settingself.client.close() actually setsself.client to equalNone, which is useful in cases where you might want to check if a connection is already open.

Executing Unix Commands

We now have a wonderful Python class that can find RSA keys, connect, and disconnect. Itdoes lack the ability to do, well, anything useful.

We can fix this and finally begin doing "stuff" with a brand new method to execute commands, which I'll aptly dubexecute_commands() (that's correct, "commands" as in potentially-more-than-one, we'll touch on that in a moment). The legwork of all this is done by the Paramiko client's built-inexec_command() method, which accepts a single string as a command and executes it:

...classRemoteClient:...defexecute_commands(self,commands:List[str]):"""        Execute multiple commands in succession.        :param List[str] commands: List of unix commands as strings.        """forcmdincommands:stdin,stdout,stderr=self.connection.exec_command(cmd)stdout.channel.recv_exit_status()response=stdout.readlines()forlineinresponse:LOGGER.info(f"INPUT:{cmd}\n\                    OUTPUT:{line}")

The function we just createdexecute_commands() expects alist of strings to execute as commands. That's partially for convenience, but it's also because Paramiko won't run any "state" changes (like changing directories) between commands, so each command we pass to Paramiko should assume we're working out of our server's root. I took the liberty of passing three such commands like so:

...defexecute_command_on_remote(remote):"""Execute UNIX command on the remote host."""remote.execute_commands(["cd /var/www/ && ls","tail /var/log/nginx/access.log","ps aux | grep node","cd /uploads/ && ls",])

I can view the contents of a directory by chainingcd path/to/dir && ls, but runningcd path/to/dir followed byls would result in nothingness becausels the second time returns the list of files in our server's root.

You'll noticeclient.exec_command(cmd) returns three values as opposed to one: this can be useful to see whichinput produced whichoutput. For example, here are the full logs for the example I provided where I passed three commands toremote.execute_commands():

2020-01-02 23:20:16.103 | paramiko_tutorial.client:execute_cmd:95 - INPUT:cd /var/www/&&ls | OUTPUT: django2020-01-02 23:20:16.103 | paramiko_tutorial.client:execute_cmd:95 - INPUT:cd /var/www/&&ls | OUTPUT: ghost2020-01-02 23:20:16.103 | paramiko_tutorial.client:execute_cmd:95 - INPUT:cd /var/www/&&ls | OUTPUT: hackers-hbs2020-01-02 23:20:16.103 | paramiko_tutorial.client:execute_cmd:95 - INPUT:cd /var/www/&&ls | OUTPUT: html2020-01-02 23:20:16.104 | paramiko_tutorial.client:execute_cmd:95 - INPUT:cd /var/www/&&ls | OUTPUT: hustlers2020-01-02 23:20:16.104 | paramiko_tutorial.client:execute_cmd:95 - INPUT:cd /var/www/&&ls | OUTPUT: pizza2020-01-02 23:20:16.104 | paramiko_tutorial.client:execute_cmd:95 - INPUT:cd /var/www/&&ls | OUTPUT: toddbirchard2020-01-02 23:20:16.196 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 134.209.37.49 - -[03/Jan/2020:03:42:34 +0000]"GET / HTTP/2.0" 404 139"-""Mozilla/5.0 (compatible; NetcraftSurveyAgent/1.0; +info@netcraft.com)"2020-01-02 23:20:16.196 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 54.36.148.187 - -[03/Jan/2020:03:43:49 +0000]"GET /robots.txt HTTP/1.1" 404 149"-""Mozilla/5.0 (compatible; AhrefsBot/6.1; +https://ahrefs.com/robot/)"2020-01-02 23:20:16.196 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 54.36.150.104 - -[03/Jan/2020:03:43:50 +0000]"GET / HTTP/1.1" 404 139"-""Mozilla/5.0 (compatible; AhrefsBot/6.1; +https://ahrefs.com/robot/)"2020-01-02 23:20:16.196 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 46.229.168.146 - -[03/Jan/2020:03:48:49 +0000]"GET /robots.txt HTTP/1.1" 200 92"-""Mozilla/5.0 (compatible; SemrushBot/6~bl; +https://www.semrush.com/bot.html)"2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 46.229.168.153 - -[03/Jan/2020:03:48:50 +0000]"GET /the-art-of-technical-documentation/ HTTP/1.1" 200 9472"-""Mozilla/5.0 (compatible; SemrushBot/6~bl; +https://www.semrush.com/bot.html)"2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 157.55.39.171 - -[03/Jan/2020:03:50:20 +0000]"GET /robots.txt HTTP/1.1" 200 94"-""Mozilla/5.0 (compatible; bingbot/2.0; +https://www.bing.com/bingbot.htm)"2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 40.77.167.220 - -[03/Jan/2020:03:50:24 +0000]"GET / HTTP/1.1" 200 3791"-""Mozilla/5.0 (compatible; bingbot/2.0; +https://www.bing.com/bingbot.htm)"2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 54.36.150.64 - -[03/Jan/2020:03:56:47 +0000]"GET /the-ruin-of-feeling-powerless/ HTTP/2.0" 200 9605"-""Mozilla/5.0 (compatible; AhrefsBot/6.1; +https://ahrefs.com/robot/)"2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 54.36.150.187 - -[03/Jan/2020:04:05:01 +0000]"GET /bigquery-and-sql-databases/ HTTP/2.0" 404 146"-""Mozilla/5.0 (compatible; AhrefsBot/6.1; +https://ahrefs.com/robot/)"2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT:tail /var/log/nginx/access.log | OUTPUT: 46.229.168.149 - -[03/Jan/2020:04:13:41 +0000]"GET /robots.txt HTTP/1.1" 502 182"-""Mozilla/5.0 (compatible; SemrushBot/6~bl; +https://www.semrush.com/bot.html)"2020-01-02 23:20:16.322 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux |grepnode | OUTPUT: ghost 1354 39.2 3.4 1223844 140172 ? Sl 04:20 0:05 /usr/bin/node current/index.js2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux |grepnode | OUTPUT: ghost 1375 36.8 3.3 1217696 135548 ? Sl 04:20 0:04 /usr/bin/node current/index.js2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux |grepnode | OUTPUT: ghost 1395 46.8 3.6 1229824 147384 ? Sl 04:20 0:06 /usr/bin/node current/index.js2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux |grepnode | OUTPUT: ghost 1410 37.7 3.2 1216320 132912 ? Sl 04:20 0:04 /usr/bin/node current/index.js2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux |grepnode | OUTPUT: root 1848 0.0 0.0 13312 3164 ? Ss 04:20 0:00 bash-c ps aux |grepnode2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux |grepnode | OUTPUT: root 1850 0.0 0.0 14856 1104 ? S 04:20 0:00grepnode

Some beautiful stuff here. Now you can see which sites are on my server, which bots are spamming me, and how many node processes I'm running.

I don't want to waste much more time on the art of executing commands, but it's worth mentioning the presence of why we callstdout.channel.recv_exit_status() after each command. Waiting forrecv_exit_status() to come back after runningclient.exec_command() forces our commands to be runsynchronously, otherwise there's a likely chance our remote machine won't be about to decipher commands as fast as we pass them along.

Uploading (and Downloading) Files via SCP

SCP refers to both the protocol for copying files to remote machines (secure copy protocol) as well as the Python library, which utilizes this. We've already installed the SCP library, so import that shit.

TheSCP andParamiko libraries complement one another to make uploading via SCP super easy.SCPClient() creates an object which expects "transport" from Paramiko, which we provide withself.conn.get_transport(). Creating an SCP connection piggybacks off of our SSH client in terms of syntax, but these connections areseparate. It's possible to close an SSH connection and leave an SCP connection open, so don't do that. Open an SCP connection like this:

self.scp= SCPClient(self.client.get_transport())

Uploading a single file is boring, so let's upload an entire directory of files instead.bulk_upload() accepts a list of file paths:

classRemoteClient:...defbulk_upload(self,filepaths:List[str]):"""        Upload multiple files to a remote directory.        :param List[str] filepaths: List of local files to be uploaded.        """try:self.scp.put(filepaths,remote_path=self.remote_path,recursive=True)LOGGER.info(f"Finished uploading{len(filepaths)} files to{self.remote_path} on{self.host}")exceptSCPExceptionase:LOGGER.error(f"SCPException during bulk upload:{e}")exceptExceptionase:LOGGER.error(f"Unexpected exception during bulk upload:{e}")

Our method is expecting to receive a list of strings, each representing the local path to a file, we'd like to upload.

SCP'sput() method will upload any number of files to a remote host. This will replace existing files with the same name if they happen to exist at the destination we specify. That's all it takes!

Downloading Files

The counterpart to SCP'sput() is theget() method:

classRemoteClient:...defdownload_file(self,file:str):"""Download file from remote host."""self.scp.get(file)

Our Big Beautiful Script

We now have a sick Python class to handle SSH and SCP with a remote host... let's put it to work! The following snippet is a quick way to test what we've built so far. In short, this script looks for a local folder filled with files (in my case, I filled the folder with fox gifs 🦊).

Check out how easy it is to create amain.py that handles complex tasks on remote machines thanks to ourRemoteClient class:

"""Perform tasks against a remote host."""fromconfigimport(host,local_file_directory,password,remote_path,ssh_key_filepath,username,)from.clientimportRemoteClientfrom.filesimportfetch_local_filesdefrun():"""Initialize remote host client and execute actions."""client=RemoteClient(SSH_REMOTE_HOST,SSH_USERNAME,SSH_PASSWORD,SSH_KEY_FILEPATH,SCP_DESTINATION_FOLDER,)upload_files_to_remote(client)execute_command_on_remote(client,["mkdir /uploads","cd /uploads/ && ls",],)defupload_files_to_remote(client:RemoteClient):"""    Upload files to remote via SCP.    :param RemoteClient client: Remote server client.    """local_files=fetch_local_files(LOCAL_FILE_DIRECTORY)client.bulk_upload(local_files)defexecute_command_on_remote(client:RemoteClient,commands:List[str]):"""    Execute a UNIX command remotely on a given host.    :param RemoteClient client: Remote server client.    :param List[str] commands: List of commands to run on remote host.    """client.execute_commands(commands)

Here's the output of the our upload function:

2020-01-03 00:00:27.215 | paramiko_tutorial.client:__upload_single_file:85 - Uploaded data/fox1.gif to /uploads/2020-01-03 00:00:27.985 | paramiko_tutorial.client:__upload_single_file:85 - Uploaded data/fox2.gif to /uploads/2020-01-03 00:00:30.015 | paramiko_tutorial.client:__upload_single_file:85 - Uploaded data/fox3.gif to /uploads/2020-01-03 00:00:30.015 | paramiko_tutorial.client:bulk_upload:73 - Finished uploading 3 files to /uploads/ on 149.433.117.1425

It worked! Don't believe me? Why don't we check for ourselves by runningremote.execute_commands(['cd /var/www/ && ls'])?

2020-01-03 00:08:55.955 | paramiko_tutorial.client:execute_commands:96 - INPUT:cd /uploads/&&ls | OUTPUT: fox1.gif2020-01-03 00:08:55.955 | paramiko_tutorial.client:execute_commands:96 - INPUT:cd /uploads/&&ls | OUTPUT: fox2.gif2020-01-03 00:08:55.956 | paramiko_tutorial.client:execute_commands:96 - INPUT:cd /uploads/&&ls | OUTPUT: fox3.gif

There you have it. Straight from the fox's mouth.

Take It And Run With It

This is where I'd like to take a moment to thank all of you, and apologize that you're still here. I swore an oath to myself to stop posting tutorials over two thousand words long, and this one is looking to push five thousand words of nonsense. I'll work on that. New year, new me.

For your convenience, I've uploaded the source for this tutorial toGithub. Feel free to take this and run with it! To close things out, I'll leave you with the meat and potatoes of theRemoteClient class we put together:

"""Client to handle connections and actions executed against a remote host."""fromosimportsystemfromtypingimportListfromparamikoimportAutoAddPolicy,RSAKey,SSHClientfromparamiko.auth_handlerimportAuthenticationException,SSHExceptionfromscpimportSCPClient,SCPExceptionfromlogimportLOGGERclassRemoteClient:"""Client to interact with a remote host via SSH & SCP."""def__init__(self,host:str,user:str,password:str,ssh_key_filepath:str,remote_path:str,):self.host=hostself.user=userself.password=passwordself.ssh_key_filepath=ssh_key_filepathself.remote_path=remote_pathself.client=Noneself._upload_ssh_key()@propertydefconnection(self):"""Open SSH connection to remote host."""try:client=SSHClient()client.load_system_host_keys()client.set_missing_host_key_policy(AutoAddPolicy())client.connect(self.host,username=self.user,password=self.password,key_filename=self.ssh_key_filepath,timeout=5000,)returnclientexceptAuthenticationExceptionase:LOGGER.error(f"AuthenticationException occurred; did you remember to generate an SSH key?{e}")exceptExceptionase:LOGGER.error(f"Unexpected error occurred while connecting to host:{e}")@propertydefscp(self)->SCPClient:conn=self.connectionreturnSCPClient(conn.get_transport())def_get_ssh_key(self):"""Fetch locally stored SSH key."""try:self.ssh_key=RSAKey.from_private_key_file(self.ssh_key_filepath)LOGGER.info(f"Found SSH key at self{self.ssh_key_filepath}")returnself.ssh_keyexceptSSHExceptionase:LOGGER.error(f"SSHException while getting SSH key:{e}")exceptExceptionase:LOGGER.error(f"Unexpected error while getting SSH key:{e}")def_upload_ssh_key(self):try:system(f"ssh-copy-id -i{self.ssh_key_filepath}.pub{self.user}@{self.host}>/dev/null 2>&1")LOGGER.info(f"{self.ssh_key_filepath} uploaded to{self.host}")exceptFileNotFoundErrorase:LOGGER.error(f"FileNotFoundError while uploading SSH key:{e}")exceptExceptionase:LOGGER.error(f"Unexpected error while uploading SSH key:{e}")defdisconnect(self):"""Close SSH & SCP connection."""ifself.connection:self.client.close()ifself.scp:self.scp.close()defbulk_upload(self,filepaths:List[str]):"""        Upload multiple files to a remote directory.        :param List[str] filepaths: List of local files to be uploaded.        """try:self.scp.put(filepaths,remote_path=self.remote_path,recursive=True)LOGGER.info(f"Finished uploading{len(filepaths)} files to{self.remote_path} on{self.host}")exceptSCPExceptionase:LOGGER.error(f"SCPException during bulk upload:{e}")exceptExceptionase:LOGGER.error(f"Unexpected exception during bulk upload:{e}")defdownload_file(self,filepath:str):"""        Download file from remote host.        :param str filepath: Path to file hosted on remote server to fetch.        """self.scp.get(filepath)defexecute_commands(self,commands:List[str]):"""        Execute multiple commands in succession.        :param List[str] commands: List of unix commands as strings.        """forcmdincommands:stdin,stdout,stderr=self.connection.exec_command(cmd)stdout.channel.recv_exit_status()response=stdout.readlines()forlineinresponse:LOGGER.info(f"INPUT:{cmd}\n\                    OUTPUT:{line}")

The full source code for this tutorial can be found here:

$ git clone https://github.com/hackersandslackers/paramiko-tutorial.git$cd paramiko-tutorial$ make install$ make run