297

I was givenAWS Console access to an account with 2 instances running that I cannot shut down (in production). I would, however, like to gain SSH access to these instances, is it possible to create a new Keypair and apply it to the instances so I can SSH in? Obtaining the existingpem file for the keypair the instances were created under is currently not an option.

If this isn't possible is there some other way I can get into the instances?

Andrzej Sydor's user avatar
Andrzej Sydor
1,4166 gold badges14 silver badges32 bronze badges
askedJul 15, 2010 at 23:02
Chris Wagner's user avatar
6

11 Answers11

201

You can't apply a keypair to a running instance. You can only use the new keypair to launch a new instance.

For recovery, if it's an EBS boot AMI, you can stop it, make a snapshot of the volume. Create a new volume based on it. And be able to use it back to start the old instance, create a new image, or recover data.

Though data at ephemeral storage will be lost.


Due to the popularity of this question and answer, I wanted to capture the information in the link that Rodney posted on his comment.

Credit goes toEric Hammond forthis information.

Fixing Files on the Root EBS Volume of an EC2 Instance

You can examine and edit files on the root EBS volume on an EC2 instance even if you are in what you considered a disastrous situation like:

  • You lost your ssh key or forgot your password
  • You made a mistake editing the /etc/sudoers file and can no longergain root access with sudo to fix it
  • Your long running instance is hung for some reason, cannot becontacted, and fails to boot properly
  • You need to recover files off of the instance but cannot get to it

On a physical computer sitting at your desk, you could simply boot the system with a CD or USB stick, mount the hard drive, check out and fix the files, then reboot the computer to be back in business.

A remote EC2 instance, however, seems distant and inaccessible when you are in one of these situations. Fortunately, AWS provides us with the power and flexibility to be able to recover a system like this, provided that we are running EBS boot instances and not instance-store.

The approach on EC2 is somewhat similar to the physical solution, but we’re going to move and mount the faulty “hard drive” (root EBS volume) to a different instance, fix it, then move it back.

In some situations, it might simply be easier to start a new EC2 instance and throw away the bad one, but if you really want to fix your files, here is the approach that has worked for many:

Setup

Identify the original instance (A) and volume that contains the broken root EBS volume with the files you want to view and edit.

instance_a=i-XXXXXXXXvolume=$(ec2-describe-instances $instance_a |  egrep '^BLOCKDEVICE./dev/sda1' | cut -f3)

Identify the second EC2 instance (B) that you will use to fix the files on the original EBS volume. This instance must be running in the same availability zone as instance A so that it can have the EBS volume attached to it. If you don’t have an instance already running, start a temporary one.

instance_b=i-YYYYYYYY

Stop the broken instance A (waiting for it to come to a complete stop), detach the root EBS volume from the instance (waiting for it to be detached), then attach the volume to instance B on an unused device.

ec2-stop-instances $instance_aec2-detach-volume $volumeec2-attach-volume --instance $instance_b --device /dev/sdj $volume

ssh to instance B and mount the volume so that you can access its file system.

ssh ...instance b...sudo mkdir -p 000 /vol-asudo mount /dev/sdj /vol-a

Fix It

At this point your entire root file system from instance A is available for viewing and editing under /vol-a on instance B. For example, you may want to:

  • Put the correct ssh keys in /vol-a/home/ubuntu/.ssh/authorized_keys
  • Edit and fix /vol-a/etc/sudoers
  • Look for error messages in /vol-a/var/log/syslog
  • Copy important files out of /vol-a/…

Note: The uids on the two instances may not be identical, so take care if you are creating, editing, or copying files that belong to non-root users. For example, your mysql user on instance A may have the same UID as your postfix user on instance B which could cause problems if you chown files with one name and then move the volume back to A.

Wrap Up

After you are done and you are happy with the files under /vol-a, unmount the file system (still on instance-B):

sudo umount /vol-asudo rmdir /vol-a

Now, back on your system with ec2-api-tools, continue moving the EBS volume back to it’s home on the original instance A and start the instance again:

ec2-detach-volume $volumeec2-attach-volume --instance $instance_a --device /dev/sda1 $volumeec2-start-instances $instance_a

Hopefully, you fixed the problem, instance A comes up just fine, and you can accomplish what you originally set out to do. If not, you may need to continue repeating these steps until you have it working.

Note: If you had an Elastic IP address assigned to instance A when you stopped it, you’ll need to reassociate it after starting it up again.

Remember! If your instance B was temporarily started just for this process, don’t forget to terminate it now.

Chris Wagner's user avatar
Chris Wagner
21k8 gold badges77 silver badges96 bronze badges
answeredJul 16, 2010 at 22:29
Rodney Quillo's user avatar
Sign up to request clarification or add additional context in comments.

2 Comments

can you tell us a step-by-step guide for this (or point to it). In my case, I have an existing running instance and need to log in to it from a remote location, where I dont have the private key.
This saved my day today. I just had to make one change.sudo mount /dev/sdj /vol-a did not work for me on instance B. I followed this (serverfault.com/a/632906) and didsudo mount /dev/xvdj1 /vol-a and it worked. Also, please make sure you are creating instance B in the same availability zone as the one your EBS is present in. My EBS was in ap-south-1a but my instance B was created in ap-south-1b. I had to create a new instance and explicitly mention the availability zone in the subnet section of network setting
99

Though you can't add a key pair to a running EC2 instance directly, you can create a linux user and create a new key pair for him, then use it like you would with the original user's key pair.

In your case, you can ask the instance owner (who created it) to do the following. Thus, the instance owner doesn't have to share his own keys with you, but you would still be able to ssh into these instances. These steps were originally posted by Utkarsh Sengar (aka.@zengr) athttp://utkarshsengar.com/2011/01/manage-multiple-accounts-on-1-amazon-ec2-instance/. I've made only a few small changes.

  1. Step 1: login by default “ubuntu” user:

    $ ssh -i my_orig_key.pem[email protected]
  2. Step 2: create a new user, we will call our new user “john”:

    [ubuntu@ip-11-111-111-111 ~]$ sudo adduser john

    Set password for “john” by:

    [ubuntu@ip-11-111-111-111 ~]$ sudo su -[root@ip-11-111-111-111 ubuntu]# passwd john

    Add “john” to sudoer’s list by:

    [root@ip-11-111-111-111 ubuntu]# visudo

    .. and add the following to the end of the file:

    john   ALL = (ALL)    ALL

    Alright! We have our new user created, now you need to generate the key file which will be needed to login, like we have my_orin_key.pem in Step 1.

    Now, exit and go back to ubuntu, out of root.

    [root@ip-11-111-111-111 ubuntu]# exit[ubuntu@ip-11-111-111-111 ~]$
  3. Step 3: creating the public and private keys:

    [ubuntu@ip-11-111-111-111 ~]$ su john

    Enter the password you created for “john” in Step 2. Then create a key pair. Remember that the passphrase for key pair should be at least 4 characters.

    [john@ip-11-111-111-111 ubuntu]$ cd /home/john/[john@ip-11-111-111-111 ~]$ ssh-keygen -b 1024 -f john -t dsa[john@ip-11-111-111-111 ~]$ mkdir .ssh[john@ip-11-111-111-111 ~]$ chmod 700 .ssh[john@ip-11-111-111-111 ~]$ cat john.pub > .ssh/authorized_keys[john@ip-11-111-111-111 ~]$ chmod 600 .ssh/authorized_keys[john@ip-11-111-111-111 ~]$ sudo chown john:ubuntu .ssh

    In the above step, john is the user we created and ubuntu is the default user group.

    [john@ip-11-111-111-111 ~]$ sudo chown john:ubuntu .ssh/authorized_keys
  4. Step 4: now you just need to download the key called “john”. I use scp to download/upload files from EC2, here is how you can do it.

    You will still need to copy the file usingubuntu user, since you only have the key for that user name. So, you will need to move the key to ubuntu folder and chmod it to 777.

    [john@ip-11-111-111-111 ~]$ sudo cp john /home/ubuntu/[john@ip-11-111-111-111 ~]$ sudo chmod 777 /home/ubuntu/john

    Now come to local machine’s terminal, where you have my_orig_key.pem file and do this:

    $ cd ~/.ssh$ scp -i my_orig_key.pem[email protected]:/home/ubuntu/john john

    The above command will copy the key “john” to the present working directory on your local machine. Once you have copied the key to your local machine, you should delete “/home/ubuntu/john”, since it’s a private key.

    Now, one your local machine chmod john to 600.

    $ chmod 600 john
  5. Step 5: time to test your key:

    $ ssh -i john[email protected]

So, in this manner, you can setup multiple users to use one EC2 instance!!

Skatox's user avatar
Skatox
4,27412 gold badges47 silver badges50 bronze badges
answeredApr 8, 2013 at 8:59
esengineer's user avatar

3 Comments

This is useful, but as a final step should you not also delete the private key off the remote machine? That way other people with access to the instance cannot also copy it and use your key to log in.
This works for me. But how do I navigate to ubuntu user from here since the files I will be working on are in the ubuntu user directory. This will take me to john user group. Ubuntu 14.04.4 LTS
This didn't work for me. It gave invalid permissions. I had to create key pair from ec2 console then it started working
32

For Elastic Beanstalk environments, you can apply a key-value pair to a running instance like this:

  • Create a key-value pair from EC2 -> Key Pairs (Under NETWORK & SECURITY tab)
  • Go to Elastic Beanstalk and click on your application
  • Go toConfiguration ->Security and clickEdit
  • Choose your EC2 key pair and click Apply
  • Click confirm to confirm the update. It will terminate the environment and apply the key value to your environment.
ssuperczynski's user avatar
ssuperczynski
3,4463 gold badges46 silver badges64 bronze badges
answeredMar 20, 2020 at 13:43
Kerem's user avatar

Comments

14

On your local machine, run command:

ssh-keygen -t rsa -C "SomeAlias"

After that command runs, a file ending in *.pub will be generated. Copy the contents of that file.

On the Amazon machine, edit ~/.ssh/authorized_keys and paste the contents of the *.pub file (and remove any existing contents first).

You can then SSH using the other file that was generated from the ssh-keygen command (the private key).

answeredMay 10, 2014 at 3:03
Dan's user avatar

1 Comment

So like @Dan mentioned it's possible to change the access to your instance editing this file, but you never'll be able to change the keypair associated with the instance at a meta data level. Dont forget to add the .pem file name in the end of your publicKey, ex:ssh-rsa AAAAB3NzaC1yc2EA...DsGt66 my-key-pair
9

I didn't find an easy way to add a new key pair via the console, but you can do it manually.

Just ssh into your EC2 box with the existing key pair. Then edit the ~/.ssh/authorized_keys and add the new key on a new line. Exit and ssh via the new machine. Success!

answeredApr 28, 2014 at 14:05
ninja123's user avatar

Comments

7

This happened to me earlier (didn't have access to an EC2 instance someone else created but had access to AWS web console) and I blogged the answer:http://readystate4.com/2013/04/09/aws-gaining-ssh-access-to-an-ec2-instance-you-lost-access-to/

Basically, you can detached the EBS drive, attach it to an EC2 that you do have access to. Add your SSH pub key to~ec2-user/.ssh/authorized_keys on this attached drive. Then put it back on the old EC2 instance. step-by-step in the link using Amazon AMI.

No need to make snapshots or create a new cloned instance.

answeredApr 9, 2013 at 19:22
Mauvis Ledford's user avatar

Comments

6

In my case I used this documentation to associate a key pair with my instance of Elastic Beanstalk

Important

You must create an Amazon EC2 key pair and configure your Elastic Beanstalk–provisioned Amazon EC2 instances to use the Amazon EC2 key pair before you can access your Elastic Beanstalk–provisioned Amazon EC2 instances. You can set up your Amazon EC2 key pairs using the AWS Management Console. For instructions on creating a key pair for Amazon EC2, see the Amazon Elastic Compute Cloud Getting Started Guide.

Configuring Amazon EC2 Server Instances with Elastic Beanstalk

answeredDec 21, 2015 at 20:35
Kamal Essajidi's user avatar

1 Comment

Thanks, @kamal-essajidi! For others using EB: once you have a key pair, you can add it to your Elastic Beanstalk in Configuration > Instances > EC2 key pair.
4

You can just add a new key to the instance by the following command:

ssh-copy-id -i ~/.ssh/id_rsa.pub domain_alias

You can configure domain_alias in ~/.ssh config

host domain_alias  User ubuntu  Hostname domain.com  IdentityFile ~/.ssh/ec2.pem
answeredMar 9, 2013 at 17:16
karser's user avatar

Comments

3

Once an instance has been started, there is no way to change the keypair associated with the instance at a meta data level, butyou can change what ssh key you use to connect to the instance.

stackoverflow.com/questions/7881469/change-key-pair-for-ec2-instance

answeredFeb 11, 2013 at 15:03
YetAnotherMatt's user avatar

Comments

3

You can actually add a key pair through the elastic beanstalk config page. it then restarts your instance for you and everything works.

answeredNov 8, 2019 at 5:04
Vishwas Vaishnav's user avatar

Comments

0

It is now possible to temporarily gain access to a running instance using any SSH key you control, without even adding it as a keypair to EC2, usingEC2 Instance Connect. It's only supported on some Linux distributions and AMIs but if you're using a relatively recent version of Amazon Linux 2 or Ubuntu it should be installed and running by default.

There's two main ways to use EC2 Instance Connect:

  • From the AWS Console, you can select the instance detail in EC2 and click the "Connect" button in the upper-right corner, then use the "EC2 Instance Connect" tab (currently the default).
  • From the AWS CLI, you first distribute your key to the instance withaws ec2-instance-connect send-ssh-public-key then connect with SSH as usual.

Note that this is only atemporary allowance, your SSH key will cease to be allowed after a short period of time, though once in you won't be disconnected. This also doesn't modify theauthorized_keys file so don't expect to see a new line there. If you want to allow your SSH key permanently, once you get in through EC2 Instance Connect, manually edit theauthorized_keys file and add the appropriate line for your SSH key.

You need to be able to reach your instance over the network to use this and it needs to allow SSH connections (set up security groups, etc.). You can run the AWS CLI command from a different place than wherever you actually SSH from, so for example you can use another EC2 instance in the same VPC to reach your currently unreachable EC2 instance.

For the Console version only, the instance specifically needs to have:

  • a public IPv4 address, and
  • a security group rule that allows TCP port 22 ingress from the EC2 Instance Connect service IP address range for its region.

The range of service IP addresses can be obtained fromthe full list of AWS IP address ranges. Some curl and jq can fetch the right range quickly (replaceus-west-1 with the desired region):

curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | \jq '.prefixes[] | select(.region == "us-west-1" and .service=="EC2_INSTANCE_CONNECT")'

the output will look something like this:

{  "ip_prefix": "13.52.6.112/29",  "region": "us-west-1",  "service": "EC2_INSTANCE_CONNECT",  "network_border_group": "us-west-1"}

The value ofip_prefix is the CIDR block that you need to allow.

answeredApr 14, 2023 at 19:37
kbolino's user avatar

Comments

Protected question. To answer this question, you need to have at least 10 reputation on this site (not counting theassociation bonus). The reputation requirement helps protect this question from spam and non-answer activity.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.