Tag: ssh

SSH Start to Finish Architecture – Forced Commands

Last week, we covered SSH Key options that can restrict how a private key is used to connect to a server. One of those options was the “command=” option, which allows restricting the key to calling only one command, regardless of the command issued as part of the ssh connection attempt. There are actually several ways to enforce this.

You can do the public key “command=” option we already covered. You can also use the sshd_config settings to apply a ForceCommand option. This is most useful for applying the same kind of forced command scenario we described last week via a “Match User” directive. It’s also useful for applying an sftp-only situation to a given user, so that the only thing the user can do is transfer files. The option would look like this, if that is your goal:

ForceCommand internal-sftp

The user’s shell needs to be a valid one for this to work, since the forced command is invoked via “ -c.” This means a shell of /bin/false or /bin/nologin is a no go. Since you are forcing a command, this should be less of a problem, though.

Finally, there is also a way to force this command with ssh-keygen options when signing a key via the certificate authority system for OpenSSH, but we will go into more detail on that when we get to the CA stuff.

The force command options don’t allow running the user’s ~/.ssh/rc, so that would not be a work around that the user could use to hack this system. ControlMaster overrides the public key force command if the option is set after a master session has already been established, so you may need to terminate all ssh connections for that user after making changes that enforce restrictions, moving forward.

A lot of people grumble about the force command options, because they believe that a single key is needed for each command that gets passed, but there is actually a means for handling that. There is an environment variable that gets set when an ssh session that wants to call a remote command is used to connect while force command is in use for a user. That environment variable is SSH_ORIGINAL_COMMAND, and it retains the command that was requested. This means you could have a wrapper script that is your forced command, have it check this environment variable for sanity (are all of the commands in the list provided in our whitelist, or not? If not, log a rejection and terminate. If so, log the call and execute.) The variable is unset if the user just tries to ssh in without calling a remote command, so be sure to check for that if you go this route. Assume if the variable is unset/empty that they tried to log in for an interactive session, and handle that however you feel is best. I would assume “log a rejection and terminate” is better, though, since an interactive session can’t be restricted properly without a restricted shell, that may still be jail broken if misconfigured. Your own needs may vary, though. Just be very thorough in your design, and be sure to sanitize all input before executing, and you should be fine.

Thanks for reading!

SSH Start to Finish Architecture – Public Key Options

Since we’ve introduced a few ways of doing tunneling, I thought now would be a good time to bring out some ways of restricting what a user can do in the event that they have access to a private key they shouldn’t.

We already talked about setting a pass phrase on the private key side, but there is still the chance that the user was able to brute force that, so let’s assume they have usable access to that private key. What options do we have for keeping things secure?

For starters, we can restrict which source IP/DNS names a user can come from using this key. If we modify the public key entry in the authorized_keys file to include a “from” directive before the key starts, we can provide a list of IP addresses/hostnames we expect to be coming from. This prevents an attack from the user’s workstation, for example. One caveat of this is, you can’t be coming through a source NAT, since that would reduce the effectiveness of this restriction. Everyone coming into the box from a source NAT VLAN or subnet would look like they are coming from the same place.

In order to set this directive, we will need to modify the public key by hand. It’s better to modify the public key before adding it to the authorized_keys file, but you can modify the authorized_keys file entry for this key, if you prefer.

To use this directive, we need to put the a “from” option with all of its values, comma separated, BEFORE the key starts. An example of how this might look follows:
from=”172.16.84.1,lanturtle,lanturtle.mydomain” ssh-rsa AAAA User_A@lanturtle

Now, let’s assume you also want to prevent proxying with this key. In order to restrict that, we need to set several options:
no-X11-forwarding – Prevents forwarding X11 sessions back to an X server via the SSH tunnel.
no-port-forwarding – Prevents port forwarding via TCP, TUN devices, and direct stdin proxying.
no-agent-forwarding – Prevents “forwarding” of the ssh-agent for handling the private key.

To add these to the example above, we would use a comma separated list of options like this:
from=”172.16.84.1,lanturtle,lanturtle.mydomain”,no-X11-forwarding,no-port-forwarding,no-agent-forwarding ssh-rsa AAAA User_A@lanturtle

We can further restrict this to only allowing this particular key to be used for running a specific command. Doing this ignores any commands passed by the user upon connection, and prevents the user from obtaining a full login shell (unless that’s the forced command.)
command=”/usr/local/bin/ssh-restricted-command.sh” (where “/usr/local/bin/ssh-restricted-command.sh” is the command you want it to run. Put the actual command you want to use, here.)
from=”172.16.84.1,lanturtle,lanturtle.mydomain”,no-X11-forwarding,no-port-forwarding,no-agent-forwarding,command=”/usr/local/bin/ssh-restricted-command.sh” ssh-rsa AAAA User_A@lanturtle

Finally, if we want to be sure that there isn’t a land mine waiting on the user’s .ssh/rc file, we can tell it to not read this by using the following option:
no-user-rc
from=”172.16.84.1,lanturtle,lanturtle.mydomain”,no-X11-forwarding,no-port-forwarding,no-agent-forwarding,no-user-rc,command=”/usr/local/bin/ssh-restricted-command.sh” ssh-rsa AAAA User_A@lanturtle

When we get into managing a certificate authority, you’ll find that these options can be set by ssh-keygen for certificate related keys, but the option names are slightly different. Unfortunately, using ssh-keygen to generate the initial public/private key with these options doesn’t work. It only works on certificates. Modifying the public key entry by hand does restrict this, however.

We will go into more detail about the forced command option (and setting one in sshd_config) next week.

Thanks for reading!

SSH Start to Finish Architecture – Tunneling IP with TUN device

We’ve covered the various tunneling / proxy capabilities of OpenSSH for tunneling TCP and setting up port proxies, but the “-w” flag can let you forward IP traffic. This happens by way of creation of a temporary “tun” device upon SSH connection, which can be configured like any network interface by standard tools such as “ifconfig.” There is some post connection set up that has to happen, including modifying route table information, so treat this like a right and proper VPN solution. That’s what it is meant to be. A VPN solution built into the SSH client and server software. I will do a proper write up with lab and examples later. For now, know that the server has to have the “PermitTunnel” option set, and it must be set to a value other than “no,” which is the default value.

The options are:
point-to-point – This allows a Layer 3 tunnel device to be configured.
ethernet – This allows a Layer 2 tunnel device to be configured.
yes – This allows either device type to be configured.

When connecting to the server, you need to pass the “-w” flag and provide a tun device id. You can also provide the remote tun device id. If you provide no remote id, it defaults to “any” which provides the next available. You may also specify “any” for the local tun device id.

The following example sets up a specific “tun0” on both the local and remote side.
ssh -w 0:0 User_B@Server_B

The following will set up a “tun” device using the next available ID on both the local and remote sides.
ssh -w any:any User_B@Server_B

The following will do the same as the example above, but saves 4 characters worth of typing.
ssh -w any User_B@Server_B

Just setting these up will not be sufficient. You will also need to configure the routes/interface information, of course. You will also need to configure any bastion host (local firewall) traffic rules as well, if you’re running one. The best explanation for this I’ve found is here:
Daemon Forums

When I do this as a lab exercise, I will show all of the detailed configuration for both the server set up, and the client set up. I feel like I half dropped the ball on this post for not having the lab set up already, but I think I got enough of the general overview across that this should help anyone interested in trying this out at least get started. We’re all still a little under the weather in the house, so a proper lab will have to wait, but I promise it will be done for this series before I am through.

Thanks for reading!

SSH Start to Finish Architecture – TCP Forwarding of STDIN/STDOUT

Last week we covered the standard TCP forwarding flags: -L, -R, and -D.

I explicitly held off on showing -W and -w, because I felt they deserved their own coverage.

We’ll cover the -w (dealing with tunnel devices) next week. This week is all about -W. Before we continue, you should note that one of my Wednesday posts was a brief overview of the netcat family of tools. If you think of “-W :” as a built in netcat for ssh, you’ll grasp the power of this flag pretty quickly. The most common example given for using this flag is the use of the ProxyCommand option to call SSH against a jumphost, and passing this flag to that host. The old way of utilizing ProxyCommand in this fashion was to actually call netcat as the remote proxy command, like so:

ssh -o ProxyCommand=’ssh jumphost netcat target_server 22′ target_server

Or if “Server_B” is our jumphost and we want to actually end up on “Server_A” by utilizing “Server_B” as a proxy, we would do it like this:
ssh -o ProxyCommand=’ssh Server_B netcat Server_A 22′ Server_A

That was all well and good, if netcat was actually installed, but sometimes it wasn’t. Luckily, SSH has its own built in netcat of sorts, via the -W flag. To do the same thing as above, we would do this, instead:
ssh -o ProxyCommand=”ssh -W %h:%p Server_B” Server_A

This is shorter, cleaner, and doesn’t rely on netcat at the jump server proxy point. Now let’s take this a step further. Let’s assume you want to pull a page down using command line, and you want to proxy through Server_B to get that page from Server_A.

echo “GET /some/page.html” | ssh -W Server_A:80 Server_B >page.html

Why might you want to do this? If you don’t have a direct route to Server_A on that port, but you can get to Server_B via SSH, and IT has access to Server_A on that port, you have a quick and dirty makeshift proxy.

Let’s sideline this thought process for a moment, and think about how many people try to lock down local accounts (stored in /etc/passwd.) Many people will give an account the “/bin/false” process as the “shell” to prevent logins to a shell. However, if you never actually attempt to get a shell with that account, you can use that account to proxy around all day long with the default sshd_config settings. If you want to disable this functionality, (and you should consider it, for the reasons demonstrated above,) you should set AllowTCPForwarding to “no” in the sshd_config file, and restart sshd.

There is a conversation floating around on social media right now about the Internet of Things botnet that took down Krebs’ blog, recently. One of the security vendors has suggested that an attack they are calling “SSHowDowN,” and this kind of functionality is exactly what they are referring to.

This kind of thing is really cool, when used correctly, but it’s also dangerous for poking holes in router and firewall access control lists.

If you have any experiences with this you’d like to share, leave a comment!

SSH – Start to Finish Architecture – TCP Tunneling

One of the great benefits of SSH is the ability to create port forwarding redirects for temporary access to a system that otherwise might not be able to access a machine. This is really easy to set up, and works quite well for many scenarios, but the downside to this is… it’s really easy to set up and works quite well for many scenarios you might not WANT it to work for. This can be used to punch holes in firewall rules, and that can get you in hot water with the wrong people, so make sure you understand your corporate policy before attempting to use anything like this at work.

SSH provides several kinds of TCP port forwarding, so we’ll cover them one at a time.
We will NOT be covering the tunneling of “TUN” devices via the -w flag this time. That will be covered at a later date.
We will also NOT be covering standard input/output forwarding via the -W flag this time. This is only going to cover TCP forwarding.

The first type of TCP forwarding we will look at is the Dynamic Port Forwarding (using SOCKS4/SOCKS5 protocol.) The flag for this is -D and takes an optional “bind_address” and a required “port” be passed to it.
-D [bind_address:]port

This creates a listening socket locally on the provided port and optionally the given bind_address that will act as a SOCKS4 / SOCKS5 application-level proxy. It’s best to set this up to bind to localhost, and then use one of the other tunneling options to point to this in order to protect the proxy from users you don’t want using it. Of course, anyone that can access the machine via ssh, will be able to use this, so be selective about where you set this up. Any application that is SOCKS aware may take advantage of this, as long as it can hit the listening port.

The next two options are really the same thing, but one sets up a listening port on the local (client) machine, and the other sets up a listening port on the remote (server) machine. The flags are -L and -R, respectively.

-L [bind_address:]port:host:hostport
-R [bind_address:]port:host:hostport

When you want to forward a local (client) listening port to the remote machine, use “L” for “local.”
When you want to forward a remote (server) listening port to the client machine, use “R” for “remote.”

In other words, that “listening port” is bound either locally, or remotely, and is one end of the tunnel. The “host” and “hostport” are not necessarily the same as the destination server you are connecting to with SSH. They just have to be a host and port that the remote system can actually get to.

“-L 8080:example.com:80” would bind a local port 8080 to the client when it establishes the connection to the remote ssh server. Then the remote ssh server would tie a connection to port 80 at example.com on its end. From there, you should be able to connect to port 8080 locally on your client machine, and talk to that remote example.com server on its port 80, all tunneled via ssh to the ssh server.

“-R 2222:localhost:22” would create a listening “localhost” bind on port 2222 on the remote server, that points to port 22 on the local client. If you were to connect outside of your network with this, it would allow people outside of the network to ssh in from that remote machine, assuming they have the correct credentials.

All of these options as given create a shell on login. You can establish JUST the tunnel by using the “-N” “-T” and “-f” options.

“-N” says to not call a remote command, just establish the tunnels.
“-T” says not to establish a pseudo TTY (PTY) terminal, which is appropriate since you aren’t passing any commands.
“-f” says to background ssh, since you probably want to do more work after the session is established.

ssh -fNT -L localhost:8080:google.com:80 User_B@jumphost

This says to background ssh after establishing a tunnel, and don’t allocate a PTY. Establish the tunnel with a locally bound port 8080, and have the remote ssh server “jumphost” establish a tunnel to “google.com” on port 80. Connect as User_B to the ssh server.

You can check with “netstat -an | grep LISTEN” to see if you have a listening port 8080 after establishing this. You can test the tunnel with netcat or telnet to localhost on port 8080 and enter “GET /” then hit enter. You might need to hit enter twice, depending. You’ll have to replace “User_B” and “jumphost” with an actual system you have access to, that also has access to the internet, of course.

I hope this didn’t muddy the waters more than it cleared them up a bit. If so, let me know in the comments, and I’ll try to clarify later.

SSH – Start to Finish Architecture – The Connection Flow

Before we get into any more advanced stuff with configuring SSH, I thought we should take a look at what actually happens when a client connects to an OpenSSH server, and what the decision tree is for granting or not granting access.

From the sshd man pages:
When a user successfully logs in, sshd does the following:
1. If the login is on a tty, and no command has been specified, prints last login time and /etc/motd (unless prevented in the configuration file or by ~/.hushlogin; see the FILES section).
~/.hushlogin
This file is used to suppress printing the last login time and /etc/motd, if PrintLastLog and PrintMotd, respectively, are enabled. It does not suppress printing of the banner specified by Banner.
2. If the login is on a tty, records login time.
3. Checks /etc/nologin; if it exists, prints contents and quits (unless root).
4. Changes to run with normal user privileges.
5. Sets up basic environment.
6. Reads the file ~/.ssh/environment, if it exists, and users are allowed to change their environment. See the PermitUserEnvironment option in sshd_config(5).
~/.ssh/environment
This file is read into the environment at login (if it exists). It can only contain empty lines, comment lines (that start with ‘#’), and assignment lines of the form name=value. The file should be writable only bythe user; it need not be readable by anyone else. Environment processing is disabled by default and is controlled via the PermitUserEnvironment option.
7. Changes to user’s home directory.
8. If ~/.ssh/rc exists, runs it; else if /etc/ssh/sshrc exists, runs it; otherwise runs xauth. The “rc” files are given the X11 authentication protocol and cookie in standard input. See SSHRC, below.
~/.ssh/rc
Contains initialization routines to be run before the user’s home directory becomes accessible. This file should be writable only by the user, and need not be readable by anyone else.
9. Runs user’s shell or command.

So from the above, we can see a few more ways to control our client connection and what it outputs when we connect. We looked at “LogLevel QUIET” in the ~/.ssh/config file last week, but we can also take advantage of the “.hushlogin” file to suppress some information.

We also see that the login gets logged only if there is a TTY associated. This is important to remember for forensics purposes.

We can temporarily disable logging into a system with SSH (other than root) by creating an /etc/nologin file, and the contents of that file will be displayed when the connection attempt gets rejected. It’s dangerous to use this if you don’t have console access, so be careful with it.

The service drops privileges and sets up a basic environment, then adjusts it from the ~/.ssh/environment file if it exists, and users are allowed to change their environment. The default behavior is to disallow this, but it’s something to check when locking down your systems. Finally, it changes to the user’s home directory to finish the environment preparations.

Next it reads and automatically runs the ~/.ssh/rc file if it exists. This is also important to know for forensics and for locking down your system. This is an excellent spot to drop a persistent misbehaving script, so it’s worth looking for and reviewing.

Finally, it runs the shell or whatever command was requested. Seems pretty simple, right? Well, the man pages stop short at that point.

So from a defense perspective, we want to review more than just the ~/.ssh/{config,authorized_keys,authorized_keys2,known_hosts} files. We also want to look at any rc and environment files in that directory. This is especially true of the root user.

Next week we’ll look at the sshd_config file and cover how to check the running configuration against the written config file.

SSH – Start to Finish Architecture – Client Config Pt. 1

This week, we’re going to focus on the client side configuration file(s) and how to use them to make ssh a more enjoyable experience. There are a lot of options in the configuration files, so we’re going to break this up into different parts. We’ll cover some common settings people might want to play with for making session handling more efficient, and then after we’ve covered some more advanced topics over the next few weeks, we’ll do a part two to go over some of the more advanced configuration options that mesh with those previously covered topics. This means part two won’t be next week.

There are two default files that handle client side ssh configuration in OpenSSH. The global file is usually found at /etc/ssh/ssh_config, and contains all of the default settings for all users on the system. The user can also write configuration options that override the global config file by including the file “config” inside that user’s .ssh directory. When the ssh client is called, the order of priority when parsing options is covered thusly:

1. command-line options
2. user’s configuration file (~/.ssh/config)
3. system-wide configuration file (/etc/ssh/ssh_config)

This whatever is given on the command line trumps everything else. The user’s config trumps the system-wide config. The system-wide config covers everything not explicitly overridden by the other two. This makes sense, but it’s important to know what the priority of parsing is when troubleshooting issues.

The focus for today will cover the ~/.ssh/config file that should contain customized settings.

One of the first options I set in my own config is the “LogLevel” setting. I prefer to suppress extraneous information (such as banners,) because I like to batch process reports from multiple servers, and I only want the information from the commands I run, not anything else. To suppress these messages, I use:

LogLevel QUIET

You can tune this to whatever level of noise you prefer, all the way up to DEBUG3, which is equivalent to “ssh -vvv” for “verbose debugging.” I don’t recommend using the user config for this, though. Just use the flags to override, and use the config to suppress as above.

The next set of options people often use are the “Match,” “User,” “Host,” and “HostName” settings.
The “User” setting is used to set what the target username to login as should be. This is like going from User_A@Workstation_A to User_B@Server_B scenario we set up before. In this case, setting “User User_B” would mean you could type “ssh Server_B” and it would know to use “User_B” as the target, instead of the default of “User_A.”

The “Host” option sets a block for options that apply to a given hostname until the next “Host” or “Match” block. The name can be a pattern match with wild cards, and can include negation via a preceding “!” if needed. “Host !*.web.com” would say “the following settings up to and not including the next Match or Host block apply to all systems that are NOT like *.web.com.” The previous example is a bad one, but it gets the point across. To set global stuff at the bottom, use a “Host *” option to match everything, and the earlier Match/Host blocks will override as needed, since they have already set values, if something matches them.

The “Match” option is used to fine tune matching against various objects. It can match a username with “Match user” or an address with “Match address” and so on. You can do a “Match host,” but just using “Host” as a block option is probably better for almost all cases.

The “HostName” option lets you set aliases within your config. This would generally be set as an option after a “Host” match. You can use “%h” to indicate the hostname that was passed to ssh on the command line before altering it with the “HostName” so that “HostName” prepends or appends to it as needed. You can also set to this an address to override whatever DNS would have returned, for example.

Using what we’ve learned so far, let’s assume that User_A wants to log in as User_B in almost all cases. However, User_A is an AIX systems administrator, and thus might need to log into a VIO server for virtual IO setup, which means the target user is most likely “padmin” rather than User_A or User_B. Also, AIX is managed by a Hardware Management Console (HMC) which often uses the “hscroot” user, rather than local accounts. Of course, every shop is different, but this is a standard practice, so we’ll go with it. The organization uses a standard naming scheme that all VIO servers have a name that starts with “vio” followed by some unique identifier. The HMCs are similarly named as “hmc” followed by some unique identifier. The VIO servers are also in a different domain than the standard, called ‘internal.net.’ We also know that if the user logs in as root, a key will never be used, so we want to skip the key exchange and force a password for root, instead. Armed with this knowledge, we can make User_A’s life easier if we build out a config that looks like this:

LogLevel QUIET
Host vio*
   HostName “%h.internal.net”
   User padmin
Host hmc*
   User hscroot
Match user root
   PubkeyAuthentication no
   PasswordAuthentication yes
   PreferredAuthentications password
Match user User_B
   PreferredAuthentications publickey,password
Host *
   User User_B

The options we set for the “Match user root” block should be fairly self explanatory at this point. There are many more options than this, and we will cover them eventually, but this is decent general overview of how to build out your config over time. Remember to put your global stuff at the bottom, since it reads top down with first match winning the option to set.

SSH – Start to Finish Architecture – Securing The Private Key

Our previous post showed how to generate the bare bones public/private key pair without using a passphrase. This is sometimes the desired configuration, but it is better to lock down the private key using a passphrase. When you generate the key pair, you can add a passphrase at the prompt that we just hit “enter” on last week, but you might want to change an existing passphrase or add a passphrase to a key that doesn’t already have one. The means for doing this is shown below:
ssh-keygen -f ~/.ssh/id_rsa -p

If the existing phrase is empty (like the one we generated last week,) this will prompt you for your new passphrase right away. If there is an existing passphrase, it will first prompt for that before prompting for the new one. Setting a passphrase on a private key is an important step to securing that key. If someone unauthorized to use that key managed to get a copy of it somehow, they won’t be able to use the key until they figure out the passphrase for it. While it is possible to brute force crack a key, if you use a decently long phrase that isn’t something commonly spoken or written, the chances of cracking it go down. Also note that SSH key passphrases allow for spaces, so you can literally write nonsense sentences, spaces and all. There is more that can be done to reduce the risk of someone using a stolen private key to do harm, but it’s on the client side, and there are caveats. We’ll cover that next week.

Now that we have a passphrase protecting our private key, what has changed in how ssh works? For starters, if you don’t use an Agent to load your keys to, every time you go to log into a server using this key, you will be prompted for the passphrase like you used to be prompted for a password. This makes convenience worse, not better. To use the agent, run ssh-add. If you’re using a standard key name such as id_rsa, id_dsa, or id_ecdsa, it will automatically find and load that key for you. For each key with a standard name it finds, it will prompt for the passphrase. You give it the phrase and it handles the rest. It acts on your behalf from then until it is told to unload a key or is stopped. When you go to login, the SSH client will see that the agent is running, and when prompted for the key by the server, it will pass that request through to the agent, which will provide proof that it knows the key, and thus you won’t be prompted. It’s like promptless SSH, but requires the extra step of loading the agent first.

If you get an error message when you run ssh-add, there is a chance that ssh-agent isn’t already running. If that is the case, you can start ssh-agent first, take the output it gives, and export those variables. For example:

ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-w8iG9Aq6KWLR/agent.1070; export SSH_AUTH_SOCK;
SSH_AGENT_PID=1071; export SSH_AGENT_PID;

If you have a key with a custom name such as id_rsa_2016, you can load these by passing their name, like so:
ssh-add /home/User_A/.ssh/id_rsa_2016

Using the agent is dangerous in a shared environment where other people have elevated privileges. Anyone with root can potentially pull the private key from memory while the agent is running on your behalf. You can unload the keys before locking your workstation if you’re paranoid enough, by using -D or -d as below:
ssh-add -D #Delete all identities
ssh-add -d /home/User_A/.ssh/id_rsa_2016 #Deletes just the id_rsa key from the agent list

You can also lock and unlock your agent using the “-x” and “-X” flags respectively, if you don’t want to completely unload for security’s sake. These will prompt you for a password to use for locking and unlocking the agent, if you choose to use them.

If you want to see which keys are loaded, you can list them with “ssh-add -l.” And if you need to be sure which public key matches the loaded private key, you can use “ssh-add -L.”

Finally, if you want to set a time limit on a key being loaded, you can use the -t flag to make it temporary. It requires a number (in seconds) for how long the key should remain loaded by the agent.

The rest of the flags are for more advanced stuff I will be covering separately, so that’s all we’ll cover for today. If you’ve kept up thus far, you’re pretty much at the level of the average SSH user at this point. (And there’s so much more to be covered.) Next week, we’ll go over some client configuration options to make session management easier.

SSH – Start to Finish Architecture – Our First Keys

Last week, we set the stage for the bare bones SSH installation and configuration for logging in with a password. This works for many people, but it isn’t very flexible. What if the user needs to run a workload remotely from a script? The script will get prompted for a password, and that isn’t handled gracefully without some helper such as an Expect wrapper. A perl call to Expect.pm, or a raw TCL based expect, or one of the myriad of other languages with an Expect module would add bloat to what would potentially be a lean and mean shell script, otherwise. Also, passwords are often cracked somewhat easily if the target system (Server_B) is configured poorly for handling repeated password attempts. Brute force attempts happen all the time on internet facing machines, even. So how do we get rid of the need for a password? We will use a public/private key pair to handle the authentication for us.

Remember that the best place to generate your public/private key pair is as the user on the machine you are coming FROM. In our example scenario, that would be User_A on Workstation_A. The key generation can be complex, but for now, we will just generate a key without a passphrase. Next week, we will cover using a passphrase on the key, why it is important to do this, and how to handle being able to still use scripts without getting prompted.

To generate our initial set of keys, we’ll use the ssh-keygen utility supplied by OpenSSH.

ssh-keygen -t rsa -b 4096 -C “User_A@Workstation_A initial key”

This should prompt for a path and name (usually the home_directory/.ssh/id_rsa) where the key will be generated.
It should also prompt for a passphrase, but we’re making that blank for now, so just hit “Enter” to move along.
Finally, it’ll do some calculations, print out a “bubble babble” output, and end. Once it’s done, you should see two new keys in the directory it gave. Do an “ls -l ~/.ssh” and see if you have an “id_rsa” and “id_rsa.pub” file, now (assuming you didn’t change the path or name when prompted.)

Okay, a few things are going on with this command. First, the flag “-t rsa” is telling the command to generate the keys using the RSA encryption algorithm. The “-b 4096” says to make that a 4096 bit RSA algorithm. The “-C” flag puts a comment in the file, which helps identify the purpose of the key and who owns it. This comment appears in the public key out of the pair. If you cat out that file, you should see the comment tag at the end of the key.

Once you have your key pair, understand that the “id_rsa” file is your private key. This should be locked down (usually with 600 permissions) and should not be copied to any other systems (barring some very rare exceptions.) The public key is “id_rsa.pub” and is what you need to push out to User_B at Server_B. The contents of this public key file actually go into a different file for that user. They should be placed into the ~User_B/.ssh/authorized_keys file. There are multiple ways to get this pushed out. Many people prefer to use the “ssh-copy-id” command to push this out, but I don’t like how it handles the authorized_keys file normally. It just appends to it blindly. I prefer to write a function that handles pushing the contents to the remote server along with catting out the remote server’s authorized_keys file, uniq the output, then write it back to the authorized_keys file. This procedure ensures only one copy of the public key ever gets pushed, no matter how many times you do this. The function details are in my “DSH – Distributed/Dancer’s Shell” product. If you don’t see a link for that yet, it’s because the product is in Alpha testing, and will be launched as a Beta soon. I’ll post a link when it goes live, for those who are interested.

Once the public key is on the remote server, make sure the permissions on the authorized_keys file are also 600. The .ssh directory should be 700. The home directory shouldn’t be readable by world. Then test your connection from User_A@Workstation_A to User_B@Server_B. It should “just work” and drop you at a shell prompt. If it doesn’t, you can check the logs on the remote server (usually /var/log/secure.log or /var/log/sshd.log or similar) for clues, and you can re-try the connection with “ssh -vvv” to get verbose output for other clues. Often it’s a permissions error, so double check those file permissions.

This will now allow User_A on Workstation_A to write scripts that can call “ssh User_B@Server_B ” for remote workloads to be run, without getting prompted for a password.

This isn’t really the best way to handle this, though, so we’ll look at how to secure the private key with a passphrase next week, and still be able to do this without getting prompted on every connection.

Thanks for sticking with me on this. There’s plenty more to come!

SSH – Start to Finish Architecture – The Basic Setup

Once upon a long time ago in a place not too far away, the internet was born. In its early days, people used basic protocols to interact with each other and share information. One batch of these protocols allowed people to connect remotely to another machine and either login for an interactive shell, or pass remote commands to be run for processing. These were telnet, rsh, and rlogin. There were also rcp and ftp for file transfers. All of these were great until people started needing to lock their doors at night because the neighbors got too nosey. Security became a concern, and SSH was born.

Today, most people think of OpenSSH as the defacto software for SSH services. It contains both a server component (sshd), an sftp subsystem (to replace FTP) and a few clients for handling remote logins and file transfers (ssh and scp.) The truth is, there are plenty of packages to choose from, for both server side and client side communication with these new protocols. I may cover some of these, but this series is going to start with OpenSSH as a focus simply because it is the most popular at this time. We’ll start with a basic setup, and then progress through more complex setups to cover various needs based on scale of the operation needing this kind of access arrangement.

An initial explanation of how this works is a user on workstation A wants to be able to log in remotely to server B to run some kind of work load. Remember that every SSH session is composed of pieces of information at its most basic level. A user (User_A) on a workstation or server (Workstation_A) wants to connect remotely to a server (Server_B) as some user (User_B) to run a work load. We will use these labels moving forward to describe how to configure the software for each kind of scenario we want to cover. Eventually there will be a third system involved (either LDAP, or a Certificate Authority, or both) so we’ll toss in an _C label when we get to that. For now, just remember “_A” will represent “coming from” and “_B” will represent “going to” for directionality of each session flow.

So before User_A can connect to Server_B with SSH, Server_B needs to have an SSH service running. The software can be installed from source, or it can be installed from whatever package management system your server OS supports. RPM/yum and DPKG/apt-get are common for RedHat and Debian based systems, respectively. AIX has installp/RPM. HP-UX has swinstall. Just use the appropriate software for the system you are supporting, but understand that some of the more advanced features will require newer versions of OpenSSH, so if you try something and it doesn’t work, check your server version.

Now that OpenSSH sshd has been installed via whatever means you chose, User_A needs client software. It should be the same deal, here. Use whatever package management software you choose. There is an OpenSSH client via the Cygwin project for Windows for the “follow along” when your workstation is Windows, but other clients may work better for most of these scenarios. PuTTy and its derivatives, WinSCP, MobaXTerm are all excellent Windows clients. However, when we get to certificate based connections, these may not be sufficient. Just install OpenSSH client for now on whatever platform you’re coming from, and let’s move on.

Now Workstation_A and Server_B have the software needed to make a connection. If User_A wants to connect to Server_B as User_B, a “User_B” account will need to exist on the server, or at least be a user that can authenticate (LDAP or similar.) For now we will assume local accounts are in use. So to make a connection, User_A on Workstation_A will need to know the password for User_B on Server_B. Then typing the following will allow a connection to be established (assuming the account is active and not locked.)

As User_A from a terminal prompt:
ssh User_B@Server_B

It’s that simple. This is almost not worth covering, I know. However, every post in this series will take a step toward making things work more smoothly from an operations and security perspective, so stick with me. Next week we’ll cover setting up public/private key pairs for authentication without a password.