Author: Stefan

SSH – Start to Finish Architecture – Client Config Pt. 1

This week, we’re going to focus on the client side configuration file(s) and how to use them to make ssh a more enjoyable experience. There are a lot of options in the configuration files, so we’re going to break this up into different parts. We’ll cover some common settings people might want to play with for making session handling more efficient, and then after we’ve covered some more advanced topics over the next few weeks, we’ll do a part two to go over some of the more advanced configuration options that mesh with those previously covered topics. This means part two won’t be next week.

There are two default files that handle client side ssh configuration in OpenSSH. The global file is usually found at /etc/ssh/ssh_config, and contains all of the default settings for all users on the system. The user can also write configuration options that override the global config file by including the file “config” inside that user’s .ssh directory. When the ssh client is called, the order of priority when parsing options is covered thusly:

1. command-line options
2. user’s configuration file (~/.ssh/config)
3. system-wide configuration file (/etc/ssh/ssh_config)

This whatever is given on the command line trumps everything else. The user’s config trumps the system-wide config. The system-wide config covers everything not explicitly overridden by the other two. This makes sense, but it’s important to know what the priority of parsing is when troubleshooting issues.

The focus for today will cover the ~/.ssh/config file that should contain customized settings.

One of the first options I set in my own config is the “LogLevel” setting. I prefer to suppress extraneous information (such as banners,) because I like to batch process reports from multiple servers, and I only want the information from the commands I run, not anything else. To suppress these messages, I use:

LogLevel QUIET

You can tune this to whatever level of noise you prefer, all the way up to DEBUG3, which is equivalent to “ssh -vvv” for “verbose debugging.” I don’t recommend using the user config for this, though. Just use the flags to override, and use the config to suppress as above.

The next set of options people often use are the “Match,” “User,” “Host,” and “HostName” settings.
The “User” setting is used to set what the target username to login as should be. This is like going from User_A@Workstation_A to User_B@Server_B scenario we set up before. In this case, setting “User User_B” would mean you could type “ssh Server_B” and it would know to use “User_B” as the target, instead of the default of “User_A.”

The “Host” option sets a block for options that apply to a given hostname until the next “Host” or “Match” block. The name can be a pattern match with wild cards, and can include negation via a preceding “!” if needed. “Host !*.web.com” would say “the following settings up to and not including the next Match or Host block apply to all systems that are NOT like *.web.com.” The previous example is a bad one, but it gets the point across. To set global stuff at the bottom, use a “Host *” option to match everything, and the earlier Match/Host blocks will override as needed, since they have already set values, if something matches them.

The “Match” option is used to fine tune matching against various objects. It can match a username with “Match user” or an address with “Match address” and so on. You can do a “Match host,” but just using “Host” as a block option is probably better for almost all cases.

The “HostName” option lets you set aliases within your config. This would generally be set as an option after a “Host” match. You can use “%h” to indicate the hostname that was passed to ssh on the command line before altering it with the “HostName” so that “HostName” prepends or appends to it as needed. You can also set to this an address to override whatever DNS would have returned, for example.

Using what we’ve learned so far, let’s assume that User_A wants to log in as User_B in almost all cases. However, User_A is an AIX systems administrator, and thus might need to log into a VIO server for virtual IO setup, which means the target user is most likely “padmin” rather than User_A or User_B. Also, AIX is managed by a Hardware Management Console (HMC) which often uses the “hscroot” user, rather than local accounts. Of course, every shop is different, but this is a standard practice, so we’ll go with it. The organization uses a standard naming scheme that all VIO servers have a name that starts with “vio” followed by some unique identifier. The HMCs are similarly named as “hmc” followed by some unique identifier. The VIO servers are also in a different domain than the standard, called ‘internal.net.’ We also know that if the user logs in as root, a key will never be used, so we want to skip the key exchange and force a password for root, instead. Armed with this knowledge, we can make User_A’s life easier if we build out a config that looks like this:

LogLevel QUIET
Host vio*
   HostName “%h.internal.net”
   User padmin
Host hmc*
   User hscroot
Match user root
   PubkeyAuthentication no
   PasswordAuthentication yes
   PreferredAuthentications password
Match user User_B
   PreferredAuthentications publickey,password
Host *
   User User_B

The options we set for the “Match user root” block should be fairly self explanatory at this point. There are many more options than this, and we will cover them eventually, but this is decent general overview of how to build out your config over time. Remember to put your global stuff at the bottom, since it reads top down with first match winning the option to set.

Fun-Day Friday – Ramblings

This is going to be short and sweet. I’ve been busy getting ready for a big event in our household, and thus I shorted myself on time to publish this. I apologize for that.

I’ll treat this as a week in review and go over some of the cool things that happened.

1) Michael W. Lucas has released his new “Mastery” book, PAM Mastery. PAM is one of those rare dark arts that most Unix SysAdmins prefer to meddle with as little as possible for fear of summoning a maentwrog or something. This book sheds some really bright light on how to tackle the appropriate incantations so you don’t do something that bites your head off in the end.

2) I’ve made some progress on troubleshooting one of the more recent modules released on the LAN Turtle. A new version of that module has already been released, so I get to start all over with testing it to see if it fixed any of the issues I was having. I’ll do that testing this weekend.

3) Guild Wars 2 released the second episode of the Living Story season 3. GW2 is one of a very small handful of computer games I still play these days. The content thus far is pretty good. I was hoping ArenaNet would put in a petrified wood node, but maybe it’ll come later.

4) My WiFi Pineappling book came in the mail. I’ll go through it, and do a review when I cover the Hak5 WiFi Pineapple NANO later down the road.

5) My wife bought me a present today. Halloween is my favorite holiday of the year, and she brought home a skull on a stand. I’ll take it to work and start decorating my cube for the holiday soon. It should go nicely with the skulls chain I hang every year.

What are your favorite games, holidays, or other activities and events? Share in the comments!

Hacker-Tool Hump-Day – Hak5 LAN Turtle Part 1

I’m going to spend a few days covering the Hak5 LAN Turtle. I may or may not break these up with some other Hacker-Tool Hump-Day topics, but for today, we’re just going to cover the initial set up of a brand new Turtle.

The first time you plug it into a USB port, you’ll need to wait a moment for it to initialize and present an IP to the host machine. Once an IP has been obtained (in the 172.16.84.x range,) bring up your favorite SSH client (such as PuTTY) and connect to 172.16.84.1 as root. The initial default password is “sh3llz” and the first time you log in you should see a screen prompting you to set a new root password.

Enter the password of your choosing. It will prompt you to repeat the password. Do so, and it will either say it didn’t succeed and start the prompt sequence over, or it will say the password was changed successfully. Hit enter to select “Okay” and move on.

The first thing you should do after setting that initial root password is to check for updates and update the device. It’s a good idea to check for updates regularly, even if they aren’t released often. Select “Config” and hit enter. At the new menu, you can see that you can change the password you just set, change the WAN or the LAN MAC addresses, disable the Turtle Shell, and Check for updates. Use the arrow keys to move down to “Check for updates,” and hit enter. If you don’t have the ethernet side plugged into a switch that has internet access, this will fail. Don’t forget to plug that cable in.

If an update is available, it will download and verify the update files, update the LAN Turtle with a warning that it could take about 5 minutes, and a note that the Turtle will reboot after updates. It will then kick you out of the SSH session. Just wait for it to update, and keep an eye on the yellow flashy light.

Once it’s been updated, you can restart your session. The HOST keys will have changed, so you should get a warning about that if you aren’t suppressing those, and after you accept the new keys, you will need to login with the default sh3llz again. It will prompt you to change that password (again) so do so at this time. If you go back through the “Config” and “Check” menus again, you should see that there are now no new updates at this time.

I don’t recommend Disabling the turtle shell unless and until you have things set up the way you want, but if you do, you can always get back to the turtle menu by calling “turtle” from the command line.

The next step is to back out to the main menu and pick your modules through the “Module configuration” menu option. The first time you go into this menu, the only option you can select is the “Module Manager.” Select this and choose “Configure.” The next menu should let you choose which modules you’d like to install (“Directory”) or which ones to delete (“Delete”) as well as updating all installed modules via “Update.” Just like the initial update of the Turtle itself, these actions require an internet capable connection to be plugged into the Ethernet side of the Turtle.

We will go over each of the modules over time, but for now you know where this is at and you can select any you would like to play with. The “Cron” module is not necessary for the “at” command, as that appears to be installed already by default.

After you’ve selected all of the modules you want (and you can select all of them… at the time of this writing, they all fit with some space to spare…) install them and then back out of the menu to the Main menu again. Now select “Exit” to drop to the command line.

The Module Manager from the Turtle menu shell is good for pre-packaged installation and configuration of stuff, but you can also install packages that may not be part of a module from the command line. The LAN Turtle is a MIPS based OpenWRT build, and thus any packages compiled for that platform -should- work for the Turtle, assuming the binary fits in the limited file system space provided. The device appears to have 40 megs total, 30 of which are assigned to /tmp as a tmpfs file system. This means 30 megs are volatile. This still shouldn’t be a problem if you select your packages carefully.

The package type for OpenWRT devices is a “.ipk” file, and you manage the packages from the command line using the “opkg” command. The “opkg list” command claims to list available packages, but seems to list installed ones instead. The “opkg list-installed” is supposed to list installed packages, but returns nothing. To fix this, you need to run an initial “opkg update” first, which will grab the repository lists and update the local opkg database with the installed information. The “opkg search” command works as advertised with or without that initial update, so you can find out what command belongs to what package. For example, “opkg search ‘*ps'” returns “busybox – ” which means ps is part of the busybox package. If you’re a fan of multiplexers, there is a package for both “screen” and “tmux.” The “tmux” binary is slightly smaller than the “screen” one and is actually my preferred multiplexer. Alas, there was no package for “dsh.” I may choose to attempt a build of this down the road, because this is an incredibly useful tool. If I do this, I’ll document the process here as well as document turning it into an official Turtle module that could be selected from the Module Manager in the Turtle menu shell.

From a Systems Administration perspective, this device does a few things. It provides a handy USB ethernet adapter if all modules are turned off. It also provides a slimmed down linux environment reminiscent of a Raspberry Pi, BeagleBone Black, or similar, with a smaller form factor, but at the cost of losing some computing power. This is good for providing a quick environment where a VM would be overkill and you’re stuck with a Windows machine as your workstation. Just be aware that your Corporate Security team may take exception to you using such a device, even if no hostile modules are installed. Make sure you are transparent and work things out with them before plugging a “rogue” device into the network in a corporate environment.

Some of the more useful SysAdmin related tools in the Module Manager are “autossh,” “sshfs,” “clomac (if the network is locked down by MAC,)” and “cron” for obvious reasons. I’ll cover each module available over time, and explain the “Configure,” “Start/Stop,” and “Enable/Disable” menu options for each in their own articles. This way each gets the right amount of focus.

Thanks for reading, and if you like this kind of thing, check out the forums at Hak5 to see the latest discussion.

SSH – Start to Finish Architecture – Securing The Private Key

Our previous post showed how to generate the bare bones public/private key pair without using a passphrase. This is sometimes the desired configuration, but it is better to lock down the private key using a passphrase. When you generate the key pair, you can add a passphrase at the prompt that we just hit “enter” on last week, but you might want to change an existing passphrase or add a passphrase to a key that doesn’t already have one. The means for doing this is shown below:
ssh-keygen -f ~/.ssh/id_rsa -p

If the existing phrase is empty (like the one we generated last week,) this will prompt you for your new passphrase right away. If there is an existing passphrase, it will first prompt for that before prompting for the new one. Setting a passphrase on a private key is an important step to securing that key. If someone unauthorized to use that key managed to get a copy of it somehow, they won’t be able to use the key until they figure out the passphrase for it. While it is possible to brute force crack a key, if you use a decently long phrase that isn’t something commonly spoken or written, the chances of cracking it go down. Also note that SSH key passphrases allow for spaces, so you can literally write nonsense sentences, spaces and all. There is more that can be done to reduce the risk of someone using a stolen private key to do harm, but it’s on the client side, and there are caveats. We’ll cover that next week.

Now that we have a passphrase protecting our private key, what has changed in how ssh works? For starters, if you don’t use an Agent to load your keys to, every time you go to log into a server using this key, you will be prompted for the passphrase like you used to be prompted for a password. This makes convenience worse, not better. To use the agent, run ssh-add. If you’re using a standard key name such as id_rsa, id_dsa, or id_ecdsa, it will automatically find and load that key for you. For each key with a standard name it finds, it will prompt for the passphrase. You give it the phrase and it handles the rest. It acts on your behalf from then until it is told to unload a key or is stopped. When you go to login, the SSH client will see that the agent is running, and when prompted for the key by the server, it will pass that request through to the agent, which will provide proof that it knows the key, and thus you won’t be prompted. It’s like promptless SSH, but requires the extra step of loading the agent first.

If you get an error message when you run ssh-add, there is a chance that ssh-agent isn’t already running. If that is the case, you can start ssh-agent first, take the output it gives, and export those variables. For example:

ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-w8iG9Aq6KWLR/agent.1070; export SSH_AUTH_SOCK;
SSH_AGENT_PID=1071; export SSH_AGENT_PID;

If you have a key with a custom name such as id_rsa_2016, you can load these by passing their name, like so:
ssh-add /home/User_A/.ssh/id_rsa_2016

Using the agent is dangerous in a shared environment where other people have elevated privileges. Anyone with root can potentially pull the private key from memory while the agent is running on your behalf. You can unload the keys before locking your workstation if you’re paranoid enough, by using -D or -d as below:
ssh-add -D #Delete all identities
ssh-add -d /home/User_A/.ssh/id_rsa_2016 #Deletes just the id_rsa key from the agent list

You can also lock and unlock your agent using the “-x” and “-X” flags respectively, if you don’t want to completely unload for security’s sake. These will prompt you for a password to use for locking and unlocking the agent, if you choose to use them.

If you want to see which keys are loaded, you can list them with “ssh-add -l.” And if you need to be sure which public key matches the loaded private key, you can use “ssh-add -L.”

Finally, if you want to set a time limit on a key being loaded, you can use the -t flag to make it temporary. It requires a number (in seconds) for how long the key should remain loaded by the agent.

The rest of the flags are for more advanced stuff I will be covering separately, so that’s all we’ll cover for today. If you’ve kept up thus far, you’re pretty much at the level of the average SSH user at this point. (And there’s so much more to be covered.) Next week, we’ll go over some client configuration options to make session management easier.

Fun-Day Friday – Not So Fun

A couple of weeks ago, a fellow HAM in the local Amateur Radio Club had a mishap with his Elecraft KX3 and PX3 rig setup. It was a nice day, weather wise, so he called an impromptu “Hamming In The Park.” This is where a bunch of us grab our portable capable rigs, head to one of the parks in town, set up, and operate in the (usually sunny) outdoors.

He let the smoke out of his KX3, and here’s the story:

http://ae5nw.net/index.php?option=com_content&view=article&id=48&Itemid=54&limitstart=25

I personally am a huge fan of Elecraft. I own the KX3 (radio,) the PX3 (waterfall display and control,) and the KXPA100 (amplifier.) I use the KX3 on the amp at the house to get up to 110W of power (comparable to most non-amp HF rigs on the market.) It’s a sweet rig, but cheap is not part of its description. I’ll definitely be taking his experience to heart and making sure my setup doesn’t connect power until everything else is hooked up moving forward.

Hacker-Tool Hump-Day – nping

Last week, we looked at a physical device that is usually used in pentests for social engineering (by dropping it in hopes than a curious random person will pick it up, plug it in, and deliver the unexpected payload.) I also showed how it could be used “for good” by a SysAdmin for normal, every day tasks.

This week, we’re going to look at a purely software tool; nping. The nping program is provided by the good folks at the http://nmap.org/ and http://sectools.org/ websites. It is provided by the nmap package, and is often overlooked by the SysAdmin community. Most companies restrict the use of nmap to the corporate security teams, and exclude use of the tool from the daily Unix SysAdmin teams. This is understandable, but if you’re in that situation, you might still find “nping” useful, and your corporate security team might be more willing to sign off on its use for troubleshooting network issues.

Many people make the mistake of thinking that ICMP Pings are great for testing latency. The truth is, they often give a false sense of how much latency exists between one system and another talking TCP for their connections. Many routers and firewalls have QoS settings that can add artificial latency based on protocol, and where ICMP might be blazing fast, TCP might be slow as dirt, and leave you scratching your head as to why. If you use a tool like nping to test your latency instead, you get a truer view of the latency involved.

How does this work?

The nping command will make a handshake request to a given port at a given IP/DNS name a given number of attempts, and then show statistics about the latency from there. There are options that let you change what kind of flags get sent for the handshake, just like with nmap, so you may have to write a wrapper script to get approval from your corporate security team, but as long as you always use the “tcp connect” option, it should do a full blown standard TCP handshake, which would prevent it from setting off any IDS/IPS alarms they may be concerned with.

nping -c 1 –tcp-connect -p 80 server_b.example.com

In the above example, we’re telling nping to make a tcp connect handshake (–tcp) to port 80 (-p 80) at server_b.example.com. We want it to stop after 1 attemp (-c 1.) This will print the time to send the packet, receive the response packet, and the max, min, and average round trip time of all packets:
SENT (0.0031s) Starting TCP Handshake > server_b.example.com:22 (127.0.0.1:22)
RECV (0.0032s) Handshake with server_b.example.com:22 (127.0.0.1:22) completed

Max rtt: 0.053ms | Min rtt: 0.053s | Avg rtt: 0.053s

Since we sent only one handshake attempt, all three “rtt” values will be the same. If we allowed more than one to be sent, we’d see different numbers there.

After the “rtt” values, it tells you how may TCP connection attempts were made, how many were successful, and how many failed.
TCP connection attempts: 1 | Successful connections: 1 | Failed: 0 (0.00%)

I usually like to send 4 to 10 packets and then check the average RTT when troubleshooting latency. Do that in a loop over several minutes (4 to 10 per iteration of the loop) and you start to see how the network is behaving over time. It’s a good tool for every SysAdmin toolbox, so don’t overlook it.

I know that there are some other tools that do similar things, such as Hping2 and Hping3, but nping is pretty straight forward.

If you have tools you’d like to see covered, leave a comment!

SSH – Start to Finish Architecture – Our First Keys

Last week, we set the stage for the bare bones SSH installation and configuration for logging in with a password. This works for many people, but it isn’t very flexible. What if the user needs to run a workload remotely from a script? The script will get prompted for a password, and that isn’t handled gracefully without some helper such as an Expect wrapper. A perl call to Expect.pm, or a raw TCL based expect, or one of the myriad of other languages with an Expect module would add bloat to what would potentially be a lean and mean shell script, otherwise. Also, passwords are often cracked somewhat easily if the target system (Server_B) is configured poorly for handling repeated password attempts. Brute force attempts happen all the time on internet facing machines, even. So how do we get rid of the need for a password? We will use a public/private key pair to handle the authentication for us.

Remember that the best place to generate your public/private key pair is as the user on the machine you are coming FROM. In our example scenario, that would be User_A on Workstation_A. The key generation can be complex, but for now, we will just generate a key without a passphrase. Next week, we will cover using a passphrase on the key, why it is important to do this, and how to handle being able to still use scripts without getting prompted.

To generate our initial set of keys, we’ll use the ssh-keygen utility supplied by OpenSSH.

ssh-keygen -t rsa -b 4096 -C “User_A@Workstation_A initial key”

This should prompt for a path and name (usually the home_directory/.ssh/id_rsa) where the key will be generated.
It should also prompt for a passphrase, but we’re making that blank for now, so just hit “Enter” to move along.
Finally, it’ll do some calculations, print out a “bubble babble” output, and end. Once it’s done, you should see two new keys in the directory it gave. Do an “ls -l ~/.ssh” and see if you have an “id_rsa” and “id_rsa.pub” file, now (assuming you didn’t change the path or name when prompted.)

Okay, a few things are going on with this command. First, the flag “-t rsa” is telling the command to generate the keys using the RSA encryption algorithm. The “-b 4096” says to make that a 4096 bit RSA algorithm. The “-C” flag puts a comment in the file, which helps identify the purpose of the key and who owns it. This comment appears in the public key out of the pair. If you cat out that file, you should see the comment tag at the end of the key.

Once you have your key pair, understand that the “id_rsa” file is your private key. This should be locked down (usually with 600 permissions) and should not be copied to any other systems (barring some very rare exceptions.) The public key is “id_rsa.pub” and is what you need to push out to User_B at Server_B. The contents of this public key file actually go into a different file for that user. They should be placed into the ~User_B/.ssh/authorized_keys file. There are multiple ways to get this pushed out. Many people prefer to use the “ssh-copy-id” command to push this out, but I don’t like how it handles the authorized_keys file normally. It just appends to it blindly. I prefer to write a function that handles pushing the contents to the remote server along with catting out the remote server’s authorized_keys file, uniq the output, then write it back to the authorized_keys file. This procedure ensures only one copy of the public key ever gets pushed, no matter how many times you do this. The function details are in my “DSH – Distributed/Dancer’s Shell” product. If you don’t see a link for that yet, it’s because the product is in Alpha testing, and will be launched as a Beta soon. I’ll post a link when it goes live, for those who are interested.

Once the public key is on the remote server, make sure the permissions on the authorized_keys file are also 600. The .ssh directory should be 700. The home directory shouldn’t be readable by world. Then test your connection from User_A@Workstation_A to User_B@Server_B. It should “just work” and drop you at a shell prompt. If it doesn’t, you can check the logs on the remote server (usually /var/log/secure.log or /var/log/sshd.log or similar) for clues, and you can re-try the connection with “ssh -vvv” to get verbose output for other clues. Often it’s a permissions error, so double check those file permissions.

This will now allow User_A on Workstation_A to write scripts that can call “ssh User_B@Server_B ” for remote workloads to be run, without getting prompted for a password.

This isn’t really the best way to handle this, though, so we’ll look at how to secure the private key with a passphrase next week, and still be able to do this without getting prompted on every connection.

Thanks for sticking with me on this. There’s plenty more to come!

Fun-Day Friday

Fridays will be “Fun-Day Friday,” at least for a while. I’ll talk about non-unix non-security stuff on this day, and today’s topic will be about one of my hobbies: Amateur Radio.

I’ve been a licensed HAM since November of 2008. In my initial test, I passed both my Technician and my General class exams, but failed the Extra when it was offered. I didn’t really expect to pass that one, but it gave me a pretty good idea of what kind of material was on it. I waited a few years before taking my Extra class exam again, because I wanted some time to get to know the hobby better before diving in that deep. I passed my Extra in 2012. On my to-do list is to get my Volunteer Examiner, but I keep getting life interruptions when I sit down to take it. It’s an open book, online test, so there’s no reason for me to not have it yet other than procrastination and interruptions in general. I’ll get it eventually.

In the grand scheme of things, I’ve enjoyed the hobby thus far. I don’t operate as much as other members of the local county club, and I get teased for this incessantly, but I enjoy the times I do operate, and that’s what matters.

HAM radio has a broad offering of ways to enjoy the hobby, so if you haven’t looked into it, you should. Some people enjoy rag chew on HF, others like to chase points in contests, and another set prefer to operate in only one mode or another, such as CW (morse code) or digital only modes where voice isn’t even carried.

The latest trend in our local club is DMR (digital mobile radio,) and the club just purchased a new DMR capable repeater for the locals to use. I picked up a new HT radio for the occasion, so I’ll probably cover a review on that next week. HAM radio will be one of the topics I bring up frequently on Fridays, but I’ll also cover some of my many other hobbies, such as knife throwing, playing music, book reviews, movie reviews, and so on. I eventually plan to spin these off onto their own blog space, but for now, I’ll share them here. Only on Fridays. You’ve now been forewarned.

Thanks for reading, and if you have any hobbies you love, post in the comments!

Hacker-Tool Hump-Day – USB Rubber Ducky

Every Wednesday (Hump Day) I intend to cover some topic related to hacking, penetration testing, making things do things they maybe weren’t meant to do in general, and so on. Eventually, Mondays will be more like what I envision “Blue Team” mindset should be, and Wednesdays will be more on what I envision “Red Team” mindset should look like. Since I’ve never had the opportunity to be on either kind of team at the place I’ve worked, I won’t say “this is how it is.” So if I use “Red Team/Blue Team” labels, understand this is from my perspective of how I understand them to be, from the outside looking in. I’ve mostly been in a Blue Team type of position (systems administration) but I’ve always tried to look at how to make things work in ways they maybe weren’t meant to work. This has helped me to be sure the stuff I run is run more securely. Any time I come up with a way to bypass, break, or otherwise torture a configuration, I look for ways to plug that hole.

Anyway, on to today’s topic. I’m a fan of Hak5 and the tools they create. Some people take a negative attitude toward them, but I think they’ve helped create some really nice tools for not only aggressive mentalities, but also for defensive ones.

Recently they held a sale in their Hak5 shop in celebration of one of their tools being shown on the show Mr. Robot. The tool in question is the USB Rubber Ducky. I bought one of these last year, and it has been useful from a SysAdmin perspective, so I thought I’d present it as my first topic. I bought one of their zippered kits during the sale, and it should be here by the time the next one of these posts goes live, so I might be covering other items from the kit at that point. I haven’t decided yet. The Ducky is available at the Hak5 shop, at this link:

USB Rubber Ducky Deluxe

So… the USB Rubber Ducky is a device that by default with stock firmware provided by the good folks at Hak5 simply emulates a USB keyboard device when you plug it in. It comes in a case with a swivel key chain piece, and you can take this apart to get to the Ducky internals. You need to do this, to do anything useful with it. The reason is simply this: the device has a microSD card that it reads a payload file from for the purposes of knowing what to “type.” The stock payload isn’t useful for anything more than amusement. You will need to be able to access this microSD card inside to put a useful payload on it. The default firmware doesn’t present a USB storage device for accessing the card, so you will have to take it apart to get that card out.

Take the swivel off, then use a large paperclip or a jewelers/glasses screwdriver to pop the two plastic pieces of the casing apart. There is a hole in the back opposite the USB connector. Just stick it in, give it a light twist, and it should come apart. The microSD card is sometimes stuck pretty good, so you may want a tool to catch the lip of the drive to get leverage for popping it out. It’s not glued in, I promise. The one that came with my first Ducky was 128 meg, so it’s not really useful for much more than keeping a payload on. The Ducky also came with a very small microSD card reader so that you can move your card to it and not have to buy a reader separately to get your payload onto the device. It’s formatted as vFAT, but I’m not sure if it can read the payload from an NTFS formatted card. I intend to test this.

There are alternative firmwares available. The only other one(s) I’ve tried are the Composite ones available as of the time of this writing, and upon initial testing, both seemed to behave identically as far as I could tell. The Composite firmware(s) present the Ducky as both a HID (keyboard) and a USB storage device. There used to be a version that would automatically launch the payload after a set amount of time, but the ones I tested seemed to have that functionality stripped. The version that has “4Cap” in the name lets you hit Caps Lock four times in a row to trigger the payload launch.

I tried the Composite firmware that had “Cap” in the name, expecting the payload to launch. I was disappointed. The only method that worked was actually pushing the button on the Ducky device itself to launch the payload.

I hopped onto the forums to ask about this, and after a few responses from forum members “winter_soldier” and “bored369” I was able to determine that the 4Cap hex file does indeed work as expected… on a Windows workstation. Another hearty “thanks” to winter_soldier and bored369 for the helpful responses on the forums. All of my testing had been done from Linux Mint, and only a press of the button on the Ducky would send the payload, there. I’ll research what the differences are to see if there’s a reason for this difference in behavior. The Mint laptop is a Lenovo ThinkPad W520, and the Windows laptop is a Lenovo Z50. I also noticed that if I do the caps lock key sequence to trigger this on Windows, it works the first time, but then starts flashing red on a second attempt without pulling it and plugging it back in. I can send the payload multiple times from one plug in if I use the button on the Ducky, instead. The button on the Ducky stops working if you use caps sequence to send the payload and it causes the flashing red light. I suspect this may be an error in the logic loop for handling the two different cases. Maybe the source is available for review. I’ll go looking to see if I can find it.

I also intend to try a case mod by drilling two holes into one side of the case. If I put a rod down in the hole to press the button, I won’t need to take the case apart, and if I use a small snip of strand from a novelty toy like the “glowing Christmas trees” you sometimes see on sale during the holidays, I can have a fiber to run down to the indicator light so I can see if it’s working (green) or if something went wrong (red) also without taking it apart.

So besides the actual USB Rubber Ducky, you also need to have a workstation to compile your payloads on, and update the firmware if needed. There are lots of examples of this on the Forums for Windows workstations, but I do my work from a Linux Mint workstation. I grabbed the encoder.jar from:
https://github.com/hak5darren/USB-Rubber-Ducky/tree/master/Encoder

This encoder lets you specify a keyboard layout along with the Ducky Script file. Speaking of the Ducky Script file, you need to write your payload in Ducky Script, then use the encoder to compile it into a binary file that the Ducky reads upon start up. The specific file name should be “inject.bin” for the Ducky to read it. The version of java used to call the encoder should be 7 or higher.

An example test Ducky Script might look like this:

DELAY 1000
STRING #this is a test of my first payload
ENTER

The “DELAY” is in milliseconds, so 1000 is “1 second.” You want a delay of 3 seconds or more with the stock firmware, most likely. Since I already had my Composite firmware in place, and it doesn’t launch until I push the button on the Ducky, I lowered my DELAY.

The “STRING” is the actual typed text that will get sent by the Ducky upon payload execution. Since the return key is a special key, the ENTER directive is needed to emulate that keypress. There are other meta keys that can be emulated. Review the documentation for what all are available, and visit the Forums if you’re having trouble with any of them. I seem to recall seeing a reported known bug about GUI vs. WINDOWS there a while back.

To encode a Ducky Script named “test.ducky” run the following:

java -jar encoder.jar -i ./test.duck -o inject.bin -l us

After you have your payload, copy it to the microSD card, slap the card back into the Ducky, and the next time you plug it in, it should type whatever you told it to type in your script.

If you are bold enough to try out one of the alternative firmwares, visit the Wiki page below for instructions. I used the dfu-programmer software provided as a link from that site from my Mint workstation, and my instructions for getting it compiled will follow.
https://github.com/hak5darren/USB-Rubber-Ducky/wiki/Flashing-ducky

I had to install two development libraries first, since configure complained about them being missing:
sudo apt-get install libusb-dev libusb-1.0.0-dev

I still had problems compiling after this, so a little digging around on the internet using the might of my Google-FU, I came across instructions for “fixing” some broken code.

After installing the missing libraries, unpack the tarball:
tar xvfz ./dfu-programmer-0.5.4.tar.gz
cd dfu-programmer-0.5.4

Modify the configure.ac file to change all references of “LDFLAGS” to “LIBS” instead:
sed -i.bak -e ‘s/LDFLAGS/LIBS/g’ ./configure.ac

Run configure:
./configure

Before running make, modify the Makefile in the src directory to move the -lusb flags from LDFLAGS down to LIBS instead. If this sed statement is confusing, ask in the comments and I’ll explain all the pieces, but today’s topic is the Ducky, not “sed.”

sed -i.bak -ne ‘/^LDFLAGS =/{h;s/LDFLAGS = \(.*\)/LIBS = \1/;x;s/LDFLAGS =.*/LDFLAGS =/};/^LIBS = .*/{x;};p’

Now that you’ve “fixed” the problem children, run “make” per normal, and you should have a file “dfu-programmer” in your src directory.

You can grab one of the .hex firmware files from the github wiki links above, and use the following steps to update it.

Take the Ducky out of its case if you haven’t already (or if you haven’t modified the case as I described earlier.)
In order to put the Ducky into “DFU” mode, hold the button down, then insert it into the USB port. If you aren’t holding it down while you insert it, it’ll be in payload mode and will try to type the payload script. Wait a few moments for the Ducky to be recognized by the system, then use the dfu-programmer software to update the firmware as below:

This step isn’t necessary, but the wiki mentions it, so if you want to make a backup of the existing firmware, do this:
sudo ./dfu-programmer at32uc3b1256 dump >dump.bin

Before flashing a new firmware hex file to the Ducky, you need to erase it:
sudo ./dfu-programmer at32uc3b1256 erase

To drop your new firmware in place, use the “flash” command as shown below. I tried both versions of the Composite firmware, so both of these are shown, but you only run it against ONE .hex file. Pick your firmware and flash only that.
sudo ./dfu-programmer at32uc3b1256 flash –suppress-bootloader-mem Composite_Duck_S003.hex
sudo ./dfu-programmer at32uc3b1256 flash –suppress-bootloader-mem Composite_Duck_4cap.hex

Finally, you have to remember to reset the Ducky after a firmware update in order to make it run payloads again.
sudo ./dfu-programmer at32uc3b1256 reset

That’s it. After the firmware is updated, you should be able to remove the Ducky and reinsert it to test functionality. Remember that if you used a firmware like the Composite versions above that requires a push of the Ducky button, you won’t see the payload until you push the button. I recommend using a test payload that doesn’t press a bunch of meta keys, first. Open a text editor, then insert the Ducky. Press the button if needed, but watch that the payload actually gets sent to the text document to verify it is functioning properly.

Hak5 touts this as a device for social engineering by virtue of curiosity. Put your malicious payload on the Ducky, then drop it in a parking lot and wait. Your payload might call up a bunch of commands to offload a Windows password dump to a remote web server, for example. So how does this tool have any kind of legitimate use in a SysAdmin tool kit?

The answer is simple. There are two scenarios where I have personally already found this tool useful from an everyday SysAdmin perspective. The first is when dealing with a console session to a remote server, and the console does not allow copy/paste. When you are dealing with a blind console that doesn’t allow pasting commands into the terminal, you have to be meticulous in your typing. If you know all of the steps you need to get the task accomplished, you can carefully craft that out into a Ducky Script, compile it, and then make that console your focused window before plugging the Ducky in for your “I’m pasting my commands anyway, so take that!” workload.

The other scenario has to do with a long ssh private key passphrase that needed to be stored into an IAM solution, and people were present who should not see this passphrase. Instead of having to start over multiple times, because the laws of nature demand that I mistype the phrase at least four times before getting it right, I was able to put the passphrase into a Ducky Script, take the Ducky to the conference room where the IAM software was being configured, set the focus on the passphrase field in the software, then plug the Ducky in. Or at least, I would have if I’d had the Ducky with me that day. Instead I had to type it out. Multiple times. Because things. Since that was the “development” environment, I’ll be doing the “production” environment with the Ducky next time.

Thanks for reading this, and if you have any ideas on other legitimate uses of the USB Rubber Ducky as an every day SysAdmin tool, share in the comments!

SSH – Start to Finish Architecture – The Basic Setup

Once upon a long time ago in a place not too far away, the internet was born. In its early days, people used basic protocols to interact with each other and share information. One batch of these protocols allowed people to connect remotely to another machine and either login for an interactive shell, or pass remote commands to be run for processing. These were telnet, rsh, and rlogin. There were also rcp and ftp for file transfers. All of these were great until people started needing to lock their doors at night because the neighbors got too nosey. Security became a concern, and SSH was born.

Today, most people think of OpenSSH as the defacto software for SSH services. It contains both a server component (sshd), an sftp subsystem (to replace FTP) and a few clients for handling remote logins and file transfers (ssh and scp.) The truth is, there are plenty of packages to choose from, for both server side and client side communication with these new protocols. I may cover some of these, but this series is going to start with OpenSSH as a focus simply because it is the most popular at this time. We’ll start with a basic setup, and then progress through more complex setups to cover various needs based on scale of the operation needing this kind of access arrangement.

An initial explanation of how this works is a user on workstation A wants to be able to log in remotely to server B to run some kind of work load. Remember that every SSH session is composed of pieces of information at its most basic level. A user (User_A) on a workstation or server (Workstation_A) wants to connect remotely to a server (Server_B) as some user (User_B) to run a work load. We will use these labels moving forward to describe how to configure the software for each kind of scenario we want to cover. Eventually there will be a third system involved (either LDAP, or a Certificate Authority, or both) so we’ll toss in an _C label when we get to that. For now, just remember “_A” will represent “coming from” and “_B” will represent “going to” for directionality of each session flow.

So before User_A can connect to Server_B with SSH, Server_B needs to have an SSH service running. The software can be installed from source, or it can be installed from whatever package management system your server OS supports. RPM/yum and DPKG/apt-get are common for RedHat and Debian based systems, respectively. AIX has installp/RPM. HP-UX has swinstall. Just use the appropriate software for the system you are supporting, but understand that some of the more advanced features will require newer versions of OpenSSH, so if you try something and it doesn’t work, check your server version.

Now that OpenSSH sshd has been installed via whatever means you chose, User_A needs client software. It should be the same deal, here. Use whatever package management software you choose. There is an OpenSSH client via the Cygwin project for Windows for the “follow along” when your workstation is Windows, but other clients may work better for most of these scenarios. PuTTy and its derivatives, WinSCP, MobaXTerm are all excellent Windows clients. However, when we get to certificate based connections, these may not be sufficient. Just install OpenSSH client for now on whatever platform you’re coming from, and let’s move on.

Now Workstation_A and Server_B have the software needed to make a connection. If User_A wants to connect to Server_B as User_B, a “User_B” account will need to exist on the server, or at least be a user that can authenticate (LDAP or similar.) For now we will assume local accounts are in use. So to make a connection, User_A on Workstation_A will need to know the password for User_B on Server_B. Then typing the following will allow a connection to be established (assuming the account is active and not locked.)

As User_A from a terminal prompt:
ssh User_B@Server_B

It’s that simple. This is almost not worth covering, I know. However, every post in this series will take a step toward making things work more smoothly from an operations and security perspective, so stick with me. Next week we’ll cover setting up public/private key pairs for authentication without a password.