In March I posted the following article on CDW blog
Revisiting NIST recommendations provides some essential techniques for protecting your organization’s accounts
Security, Computer Networking, Photography, Electronics, and Technology
In March I posted the following article on CDW blog
Revisiting NIST recommendations provides some essential techniques for protecting your organization’s accounts
Back on May 18th, I attended the inaugural BsidesNH event. It was a fantastic one-day event. The day started pretty early for me driving down from Maine arriving at Southern NH University. I arrived to pick up the fantastic badge made out of an old 3.5″ disk. After grabbing some coffee and a snack I settled into the auditorium and for a day of great talks. There were a few that stood out to me from the day that I will talk about.
The second talk of the day was Ghost in the Shell: When AppSec Goes Wrong by Tony Martin. Tony first talked about covered some basics of web application security. He framed these issues around the research he has done into various NAS devices and vulnerabilities he has discovered. Including the ability to create shadow users that have administrative access to devices but are not visible through the administrative interfaces of the device.
After lunch was Chinese and Russian Hacking Communities presented by Winnona DeSombre and Dan Byrnes, Intelligence Analyst from Recorded Future. They covered operations and cultures of Chinese and Russian underground groups. This was a very entertaining presentation and a summary of the information contained in the report: Thieves and Geeks: Russian and Chinese Hacking Communities.
The second to last talk of the day was Hunting for Lateral Movement: Offense, Defense, and Corgis presented by Ryan Nolette. He covered the ways attackers move around and infiltrate further into a network…Corgies. A great quote that stuck with me from his talk was: “If you teach an analyst how to think they will punch above their weight.” I feel this quote not only applies to security analysts but all levels of IT professionals.
BsidesNH was a well run and enjoyable event and a great addition to the Security events in New England. Thanks to all of the organizers and sponsors. I look forward to attending next year!
During my OSCP studies, I realized I needed a more efficient system for cracking password hashes. The screaming CPU fans and high CPU usage became a problem. I first tried using hashcat and the GPU on my MacBook Pro in OS X. There are some bugs and problems with hashcat on OS X that would make it crash in the middle of cracking a hash. Also, I was not interested in investing a server with a bunch of GPUs, the high costs to do this would outweigh the amount of time I need the system. All of this lead me to do a little research and found the instructions in the following link to build an AWS instance for password cracking.
Since that post was created there have been some changes to the offerings in AWS EC2 leading me write this post.
If you wish to skip ahead I have created scripts to automate the processes in the rest of this post. They are both in my
github and can be downloaded at the following links.https://github.com/suidroot/AWSScripts/blob/master/aws-ec2-create-kracker.sh
https://github.com/suidroot/AWSScripts/blob/master/configure-kracker.sh
For the rest of the article I will cover some of the instance options in EC2, installation of the needed Linux packages, the basic setup of Hashcat, running Hashcat, and finally monitoring and benchmarks of an EC2 instance.
There are many options for EC2 instances, they have a huge range
I found the g3 instances to be the more cost effective tier. For my testing I opted to use the g3.4xlarge tier. Next to choose the AMI image, appropriate the appropriate operating system.
There are two options that are I tested hashcat on they are both Ubuntu based. I’m sure there are many other available options that will work too, but I am familiar with Ubuntu systems. The first option is a standard Ubuntu image, there is nothing special about this image and it requires configuration to add the GPU drivers and a little more work.
The next option is a Deep Learning image, this image is preconfigured with the GPU drivers and was originally designed for machine learning applications. I found the the pre-configuration allowed for me skip a few steps in building out a new system.
Once you have the instance deployed there are a few steps to get the Instance prepared for hashcat, the steps are a little bit different between a Standard and a Deep Learning Ubuntu instance.
An apt
cronjob may already be running and you will have to wait it out.
This script will install all the required packages and the Nvidia GPU drivers on a vanilla Ubuntu installation.
#!/bin/bash
# mostly copied from: https://medium.com/@iraklis/running-hashcat-v4-0-0-in-amazons-aws-new-p3-16xlarge-instance-e8fab4541e9b
#
sudo apt-get update -yq
sudo apt-get install -yq build-essential linux-headers-$(uname -r) unzip p7zip-full linux-image-extra-virtual
sudo apt-get install -yq ocl-icd-libopencl1 opencl-headers clinfo
#sudo apt-get install -yq libhwloc-plugins libhwloc5 libltdl7 libpciaccess0 libpocl2 libpocl2-common ocl-icd-libopencl1 pocl-opencl-icd
sudo apt-get install -yq python3-pip
pip3 install psutil
sudo touch /etc/modprobe.d/blacklist-nouveau.conf
sudo bash -c "echo 'blacklist nouveau' >> /etc/modprobe.d/blacklist-nouveau.conf"
sudo bash -c "echo 'blacklist lbm-nouveau' >> /etc/modprobe.d/blacklist-nouveau.conf"
sudo bash -c "echo 'options nouveau modeset=0' >> /etc/modprobe.d/blacklist-nouveau.conf"
sudo bash -c "echo 'alias nouveau off' >> /etc/modprobe.d/blacklist-nouveau.conf"
sudo bash -c "echo 'alias lbm-nouveau off' >> /etc/modprobe.d/blacklist-nouveau.conf"
sudo touch /etc/modprobe.d/nouveau-kms.conf
sudo bash -c "echo 'options nouveau modeset=0' >> /etc/modprobe.d/nouveau-kms.conf"
sudo update-initramfs -u
sudo reboot
### Install nVidia Drivers
wget http://us.download.nvidia.com/tesla/410.104/NVIDIA-Linux-x86_64-410.104.run
sudo /bin/bash NVIDIA-Linux-x86_64-410.104.run --ui=none --no-questions --silent -X
In comparison the previous script there is a much simpler script to prepare the Deep Learning instance. The main focus is installing the needed archive extraction tools.
#!/bin/bash
sudo apt update
sudo apt upgrade
sudo apt install clinfo unzip p7zip-full
sudo apt install build-essential linux-headers-$(uname -r) # Optional
sudo apt-get install -yq python3-pip
pip3 install psutil
Now we need to download and extract the star of the show Hashcat. The link in the wget below points to the the most recent version as of writing however you might want to check to see if there is a more recent version at the main site: https://hashcat.net/hashcat/
wget https://hashcat.net/files/hashcat-5.1.0.7z
7z x hashcat-5.1.0.7z
You will need some wordlists for hashcat to use to crack passwords, he commands listed are for some wordlists I like to use when cracking. You should however add whichever lists are your favories.
mkdir ~/wordlists
git clone https://github.com/danielmiessler/SecLists.git ~/wordlists/seclists
wget -nH http://downloads.skullsecurity.org/passwords/rockyou.txt.bz2 -O ~/wordlists/rockyou.txt.bz2
cd ~/wordlists
bunzip2 ./rockyou.txt.bz2
cd ~
Now it is time to run hashcat and crack some passwords. When running hashcat I had the best performance with the arguments-O -w 3
. Below is an example command line I’ve used
./hashcat-5.1.0/hashcat64.bin --username -m 1800 ./megashadow256.txt wordlists/rockyou.txt -r hashcat-5.1.0/rules/best64.rule -O -w 3
nvidia-smi
ubuntu@ip-172-31-17-6:~$ sudo nvidia-smi
Fri Apr 26 14:43:49 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104 Driver Version: 410.104 CUDA Version: 10.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla M60 Off | 00000000:00:1E.0 Off | 0 |
| N/A 37C P0 42W / 150W | 0MiB / 7618MiB | 97% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
This example shows a GPU being used by hashcat.
ubuntu@ip-172-31-17-6:~$ sudo nvidia-smi
Fri Apr 26 14:44:44 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104 Driver Version: 410.104 CUDA Version: 10.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla M60 Off | 00000000:00:1E.0 Off | 0 |
| N/A 46C P0 141W / 150W | 828MiB / 7618MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 11739 C ./hashcat-5.1.0/hashcat64.bin 817MiB |
+-----------------------------------------------------------------------------+
Finally here is a benchmark I ran on a g3.4xlarge instance. This instance type contains 1 GPU. These results give an idea of performance for this AWS EC2 instance type.
ubuntu@ip-172-31-17-6:~$ ./hashcat-5.1.0/hashcat64.bin -O -w 3 -b
hashcat (v5.1.0) starting in benchmark mode...
* Device #2: Not a native Intel OpenCL runtime. Expect massive speed loss.
You can use --force to override, but do not report related errors.
nvmlDeviceGetFanSpeed(): Not Supported
OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: Tesla M60, 1904/7618 MB allocatable, 16MCU
OpenCL Platform #2: The pocl project
====================================
* Device #2: pthread-Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz, skipped.
Benchmark relevant options:
===========================
* --optimized-kernel-enable
* --workload-profile=3
Hashmode: 0 - MD5
Speed.#1.........: 11611.6 MH/s (90.74ms) @ Accel:512 Loops:512 Thr:256 Vec:4
Hashmode: 100 - SHA1
Speed.#1.........: 4050.2 MH/s (65.01ms) @ Accel:512 Loops:128 Thr:256 Vec:2
Hashmode: 1400 - SHA2-256
Speed.#1.........: 1444.5 MH/s (91.98ms) @ Accel:256 Loops:128 Thr:256 Vec:1
Hashmode: 1700 - SHA2-512
Speed.#1.........: 499.4 MH/s (66.78ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Hashmode: 2500 - WPA-EAPOL-PBKDF2 (Iterations: 4096)
Speed.#1.........: 189.8 kH/s (42.76ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Hashmode: 1000 - NTLM
Speed.#1.........: 18678.1 MH/s (56.58ms) @ Accel:512 Loops:512 Thr:256 Vec:2
Hashmode: 3000 - LM
Speed.#1.........: 10529.6 MH/s (50.60ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Hashmode: 5500 - NetNTLMv1 / NetNTLMv1+ESS
Speed.#1.........: 10650.8 MH/s (49.60ms) @ Accel:512 Loops:256 Thr:256 Vec:1
Hashmode: 5600 - NetNTLMv2
Speed.#1.........: 829.3 MH/s (80.24ms) @ Accel:256 Loops:64 Thr:256 Vec:1
Hashmode: 1500 - descrypt, DES (Unix), Traditional DES
Speed.#1.........: 442.0 MH/s (37.81ms) @ Accel:4 Loops:1024 Thr:256 Vec:1
Hashmode: 500 - md5crypt, MD5 (Unix), Cisco-IOS $1$ (MD5) (Iterations: 1000)
Speed.#1.........: 4209.1 kH/s (51.39ms) @ Accel:1024 Loops:500 Thr:32 Vec:1
Hashmode: 3200 - bcrypt $2*$, Blowfish (Unix) (Iterations: 32)
Speed.#1.........: 7572 H/s (33.02ms) @ Accel:16 Loops:4 Thr:8 Vec:1
Hashmode: 1800 - sha512crypt $6$, SHA512 (Unix) (Iterations: 5000)
Speed.#1.........: 76958 H/s (83.99ms) @ Accel:512 Loops:128 Thr:32 Vec:1
Hashmode: 7500 - Kerberos 5 AS-REQ Pre-Auth etype 23
Speed.#1.........: 149.4 MH/s (56.00ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Hashmode: 13100 - Kerberos 5 TGS-REP etype 23
Speed.#1.........: 152.1 MH/s (55.00ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Hashmode: 15300 - DPAPI masterkey file v1 (Iterations: 23999)
Speed.#1.........: 32703 H/s (84.02ms) @ Accel:256 Loops:64 Thr:256 Vec:1
Hashmode: 15900 - DPAPI masterkey file v2 (Iterations: 7999)
Speed.#1.........: 21692 H/s (96.24ms) @ Accel:256 Loops:128 Thr:32 Vec:1
Hashmode: 7100 - macOS v10.8+ (PBKDF2-SHA512) (Iterations: 35000)
Speed.#1.........: 5940 H/s (40.09ms) @ Accel:64 Loops:32 Thr:256 Vec:1
Hashmode: 11600 - 7-Zip (Iterations: 524288)
Speed.#1.........: 4522 H/s (55.87ms) @ Accel:256 Loops:128 Thr:256 Vec:1
Hashmode: 12500 - RAR3-hp (Iterations: 262144)
Speed.#1.........: 18001 H/s (56.74ms) @ Accel:4 Loops:16384 Thr:256 Vec:1
Hashmode: 13000 - RAR5 (Iterations: 32767)
Speed.#1.........: 18135 H/s (55.93ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Hashmode: 6211 - TrueCrypt PBKDF2-HMAC-RIPEMD160 + XTS 512 bit (Iterations: 2000)
Speed.#1.........: 121.7 kH/s (59.39ms) @ Accel:128 Loops:32 Thr:256 Vec:1
Hashmode: 13400 - KeePass 1 (AES/Twofish) and KeePass 2 (AES) (Iterations: 6000)
Speed.#1.........: 68380 H/s (158.89ms) @ Accel:512 Loops:256 Thr:32 Vec:1
Hashmode: 6800 - LastPass + LastPass sniffed (Iterations: 500)
Speed.#1.........: 1088.7 kH/s (48.51ms) @ Accel:128 Loops:62 Thr:256 Vec:1
Hashmode: 11300 - Bitcoin/Litecoin wallet.dat (Iterations: 199999)
Speed.#1.........: 2107 H/s (78.97ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Started: Fri Apr 26 14:36:56 2019
Stopped: Fri Apr 26 14:42:03 2019
If you’ve made it this far congratulation and happy cracking!