Tuesday, April 4, 2017

Malware Analysis Course on Pluralsight!

Since 2010, I have been running my Introduction to Malware Analysis course at various conferences and organizations, and have taught over 200 students. I've heard from many of my former students that they've used what they learned in the course to help them successfully combat malware in their organizations - some have even gone into the malware analysis field themselves!

I only teach my course once or twice a year; for the past few years it has only been at DerbyCon. The problem with that is the material sits unused for most of the year, with no one gaining benefit from it.

So, when I was approached by the great people at Pluralsight to record my course and put it online, I jumped at the chance. It was a long process to do, but well worth it. This week, the course was released under the name Malware Analysis Fundamentals.

Malware Analysis Fundamentals is an online version of my Intro to Malware Analysis course. The course takes you from knowing nothing about malware analysis to being able to manually analyze malware in a safe and consistent manner. Like my regular course, you still analyze real malware using the techniques used by incident responders everywhere.

The one thing that I found out while creating this was that its not possible to put everything from my sit-down course into the online version. If I did, the course would have been at least double the length and no one wants to sit through that! Therefore, Malware Analysis Fundamentals gets to the essence of the material and teaches the fundamentals needed to get the job done.

There is also more to come. I already have plans for other MA courses at Pluralsight, branching into more advanced techniques and courses on analyzing alternative forms of malware.

I hope you enjoy the course and I look forward to hearing everyone's thoughts on the course!

Tuesday, November 15, 2016

Malicious DNS Namespace Collisions

Over the last few weeks, I've noticed a problem come up again in multiple places that I first saw many years ago and apparently is still very common - DNS Namespace Collisions. DNS namespace collisions occur when a private domain name is able to be resolved on the public Internet; whether it is intentional or not. ICANN has a lot of information on this if you are looking for a deep dive on the subject; instead I will be focusing on the potential security issues.

The Issue

Let's start with an example. Suppose you own the Internet domain example.org. This is your Internet presence - all your emails are @example.org, your web servers are in this domain, even your Active Directory domain is corp.example.org. All is well in the world.

When configuring hosts in your organization, one of the things you will do is set up your DNS suffix search list. This is the list of domains your systems will add to a host name if they can't initially resolve it. In our scenario, your DNS suffix search list is example.org and corp.example.org. So, if a host attempts to resolve mailserver, they might also try mailserver.example.org and mailserver.corp.example.org.

And let's also suppose that you follow good security practices and have split DNS so no one on the Internet can resolve your internal host names. You also do not allow internal hosts to directly resolve Internet host names.

Your happy domain.

Any issues so far? Nope. The computer gods are smiling upon us.

As your organization expands, you find the need to add a new internal domain so you choose example.com. Uh oh! You don't own that domain on the Internet, but you'll only be using it on the internal network. Not an issue, right? No, it is a problem.

The issue lies in that you do not own the domain example.com but are using it internally; this is a DNS name collision. The issue comes into play soon as a host accesses the Internet directly (from home, a client's network, etc.). When this happens, they won't be able to resolve hosts with the suffix example.org or corp.example.org - but as soon as they try to resolve with the suffix example.com (which you don't own) they will succeed.

So how is this an issue? In the best case, it isn't. If your hosts try to resolve something that example.com can't resolve then aside from some information leakage things should be OK. However, what if they try to resolve something that does exist in example.com and then try to start using it?

On the Internet, only example.com will resolve.
For example, our hosts are on the Internet and are trying to the internal mail server host name. The only one that is resolvable is mailserver.example.com, which we don't own. They then start to send emails - your private, internal-only emails - through a server you don't own. See the issue now?

This only happens if that host name already exists in the external domain, right? Wrong.

If DNS wildcards are used, now all of a sudden any host name is being resolved beyond your control and your hosts are sending data to potentially malicious servers. Think of how easy it would be to gain information on your organization or compromise your hosts if I could tell your hosts where their proxy, active directory, or mail servers were when they were outside your organization. And how would you ever know?

This is not a theoretical attack. In the last few weeks I have found multiple organizations where this is occurring. Specifically, they are using domains internally that they do not own, their hosts go outside their organization and are resolving these domains to malicious IP addresses.

And there are organizations that are squatting on multiple domains (including obviously internal ones) and setting up wildcard DNS to point them to their own IPs. For what purpose? I don't know, but I suspect it can't be good.

Detection, Prevention, and Response

So how can you detect this? A few ways:
  1. Create a list from your DNS for all domains being used by your clients related to your organization. Make sure you own all those domains. If not, what IPs do they resolve to? Consider switching from them. (This is also a good threat hunting technique!)
  2. Windows hosts like to resolve wpad/wpad.dat when browsing. The DNS search suffix tends to get added to that, so look for any HTTP requests to the Internet for wpad.dat, then look for what domains the requests are to. Even if they are not your own hosts (e.g. consultants), you should still be concerned as they could be used as a pivot point into your network.
By the way, wpad.dat is not something you want your hosts doing this with.

Prevention of this is actually pretty easy - just make sure you own any domain you use, or use ones that do not have Internet TLDs. (However, from my research there may be issues on this with some versions of Windows.)

If you do find this happening on your network, I would suggest immediately looking to see what your hosts are resolving, what data is going out, and more importantly, what is coming back in.

I would also recommend blocking the IP addresses and external domains on your Internet devices to prevent internal hosts from accessing them.

In the end, this is a big problem that I don't think many realize is going on. Fortunately, its fairly easy to detect and start investigating. Doing it now will probably save you a lot of hurt in the long run.

Wednesday, April 29, 2015


MASTIFF has been a pet project of mine for about two years now. While it has not progressed as far as I would have liked, we made a major announcement this week.

On Monday, a free online interface to MASTIFF was released at https://mastiff-online.korelogic.com/. This interface allows anyone to upload files, have MASTIFF process the files, and see the results generated.

If you are not familiar with MASTIFF, it is an open source framework for automating the static analysis of malware. It essentially will determine the type of file you are analyzing and only run the static analysis techniques for that file against it. This allows fast extraction of data the analyst can then examine.

The online interface was created for two reasons:

1. When you start processing a number of different file types, the pre-requisites start to get cumbersome and difficult to install. The online interface alleviates this by allowing you to analyze files without installing everything.

2. Our #1 request was a web interface to the system. While the interface used on MASTIFF Online is not open source itself, we are hoping this will give users what they want.

If anyone has any questions/comments/suggestions to MASTIFF or the site, please let me know!

Monday, February 10, 2014

Installing Yara into IDA Pro 64-bit Linux

tl;dr Install a 32-bit VM, compile Yara, copy files over. See link below for files to just install.

Last Friday, pnX posted that he updated his awesome IDA plug-in, IDAScope, to include Yara support. This means that you can now run Yara sigs against files you are reversing to help in the analysis process.

After I installed the new version of IDAScope into IDA Pro, however, I received errors stating that Yara could not be imported. I thought this was odd as I had Yara installed on my system, until I remembered how IDA works on a 64-bit Linux system.

The following is based off my observations and experiences. If I am incorrect on this, please forgive me and let me know in the comments.

IDA is a 32-bit program. Even the 64-bit version of IDA is compiled as a 32-bit program.

$ file idaq idaq64
idaq:   ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), BuildID[sha1]=0xcb635dd38de5c73f050de37a0f2e492688b3ab9a, stripped
idaq64: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), BuildID[sha1]=0x1f03dcff4bfd776b23df71c8d9d471fb63b0bf48, stripped

This causes a number of interesting issues on 64-bit Linux systems, especially with Python. Hex Rays has gotten the majority of these fixed in the default install so you don't worry about them, and the way it does this with Python is by allowing you to install a bundled Python into the IDA Pro directory. (There are other ways, but I have not done them.) This gives you a working "out of the box" product.

This also means that when you want to install a new Python library and use it in IDA, you have to install it into the IDA's bundled Python directory as well. If this is a pure Python module, then no problem. Just copy and it should work. Yara is different.

Since Yara compiles as a 64-bit library on a 64-bit system, and yara-python does the same, we can't just install it directly into the IDA Python directory. If you do, you'll receive errors that IDA is unable to load a 64-bit module.

In order to get Yara working, we'll need to compile it as a 32-bit library. The easiest way, IMO, to do this is to load a 32-bit Linux system into a VM, compile Yara, then copy the files into your IDA installation. I did this in a Debian 6.0.3 and it worked without a problem. Just to be safe, make sure you are using a system with Python 2.7 as well since that is what IDA bundles.

There are two files you will need: the Yara library libyara.so.0 and the Yara Python library yara.so (located in the Python dist-packages directory after installation). Follow the instructions to compile and install Yara in your 32-bit VM, and copy the files onto your 64-bit system. libyara.so.0 goes into your base IDA install directory, and yara.so goes into the python directory underneath that.

After you do that, Yara-python will be installed and will work great!

Don't want to go through all the trouble of installing a 32-bit VM, compiling, and copying? I don't blame you. I uploaded the version I compiled to my Google Drive here.

yara-ida-libs.tgz (SHA256: 38674b584adf3932e5cd1cafbd0bb288b7db3302304a83041bad9295472aa064)

Just untar this into your base install dir for IDA and you should be good to go.

Hex Rays has published instructions on how to install Python packages from Pip on a 64-bit system. I recommend checking them out. This time, my way just felt easier.

Wednesday, September 4, 2013

Installing BinDiff on Linux Mint 14

I recently upgraded my system to Linux Mint 14 and went about re-installing all my software. When I got to Zynamics/Google BinDiff, I found I had an issue:
$ sudo dpkg -i bindiff401-debian50-amd64.deb

Selecting previously unselected package bindiff.
Unpacking bindiff (from bindiff401-debian50-amd64.deb) ...
dpkg: dependency problems prevent configuration of bindiff:
 bindiff depends on sun-java6-jre; however:
  Package sun-java6-jre is not installed.

Unfortunately, BinDiff requires sun-java6-jre, which is not in the Linux Mint repository, nor any other repository I could find. I could circumvent this by installing BinDiff by using the --ignore-depends=sun-java6-jre option to dpkg. However, every time I went to install updates I would get an error message that BinDiff was broken, and be prompted to uninstall it before I could continue.

However, I found a work-around - create a dummy package named sun-java6-jre using the tool equivs. (There are some docs out there on this, but I was unable to find a non-Google cached copy, so here was what I did.)

Linux Mint has equivs in its repository, so if its not already installed, apt-get it.

Next, run equivs-control sun-java6-jre and this will create a file named sun-java6-jre that you will need to modify.

At minimum, you'll need to uncomment and/or fill out the following fields:
  • Package
  • Version
  • Maintainer
I also filled out the description fields so I would remember what it was.

After the file is modifoed, run equivs-build sun-java6-jre and you should see something similar to below:
$ equivs-build sun-java6-jre
dpkg-deb: building package `sun-java6-jre' in `../sun-java6-jre_6.0_all.deb'.

The package has been created.
Attention, the package has been created in the current directory,
not in ".." as indicated by the message above!
Once that has successfully completed, you should have a sun-java6-jre_6.0_all.deb file in your directory. If that failed, you probably forgot to modify one of the fields in the file.

Finally, just dpkg -i the new deb file and BinDiff, and you should be ready to go!
$ sudo dpkg -i sun-java6-jre_6.0_all.deb
Selecting previously unselected package sun-java6-jre.
(Reading database ... 237677 files and directories currently installed.)
Unpacking sun-java6-jre (from sun-java6-jre_6.0_all.deb) ...
Setting up sun-java6-jre (6.0) ...
$ sudo dpkg -i bindiff401-debian50-amd64.deb
Selecting previously unselected package bindiff.
(Reading database ... 237681 files and directories currently installed.)
Unpacking bindiff (from bindiff401-debian50-amd64.deb) ...
bindiff license has already been accepted
Setting up bindiff (4.0.1) ...

Then you are good to go!

Sunday, May 19, 2013

My Take on the City of Akron Hack

On Thursday, May 16, 2013, a Turkish hacking group called Turkish Ajan hacked into the City of Akron and released a number of files that contain personal information on a number of Akron citizens. According to the city, the attackers were able to gain access into some internal systems where they obtained tax information.

The news has died down on this for the moment, but from the information that has been released, there are some things we can infer:

1. The attackers compromised the city's public website. From the errors that were being displayed on the site, information that has been released from the city, and the way this group works, it was likely through SQL injection (although this has not been specifically stated yet).

2. The attackers compromised the city's internal systems and obtained access to tax systems. It is unknown if they were able to do from the city's public website, through the tax paying system, or some other server. In any case, this appears to be where the attackers got the files they posted.

3. Around 25K people are affected.

4. The FBI is involved and was called in quickly after the compromise was discovered. IMO, this is good.

Any additional information on what happened is pretty much speculation. Trust me, I've been speculating alot and have a pretty good idea of what happened, but I have no proof. Hopefully whoever is doing the forensics for the city will have their findings released at some point. As an Akron citizen, and tax payer, I want to know this information.

However, there is one thing that I think needs brought up. Why was this information stored unencrypted? If it was encrypted, how did the attackers obtain access to the keys to decrypt it?

The information that was released contains social security numbers of both the taxpayer and their spouse, and credit card numbers. According to PCI standards (and my understanding), the credit card numbers should have been encrypted. The federal government is required to comply with PCI, what about the city of Akron government?

As for the SSNs, I don't know of any specific regulations that requires that information to be encrypted (please let me know if there is), but I can't imagine that there is any reason it shouldn't be. I have a feeling there are at least 25,000 people who agree with me.

One final item of note. The press has been getting quotes from Deputy Mayor Rick Merolla. With all due respect sir, shut up. I can only imagine your IT and information security people are cringing whenever they read your quotes pertaining to the security of the city of Akron systems.

I'm sure you are very smart, but its obvious you are not familiar with information security. Quotes such as "Our systems are all, all our virus protection, intrusion protection systems, all of our virus software is still up to date so we are still not sure how they got in" show this. Let those performing the investigation or the talented IT personnel you employ speak on these things.

If you like, I am personally offering to give you a training course on information security, hackers, and how attacks take place. This will at least give you an idea on why the things you have been quoted as saying are cringe-worthy.

Friday, April 19, 2013

MASTIFF 0.6.0 Released!

The latest version of MASTIFF, 0.6.0, has just been released! Run over to the download site and grab the latest version!

The official changelog is located here, but the major improvements are described below.

Upgrading MASTIFF to the latest version is easy. You can follow this process:
  1. Download and install pydeep.
  2. Download MASTIFF 0.6.0 and untar it.
  3. Run "make test" to ensure you are not missing any dependencies.
  4. Run "sudo make install" to install the latest version.
  5. Copy the analysis plug-ins (the plugins directory in the tarball) to your location of choice and ensure the config file is pointing to that directory.
  6. Add any new options to your MASTIFF config file. The easiest way may be to use sdiff.


MASTIFF now has a queueing system so multiple files can be analyzed by the framework. To utilize this, give MASTIFF a directory instead of a file to analyze. It will find all files in that directory and its subdirectories, add them to the queue, and begin processing.

The queue is maintained within the MASTIFF database. So, if you have to stop MASTIFF in the middle of its run, it will begin re-processing the queue when its restarted. Some additional options have been added to allow you to work with the queue:
  • --clear-queue: This will clear the current queue.
  • --ignore-queue: This will ignore the queue and just process the file you give it.
Analysis plug-ins are also taking advantage of the queue. The pdf-parser and ZipExtract plug-ins have a new option ("feedback") which allow you to feed files from the plug-ins back into the queue for processing. For example, the ZipExtract plug-in will add all files that were extracted from the archive into the queue for processing.

Fuzzy Hashing

Fuzzy hashing is not something new within MASTIFF. However, we have changed the Python library used for it. Previously, we used pyssdeep but found that there were a number of stability issues with it on OSX and when processing large amounts of files.

Therefore, we have switched to pydeep (https://github.com/kbandla/pydeep). Our testing has shown it to be much more stable thus far.


There was some confusion on which Python libmagic libraries to use when installing MASTIFF. To help alleviate some of that, the framework has been modified to use two different libmagic libraries:
If either library is installed, MASTIFF will utilize them.

Other Changes

A number of other bug fixes and improvements have been made. Please see the changelog file for a complete list.

As always, if you have any questions, please email mastiff-project@korelogic.com.

We have alot of great things coming down the pipe for MASTIFF, but if you have any suggestions, enhancements or plug-ins, let us know!