We’ve done several assessments of late where we needed to (ab)use MQ services. We’ve detailed our experiences and results below. Built a tool, punch-q, so you don’t have to go through the same, and included some info for blue teams, including an osquery extension.

Depending on how old a version you are working with, or which document you read online, you might know IBM’s Message Queue solution as MQSeries, Webshere MQ or IBM MQ. The latter being the latest name it got around 2014 with the release of version 8. Nonetheless, in the last few months I have come across a number of distinct instances of MQ, each used in their own interesting ways for arbitrary systems integrations. Be it for simple messages being passed around or to facilitate file transfers, MQ played a significant role when it came to the overall business processes these companies had. In order to help me understand the technology better, I discovered some prior research by the folks at MWR, with a very informative talk done at Defcon 15 called MQ Jumping. A subsequent white paper was released and is definitely worth a read.

mq threat modelling

When threat modelling MQ installations and the roles they played within organisations, in all cases being able to inject messages into arbitrary queues would have either started some process or transaction, bypassed a specific check performed as part of a business process or outright committed a transaction. Additionally, being able to read the messages themselves would also have aided in gaining significant insights into the structure and formatting of messages (not to mention the sensitivity of the contents some of the messages might have). These risks were in turn quickly translated into real, demonstrable financial losses for the affected organisations, primarily as a result of a concerning and often misunderstood attack surface within typical MQ environments.

connecting to mq

Now before we can get to trying to hack some message queues, we need to be able to connect to a queue manager. More specifically, depending on what tooling you use and what you want to do, you may need a few bits of information to interact with a queue and its messages. Those typically are:

• IP Address / Hostname & Port
• Queue Manager Name (retrievable using Wireshark as discussed in the Defcon talk, but not always a requirement)
• Channel name to use
• Username and password for the channel (hopefully optional ;p)

To be able to get and put messages to and from a queue, you need to connect to a channel that acts like a conduit to a message queue. Using something like IBM MQ Explorer, it is possible to connect to a queue manager and interact with it to perform administrative tasks. Depending on the level of access obtained, one would be able to list other channels, queues and services as well as get and put messages into a queue. With a list of known Server-connection queue names and some commonly used usernames and passwords, one could try and guess a few combinations to access a queue manager. Success here really depends on how the queue manager is configured or how good your enumeration phase was. ;)

In my adventures with MQ on Linux and AIX thus far, unless it’s a brand spanking new installation of MQ, chances are good you may find an unauthenticated Server-connection channel. This may often be the case for one of the default Server-connection channels such as SYSTEM.ADMIN.SVRCONN. In my experience, it seems like this is most common on queue managers that are older than version 7.1, or were upgraded to version 7.1 (or newer) which introduced channel authentication records (CHLAUTH), but disables it when upgrading. A fresh installation post version 7.1 however will have a few default CHLAUTH rules configured, ultimately requiring you to at least authenticate to a channel. Of course, these my be disabled by an administrator at a later stage or misconfigured and be taken advantage of. Or, if a new installation is done from scratch but the MQ configuration output from a dmpmqcfg is run with runmqsc, the old configuration is applied which might leave it in a vulnerable state again. Finally, if you manage to gain shell access to an MQ server, running dmpmqcfg yourself might provide you with some insights into channel names and their respective CHLAUTH configurations.

Nonetheless, depending on the level of access obtained to an MQ instance, you may be in a position to read and write messages to a message queue. If you have administrative access to the queue manager, well then you can also do interesting things like execute operating system commands using MQ services. Reverse shells anyone? Services are not the only way to ultimately get command execution, but it sure is an easy way.

manipulating queues

Reading messages is definitely interesting when attacking business processes and MQ Explorer lets you do this, together with the ability to peek into a queue’s current messages. Simply right click the queue you are interested in and click “Browse Messages”:

Easy enough, right?

Errr, no. If there were messages in the queue at the time of you hitting that “Browse” menu item, then sure, we would be able to see them here. More often than not though, messages flow through these queues incredibly fast (no really, it’s pretty freaking fast) which doesn’t give you a lot of time to play with when you are looking for a specific message. This was problem number one I faced, but more on that later. If you do manage to open the message browser at the right time (or messages have been piling up because whatever is supposed to pop them off the queue is not doing that), the message browser may look something like this:

MQ Explorer also lets you put test messages into a queue. This is great if you want to inject your fabricated message into a queue to either start that new business process, or end it off as if all of the business checks before it passed. Again, simply right click the queue, select “Put Test Message” and enter the payload you want to inject:

This is super useful and easy to use, however, it’s not awesome to use if you have to use non-ascii printable characters as part of your message. Yes, some systems have fixed length, custom payloads as part of their messages with weird as heck message structures. This was problem number two.

sharpening the axe

You probably know enough now to start asking uncomfortable questions at your organisation and can stop reading. But we are not done. Remember the two primary problems faced? Being able to read messages on the queue and easily being able to place new ones on the queue? Well, to try and make my life easier and not need MQ Explorer I searched for scripts / code that I could use to speed things up. I discovered a python library called pymqi which effectively wraps a bunch of IBM MQ C client libraries and provides a python like interface to MQ. The Github project contains many usage examples which were perfect for my needs. After a pretty painful experience of getting the MQ client and SDK installed from IBM’s website (IBM provide RPM’s, I was using a Kali box, I needed yum, gross…), I finally got pymqi to import in a python REPL.

introducing punch-q

Following along with some examples from the pymqi documentation, I managed to get to a point where I could dump channel / queue names and start and stop services (for code execution via MQ). A few assessments (and some frustrating header file lookups later), I took all of these separate scripts I had and bundled them together, giving it an utterly unoriginal name of punch-q. With punch-q as it stands today, one can:

• Enumerate basic MQ information.
• Brute force Channels and Credentials (to discover those unauthenticated Channels).
• Show channels / queues (with some interesting information such as queue depths).
• Sniff messages without removing the original message from the queue.
• Put new messages onto a queue either as a command argument or sourced from a file.
• Execute arbitrary OS commands via MQ services.

The source code for punch-q is available on Github here: https://github.com/sensepost/punch-q. It does depend on the IBM MQ  client libraries (for pymqi) so installation can be a bit of a pain, unfortunately.

channel enumeration

Taking a look at some examples of punch-q in action, the first would be a simple channel name enumeration running against a dockerized MQ installation. A word list with some common channel names is included as part of punch-q, but if you provided a hostname as the target MQ instance to attack, punch-q will generate a few more channel names to give a try based purely on conventions observed in the wild:

An important observation to make here is that only an IP address and port was used for the enumeration to take place; not a queue manager name. Also in this example, none of the Server-connection channels were unauthenticated, but should you have valid credentials, this enumeration phase will clearly indicate success. Another example of where one may add credentials to test across all of the channels may look as follows:

This time round, one can see I have access to the DEV.ADMIN.SVRCONN channel with the credentials provided. The credentials used were added to the docker.yml configuration file, but punch-q will also let you “override” configurations specified in the YAML file via command-line arguments. In this case with --username and --password.

credentials brute force

A simple credential brute forcer also exists in punch-q:

Admittedly, there are better options for credential brute forcing such as SSH, especially since you will most probably be interested in the mqm user which would also be an operating system user. Nevertheless, maybe SSH has fail2ban or auth failures via SSH is monitored, so here you go.

sniff / save / put messages

A number of punch-q commands support interacting with messages. This is really where the biggest impact is. Many commands can save copies of messages from a queue, which includes the command that sniffs messages live as they enter a message queue.

Sometimes queues get backed up for various reasons, and it may be interesting to see what the messages that are on the queue look like offline, especially if you want to perform some good ‘ol fashioned grep on the contents. So, with punch-q you can simply run the messages save command and grab copies of all of the messages that are in a specific queue:

The sniff command has the --store flag that would also dump copies of messages as they are also printed  live to the screen.

One can also put new message into a queue too (much like the “Put test message” feature in MQ Explorer). A message could be provided either as a simple string via the --source-string flag, or via a filename with the --source-file flag which will have punch-q read the target files contents and push that onto the target message queue:

command execution

Ahhh yes, command execution. In punch-q, this is implemented using MQ services which quite literally provides a start command field to run stuff (thanks IBM):

While the command reverse command is merely a convenience method for a reverse shell, it is also possible to fire off once off commands using the command execute  command. Using this method though you are performing blind command execution, so keep that in mind.

never forget the blue team

Apart from the obvious configuration fixes needed to harden your MQ environment, monitoring is obviously a good thing too. As expected, all of the channel and credential brute forcing stuff mentioned in this post is logged. Now, I really don’t think the logs MQ generates are great (multiline, freeform logs anyone?), but it’s at least a source of information that you can use. Error logs are typically stored in /var/mqm/qmgrs/<QUEUE MANAGER NAME/errors and seem to provide enough intel on what is going wrong. Both the credential and channel name brute forces are clearly visible in the MQ error logs.

The codes AMQ5534E and AMQ9999E were used simply because the log files were examined while the brute forces were run using punch-q. A lot of other interesting information is also logged that warrants more investigation. However, given the two simple examples above, one would already be in a position to ask: “Why would systems that are built to integrate with queue and channel names configured to be static be getting the channel names / authentication information wrong?”.

Services started on MQ are also logged. Surprisingly (or not?) in the error log. I am not sure why, but at least its logged and can also be monitored for.

As for “Who is connected to my queue manager?” and “What are they doing?”, well, that seems like a harder question to answer. So far, I have not been able to find a way to enable such event logging that could be parsed using common tools. It is possible to use runmqsc and issue the display conn(*) where(channel NE '') APPLTAG CHANNEL CONNAME CONNOPTS command as an example to get a list of the currently connected clients.

While not great, it does provide a way to maybe script the query, parsing the resultant output that one could generate. Depending on your environment, these connections would probably not be changing a lot, so any differential output in who is connected to your queue manager may be interesting.

osquery monitoring

Scripting the connected clients MQ query and parsing it is an effective way of accomplishing this, but, I wanted to give writing a custom osquery table a shot too. With osquery, the differential output approach is assumed, meaning that any changes to the connected clients will end up in the final log file. It is possible to write custom table extensions for osquery with bindings available both in Python and in Golang. I chose Golang to get a single, compiled extension I could easily move around. The Golang bindings project page provide just enough information to get you started, and with a minimal amount of effort I managed to get osquery to load my custom table extension.

The harder part was getting data out of MQ. In fact, parsing the output of the display conn(*) query absolutely sucked. I managed to get it to work OK, meaning that with the extension loaded one could run select * from mq_clients; to get a list of the currently connected MQ clients to a queue manager.

The source code for this table plugin can be found in the punch-q repository here and should be easily to compile once the osquery-go dependencies have been resolved.

The custom table plugin could also easily be extended to include other bits of information. There is a downside to this approach though. Cases where applications may very quickly connect to the queue manager, perform some work and disconnect may not be immediately detected using a script or the osquery table approach. However, something like the sniff command punch-q provides would definitely be detected this way. YMMV.

summary

In a number of recent assessments we had great success with targeting IBM MQ instances that perform various middleware tasks. Many of these were as a result of a misunderstood attack surface which ultimately resulted in the ability to demonstrate interesting attacks on business processes which many times exist to prevent fraud or perform some compliance function. To help with the interactions with IBM MQ, a small toolkit called punch-q was built and is available for here: https://github.com/sensepost/punch-q.