MessagePack is a binary serialization format. There are lots of open source implementations of this protocol on various languages including C/C++. It’s good to do something good in new year. For example, it can be a little contribution to an open source project. Let’s check quickly if the implementation on C/C++ has any memory corruption issues. One of the best ways is of course fuzzing.

We are going to use American Fuzzy Lop (AFL) because it’s just a great tool. First, we need to clone the repo, and build msgpack with AFL. We are going to use AddressSanitizer (ASan) — it’s another great tool. If you have not heard about it — it’s a fast and cool memory checker which can detect issues like read/write buffer overflows, use-after-free, etc.

Fuzzing MessagePack with American Fuzzy Lop

Here are commands which builds msgpack instrumented with AFL and ASan:

It’s going to install msgpack binaries to /path/to/build you specified in -DCMAKE_INSTALL_PREFIX option. You need to specify a real path here. You might notice that it uses -m32 option which tells GCC to build a 32-bit binaries. At first, I tried to run fuzzing with with 64-bit binaries, but AFL and ASan complained about memory, but it worked fine with 32-bit.

Once we built msgpack, we need to create a couple of files with serialized data for fuzzing. I used the following program for that:

Then, we need to compile it with AFL as well, and finally run it to generate serialized data:

Next, we need to create a simple program which uses msgpack API for deserialization. Finally, we are going to run this program while fuzzing. I wrote two programs. First one uses C API:

And second one uses C++ API (it’s much shorter, by the way):

Now we are good to go and start fuzzing. We should create “in” directory for input data for AFL, and copy all three *.ser files to there. Here is a command which runs fuzzing for msgpack implementation on C (just replace “deserialize” with “deserialize_cpp” to run it for C++ implementation):

I was not interested in memory leak, so I passed “detect_leaks=0” option to AddressSanitizer. But someone may try to check msgpack for memory leaks.

I ran fuzzing on my old laptop, and it took a couple of days to find issues. So, the lesson here is that you have to be patient when you fuzz software. Or, you need faster hardware.

And now the most exciting part.

Bugs in MessagePack

Fuzzing discovered a couple of integer overflows which lead to buffer overflows.

One problem was in template_callback_ext() function. While parsing corrupted data, it’s possible that zero is passed to “l” parameter of template_callback_ext() function. This leads to assigning a negative value to o->via.ext.size which is implicitly being converted to a huge positive number.

It may then lead to reading o->via.ext.ptr out of its bounds, for example, in msgpack_object_bin_print() function:

Depending on where this function is called, and what “out” is, it may potentially lead to reading confidential data, and sending it to untrusted party. It’s a read buffer overflow, similar to Heartbleed.

Next issue was in template_callback_array() and template_callback_map() functions. An integer overflow may occur there while calculating an amount of memory to be allocated. It may potentially lead to allocating zero bytes to o->

As a result, it may lead to a buffer overflow, for example, in template_callback_map_item() and template_callback_array_item() functions:

Finally, there were a couple of similar integer overflows in include/msgpack/v2/unpack.hpp

My patches were accepted with minor modifications.

Thanks to Takatoshi Kondo who integrated the patches quickly!

Originally published at on January 5, 2017.

I write about Java, security, electronics and DIY

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store