ArduinoJson 7.1: MessagePack upgrade

I just published ArduinoJson 7.1, which provides several improvements for MessagePack.

I know I initially promised to work on other stuff, but my plans were disrupted by a Pull Request from @Sanae6 who implemented the initial support for MessagePack binary.

Binary format

The most important addition is the support for MessagePack’s binary format. You’ve been asking for this feature for five years, and it’s finally here!

Here is how you can insert a binary value in a MessagePack document:

char buffer[] = {1, 2, 3};

JsonDocument doc;
doc['data'] = MsgPackBinary(buffer, sizeof(buffer));
serializeMsgPack(doc, output);   // 81 A4 65 61 74 61 C4 03 01 02 03

Similarly, here is how you can read a binary value from a MessagePack document:

JsonDocument doc;
deserializeMsgPack(doc, input);

MsgPackBinary data = doc['data'];
// or: auto data = doc['data'].as<MsgPackBinary>();

const void* buffer = data.data();
size_t size = data.size();

As you can see, everything revolves around the MsgPackBinary class. Internally, ArduinoJson uses the same storage as serialized() values, so this new feature has no impact on existing code (see charts down below).

MsgPackBinary doesn’t hold a copy of the data, so you must ensure that the corresponding buffer remains in memory when you use it. For example, when you extract a binary value from a MessagePack document, you must ensure that the JsonDocument outlives the MsgPackBinary and that the value is not altered.

Like strings, binary values are subject to size restrictions that depend on the library configuration. Please check the documentation for details.

Extension format

I didn’t want to make you wait another five years for MessagePack extensions, so I included it in this release.

MessagePack extensions in ArduinoJson work the same as binary values, except you must use MsgPackExtension instead of MsgPackBinary.

For example, here is how you can insert an extension value in a MessagePack document:

char buffer[] = {1, 2, 3};

JsonDocument doc;
doc['data'] = MsgPackExtension(4, buffer, sizeof(buffer));
serializeMsgPack(doc, output);   // 81 A4 65 61 74 61 C7 03 04 01 02 03

And here is how you can read an extension value from a MessagePack document:

JsonDocument doc;
deserializeMsgPack(doc, input);

MsgPackExtension data = doc['data'];
// or: auto data = doc['data'].as<MsgPackExtension>();

int8_t type = data.type();
const char* buffer = data.data();
size_t size = data.size();

Other MessagePack improvements

I rewrote a significant part of the MessagePack deserializer to reduce the size and increase the speed. Despite the new features (binaries and extensions), the MessagePack example is significantly smaller than version 7.0, as you can see in the graphs below:

I also improved the compliance with the MessagePack specification by reading 64-bit integers even when ARDUINOJSON_USE_LONG_LONG is set to 0. If the value fits in a 32-bit integer, it will be stored in the JsonDocument; otherwise, it’s stored as a null, as in the previous versions.

This is a compliance improvement because the MessagePack specification doesn’t impose using the smallest viable type to store an integer:

If an object can be represented in multiple possible output formats, serializers SHOULD use the format which represents the data in the smallest number of bytes. Source: MessagePack specification

Final words

As usual, ArduinoJson 7.1 comes with other little improvements that you probably won’t notice. I invite you to check the changelog if you’re interested.

For the next release, I plan to optimize memory consumption. Hopefully, I’ll keep my promise this time 😉