ArduinoJson 6.8.0: more with less!

I promised you many times that the breaking changes in ArduinoJson 6 were behind us and that no more should come. Well… I didn’t hold my promise but don’t worry! There are just a few breaking changes in ArduinoJson 6.8.0; the majority of the improvements comes without breaking your code.

Mandatory argument for DynamicJsonDocument’s constructor

In ArduinoJson 6.7, I removed the automatic extension of the DynamicJsonDocument for the following reasons:

• it wasted a lot of heap memory
• it caused heap fragmentation
• it was in complete contradiction with the fixed-allocation / no-fragmentation philosophy of the library
• it forced me to keep virtual functions
• it pretended to help the users but was indeed a ticking bomb 💣

So, starting with version 6.7, the DynamicJsonDocument has a fixed capacity, just like the StaticJsonDocument.

However, I made the mistake of preserving the default constructor of DynamicJsonDocument, the one without any parameter. I thought that a default capacity would be OK for most users. It turned out to be a useless and confusing idea.

So, starting from version 6.8, you have to specify the capacity of the DynamicJsonDocument to its constructor:

// in ArduinoJson 6.7
DynamicJsonDocument doc;

// in ArduinoJson 6.8
DynamicJsonDocument doc(2048);

I know, it fills like a regression, but really, it makes things more explicit.

Nesting limit

ArduinoJson’s parser contains a function that calls itself every time a new array or object begins. As a consequence, this recursive function uses a quantity of stack memory proportional to the “nesting” (or “depth”) of the input document.

This recursive function presents a security risk because an attacker could craft a special JSON document to overflow the stack. To protect against this potential attack, ArduinoJson always had a setting to limit the nesting level of the input document.

In ArduinoJson 5, the nesting limit was specified as an additional argument to JsonBuffer::parseObject():

// in ArduinoJson 5
JsonObject& obj = jb.parseObject(input, 20);

It was not possible to take the same approach in ArduinoJson 6, because the additional argument specifies the size of the input. Up till now, a member variable of JsonDocument specified the nesting limit:

// in ArduinoJson 6.7
doc.nestingLimit = 20;
deserializeJson(doc, input)

However, it was awkward to use the JsonDocument to hold a setting for the parser. In ArduinoJson 6.8, I moved the nesting limit back to an optional argument, except that it must have the type DeserializationOption::NestingLimit to solve the ambiguity with the other integer parameter:

// in ArduinoJson 6.8
deserializeJson(doc, input, DeserializationOption::NestingLimit(20));

I think we agree that this syntax is an improvement over the previous ones.

Using a JsonDocument like a JsonVariant

Here is the deserialization example from ArduinoJson 6.7:

StaticJsonDocument<200> doc;
deserializeJson(doc, input);

JsonObject root = doc.as<JsonObject>();

const char* sensor = root["sensor"];
long time = root["time"];

As you see, before extracting the values from the JsonDocument, we had to call JsonDocument::as<T>().

Starting with version 6.8, you can directly extract the values from the JsonDocument:

StaticJsonDocument<200> doc;
deserializeJson(doc, input);

const char* sensor = doc["sensor"];
long time = doc["time"];

Of course, you can still call JsonDocument::as<T>() if you need, but it’s not mandatory anymore.

Automatic conversion of JsonDocument

We just saw that we could get rid of JsonDocument::as<T>() when deserializing a document. What about JsonDocument::to<T>() then? Good question.

Here is how we perform JSON serialization in ArduinoJson 6.7:

StaticJsonDocument<200> doc;

JsonObject root = doc.to<JsonObject>();

root["sensor"] = "gps";
root["time"] = 1351824120;

As you probably guessed, in version 6.8, we can get rid of JsonDocument::to<T>() as well:

StaticJsonDocument<200> doc;

doc["sensor"] = "gps";
doc["time"] = 1351824120;

The JsonDocument automatically converts to the appropriate type (array or object) depending on the way you use it. In this example, we use the document as an object, so the JsonDocument automatically becomes an object.

Of course, the automatic conversion only occurs on the first use. Once the JsonDocument contains something, it keeps its type until you call JsonDocument::clear() or JsonDocument::to<T>().

Constructing JsonDocuments

In ArduinoJson 6.7, it was possible to copy-construct a JsonDocument from another. Now, it’s also possible to construct a JsonDocument from a JsonArray, a JsonObject, or a JsonVariant.

This feature can be useful when you want to make a copy of a part of the document. For example, imagine you have to read a large JSON configuration file, but you are only interested in the “network” member; you can now write:

DynamicJsonDocument getNetworkConfig() {
  DynamicJsonDocument doc(2048);
  deserializeJson(doc, CONFIG);
  return doc["network"];
}

In this example, the returned DynamicJsonDocument chooses its capacity based on the memory usage of the “network” object. In other words, the DynamicJsonDocument capacity fits exactly the size of the document.

Let’s see another example. Imagine you want to pass a part of a large configuration object to a class. The constructor of this class could have a JsonObject parameter that it would copy into a member JsonDocument:

class NetworkService {
  public:
    NetworkService(JsonObject obj) : config_(obj) {}

    void initialize();
    void shutdown();
    // etc

  private:
    DynamicJsonDocument config_;
};

These examples use a DynamicJsonDocument, but you can do the same with a StaticJsonDocument.

Assigning JsonDocuments

As we saw above, when you construct a DynamicJsonDocument from a JsonArray, a JsonObject, or a JsonVariant, it uses the minimum possible capacity.

Most of the time, that’s what you want because memory usage is as small as possible. However, it means that the DynamicJsonDocument is already full, so you cannot add more values.

To work around this limitation, you need to construct the DynamicJsonDocument in a separate statement so you can specify the capacity, and then assign the JsonArray, the JsonObject, or the JsonVariant to the DynamicJsonDocument:

DynamicJsonDocument doc(1024);
doc = theObjectToCopy;

When you assign (not construct) like that, the DynamicJsonDocument preserves it capacity, except if it’s too small. Indeed, if the DynamicJsonDocument’s capacity is insufficient, the memory pool is reallocated to match the required size.

New “pretty” JSON serializer

From the beginning, the “pretty” JSON serializer, the one that produces indented JSON documents, has always been a dark area of the library. It was implemented as a proxy Print instance that watched for incoming characters to decide how to indent the output.

I finally had the opportunity to rewrite this piece of code and make it simpler. The result is a much smaller executable, as you can see on the chart below.

This chart shows the evolution of the size of the compiled executable for the two main examples: JsonParserExample.ino and JsonGeneratorExample.ino. As you can see in the highlighted area, the size of JsonGeneratorExample.ino drops at version 6.8.

Also, there is a new compile-time option to change the indentation characters: ARDUINOJSON_TAB. By default, ARDUINOJSON_TAB is a string composed of two white spaces.

Conclusion

I only covered the significant changes in this article; there are also many small modifications. If you want to see the complete list, please read the changelog for version 6.8.0

I hope you’ll love this new version. It’s been quite a lot of work to get there. Now, I’ll focus my energy on the new edition of the book.

As usual, if you have any question or comment, feel free to open a GitHub issue. 👋