ArduinoJson 6.19: NUL, JsonString, and new defaults

Happy New Year! 🎉

I just released a new version of the library; let’s see what’s in it.

Support for NUL

This feature drove most of the changes in this release. It allows a JSON string value to contain a NUL character, like so:

{"values":"one\u0000two\u0000three}

With previous versions of ArduinoJson, if you tried to read this string, you would get a truncated result:

const char* values = doc["values"];  // "one"

As you can see, the returned string stops at the first NUL character.

Of course, it’s impossible to fix this problem for const char* because this form of string assumes that the result is zero-terminated. Also, we cannot fix this for Arduino’s String class because it doesn’t support NUL.

Now, ArduinoJson allows NUL in strings, but you need to use a string type that supports them: std::string, std::string_view, or JsonString (see next section).

Also, NUL is currently only allowed in values, not keys.

Is this feature helpful? Honestly, it will impact a small minority of our users, but it allows new usages such as binary data encoded in strings. As we’ll see, the effect on code size is reasonably small.

Promotion of JsonString

JsonString is the type returned by JsonPair::key() as in the following example:

for (JsonPair p : doc.as<JsonObject>()) {
  JsonString key = p.key();
  Serial.println(key.c_str());
}

In previous versions of ArduinoJson, JsonString was only used in this context, but now you can use it as a regular string value, such as:

doc["hello"] = JsonString("world");

Of course, as<JsonString>() and is<JsonString>() also work.

As explained in the previous section, JsonString supports NUL characters so that you can write:

doc["values"] = JsonString("one\0two\0three", 13);

Notice that you have to specify the size of the string (13 characters in this case) because we cannot rely on the zero-terminator. The above line would generate the following JSON document:

{"values":"one\u0000two\u0000three}

As you can see, serializeJson() translates NULs into \u0000.

I also implemented the safe bool idiom in JsonString, so you can write if (str) to test if the string is null.

Default configuration

ArduinoJson 6.19 introduces several changes in the default compile-time settings to simplify the library and remove some pitfalls.

First, the (undocumented) ARDUINOJSON_EMBEDDED_MODE setting was removed. It allowed selecting between the “embedded” mode, which reduces memory usage, or the non-embedded mode, which favors large data types. Now, it doesn’t matter if you target an Arduino Nano or a super-computer; ArduinoJson always assumes you’re in the former “embedded” mode. The advantage is that you now have the same data types on your unit tests running on your PC and on the final target, which should eliminate some surprises. Dependent settings, such as ARDUINOJSON_DEFAULT_NESTING_LIMIT, must now be set individually.

ARDUINOJSON_USE_DOUBLE is now 1 by default. This change has no impact on memory consumption because double values were padded. However, it slightly increases code size on most platforms.

ARDUINOJSON_USE_LONG_LONG is now 1 on 32-bit platforms. As ARDUINOJSON_USE_DOUBLE, this change doesn’t impact memory usage but slightly affects code size.

ARDUINOJSON_ENABLE_PROGMEM is now set to 1 as soon as ARDUINO is defined. This change allowed me to remove the systematic #include Arduino.h statement, which polluted the global namespace even if you don’t use any Arduino.h feature. From what I can tell, every platform that defines ARDUINO supports or emulates PROGMEM, so this change should go unnoticed.

BasicJsonDocument copy-construct and copy-assign

Because DynamicJsonDocument derives from BasicJsonDocument, everything we see in this section applies to DynamicJsonDocument too.

In the previous versions of ArduinoJson, the copy constructor and copy assignment operator of BasicJsonDocument implemented an optimization that consisted in allocating a memory pool smaller than the original. Instead of using capacity() to determine the size of the memory pool, it used memoryUsage().

This little trick allowed saving memory in a few situations but exposed some problematic inconsistencies with conventional copy semantics. For example, it posed a problem when you stored BasicJsonDocuments in a std::vector on C++03 (where move semantics are not supported).

Improvement in copyArray()

copyArray() is a utility function that copies values between a regular C array and a JsonArray in both directions.

It used to support 2-dimensional arrays but now supports n-dimensional arrays thanks to a new recursive implementation.

It also supports char[][] as a 1-dimensional array of string (since version 6.18, ArduinoJson doesn’t treat char as an integral type anymore).

Code size

As usual, I concentrated most of my effort on keeping the library lightweight.

The graph below shows the evolution of the size of the examples on AVR.

As you can see, the impact is neglectable on AVR. Now let’s see a second graph for ESP8266.

Here, we can see that all examples grew by roughly 600 bytes each. This increase is mainly due to the upgrade from float to double and from long to long long. I don’t think this will be a problem for anyone since the ESP8266 has between 512KB and 4MB of Flash memory. Still, you can go back to the original sizes by setting ARDUINOJSON_USE_DOUBLE and ARDUINOJSON_USE_LONG_LONG to 0.

That’s all for today. Let me know in the GitHub issues if you have any problems with this release.

Don’t forget that you can support the project by purchasing my book or by sponsoring me.