ArduinoJson 6.18: custom converters

Seven months have passed since the last major revision of ArduinoJson. It was about time! Today, we get a new feature that will help you structure your code more cleanly: custom converters. As you’ll see, I had to make some breaking changes to let that happen. Don’t worry; the fixes are straightforward.

Custom converters

You’ve been asking for this feature for years. Most of the time, it was in the form of “How can I store a time in JSON?” or “How can I parse a date from JSON?”. Let’s see how we used to do this and how custom converters improve the code.

Time without custom converters

Suppose you have a time and date stored in the (almost-)standard tm structure:

// get current date and time
time_t now = time(NULL);
tm timeinfo = *gmtime(&now);

// timeinfo now contains something like
// {
//   .tm_sec = 4,
//   .tm_min = 13,
//   .tm_hour = 13,
//   .tm_mday = 4,
//   .tm_mon = 4,
//   .tm_year = 121,
//   .tm_wday = 2,
//   .tm_yday = 123,
//   .tm_isdst = 0
// }

To store this time in a JSON document, we usually do something like so:

char buf[32];
strftime(buf, sizeof(buf), "%FT%TZ", &timeinfo);
doc["time"] = buf;

These three lines create the following JSON document:

{"time":"2021-05-04T13:13:04Z"}

We just learned how to insert a tm struct in a JSON document; now, let’s see how we can extract one from an existing JSON document:

tm timeinfo;
strptime(doc["time"], "%FT%TZ", &timeinfo);

We now have a way to serialize and deserialize tm structures. Everything works as expected, but if we need to manipulate many time values, our program will be cluttered with calls to strftime(), strptime(), obscure time formatting strings, and temporary buffers. Let’s see how ArduinoJson 6.18 allows us to clean up our code.

Time with custom converters

Custom converters allow ArduinoJson to work with types that it doesn’t natively support. To do so, we need to define functions with well-known signatures, and ArduinoJson will automatically call these functions.

In our case, we want to add support for the tm structure. For the serialization side, we need to define convertToJson(), like so:

bool convertToJson(const tm& src, JsonVariant dst) {
  char buf[32];
  strftime(buf, sizeof(buf), "%FT%TZ", &src);
  return dst.set(buf);
}

Starting with ArduinoJson 6.18.3, convertToJson() can return void

As you can see, this is a simple function that receives a source parameter of the type we want to support and a destination parameter of type JsonVariant. All we had to do is wrap the three lines from the previous section into a function. This simple change allows us to write this:

doc["time"] = timeinfo;

Behind the scenes, ArduinoJson will call convertToJson() and store the string "2021-05-04T13:13:04Z" in the JsonDocument.

In the same way, we can define convertFromJson() which teaches ArduinoJson how to extract a tm structure from a JsonDocument:

void convertFromJson(JsonVariantConst src, tm& dst) {
  strptime(src.as<const char*>(), "%FT%TZ", &dst);
}

Thanks to this function, we can write:

timeinfo = doc["time"];

As you can see, custom converters can significantly improve the readability of the code by hiding the dull conversion code into dedicated functions and moving it out of the main flow. This feature is especially handy for complex (nested) structures, as we’ll see in the next edition of Mastering ArduinoJson.

How does it work?

If you’re new to C++, you might be wondering how ArduinoJson can call a function that you declared out of the library. It uses a language feature called Argument Dependent Lookup (ADL for short): when resolving the overload for a function (like convertToJson()), the compiler must consider the functions declared in the same namespace as the arguments (in our case, tm and convertToJson() are in the global namespace).

Therefore, if you want to support a type defined in a namespace, you must declare convertToJson() and convertFromJson() in the same namespace.

You might also be wondering why I used the tm structure as an example instead of time_t. Unfortunately, time_t is not an actual type: it’s (most often) a typedef (i.e., an alias) of long. Because ArduinoJson already supports long, we cannot define custom converters for time_t.

What about is<T>()?

To get a complete support for your custom type in ArduinoJson, you can define canConvertFromJson(), like so:

bool canConvertFromJson(JsonVarianConst src, const tm&) {
  return src.is<const char*>();  // check that the value is a string
}

This function allows us to do this:

if (doc["time"].is<tm>()) ...

The second parameter of canConvertFromJson() is required to trigger ADL but must not be used by the function.

Supporting non-default-constructible types

The three functions we just saw work in most cases, but they require the type to be default-constructible.

For example, the following class is not default-constructible:

class Complex {
  double _real, _imag;
 public:
  explicit Complex(double r, double i) : _real(r), _imag(i) {}
  double real() const { return _real; }
  double imag() const { return _imag; }
};

To support this class in ArduinoJson, we must use another strategy. Instead of defining three free functions, we must specialize ARDUINOJSON_NAMESPACE::Converter<T>:

namespace ARDUINOJSON_NAMESPACE {
template <>
struct Converter<Complex> {
  static bool toJson(const Complex& src, JsonVariant dst) {
    dst["real"] = src.real();
    dst["imag"] = src.imag();
    return true;
  }

  static Complex fromJson(JsonVariantConst src) {
    return Complex(src["real"], src["imag"]);
  }

  static bool checkJson(JsonVariantConst src) {
    return src["real"].is<double>() && src["imag"].is<double>();
  }
};
}

Starting with ArduinoJson 6.18.3, toJson() can return void

Breaking changes

Adding the custom converters feature forced me to break some strange behaviors of earlier versions. Starting from 6.18, JsonVariant::as<T>() always returns T, which means that the following line will break:

Serial.println(doc["msg"].as<char*>());

You must replace this line with:

Serial.println(doc["msg"].as<const char*>());

There are other situations where this could occur, but as<char*>() is the most common. I added a deprecation warning for this particular case to help you progressively upgrade your code.

DeserializationError::NotSupported removed

Here is a change I have wanted to make for a long time!
I always thought that ArduinoJson’s parser was a real jerk: if there was anything in the document that it didn’t support, it discarded the whole and returned NotSupported.

In 6.18, I removed this stupid error code. Instead, deserializeJson() ignores the Unicode escape sequences when ARDUINOJSON_DECODE_UNICODE is 0 (the default is 1 since 6.16), and deserializeMsgPack() replaces unsupported values with nulls.

Support for naked char removed

JSON doesn’t have a character value type: it’s either a string or a number. Support for char in ArduinoJson has always been ambiguous: is it a signed integer, an unsigned integer, or a string with just one character? Implicit conversion to char also caused issues when constructing a std::string from a JsonVariant.

For these reasons, I decided to remove the support for char, which means the following lines will break:

char age = doc["age"];
doc["age"] = age;

You must replace these lines with:

int8_t age = doc["age"];
doc["age"] = age;

Here too, a deprecation warning will show up.

Support for Printable

ArduinoJson 6.18 embeds a converter for Printable, so you can now write:

doc["ip"] = Ethernet::localIP();

Const-aware is<T>()

I changed is<T>() to support const values. For example, JsonVariantConst::is<JsonArray>() could return true, but now it will always return false. Instead, you must call JsonVariantConst::is<JsonArrayConst>().

Code size

As usual, I always kept an eye on code size, and you can see that ArduinoJson didn’t grow and even shrunk a little:

Conclusion

From what I saw, the new custom converters modify the way we use ArduinoJson, so I’ll soon publish a new edition of Mastering ArduinoJson to reflect this change. Don’t worry: if you purchase the ebook now, you’ll get the latest edition for free, and I’ll list the changes in the introduction.

While we’re talking about the book, I must admit that I’m having some financial difficulties and the small amount of money I earn from the book cannot justify the time I spend on the libraries and technical support. However, I love doing this, and I don’t want to stop; instead, I’m seriously considering taking money from sponsors. If your company wants to support good open-source software and have its logo displayed on arduinojson.org, please contact me. I can also include some premium technical support in the package; let me know if you’re interested.

That’s all for today! I’ll see you in the next one.
Now, go upgrade your code!