ArduinoJson supports two techniques to parse very large documents: “filtering” and “deserialization in chunks.” These techniques allow you to read a document that would otherwise not fit in the RAM of your microcontroller.

You can use an external PSRAM with ArduinoJson, but you must define a custom allocator.

Filtering

Since ArduinoJson 6.15, deserializeJson() supports a filtering feature that reduces the memory consumption by ignoring irrelevant fields from the input document. It allows you to keep only the fields that matter in the JsonDocument.

To use this feature, you must create an ancillary JsonDocument that will serve as a filter. This document must contain the value true as a placeholder for each field you want to keep; every other field will be discarded. For arrays, only create one element in the filter document; it will serve as the filter for every element of the original document.

For example, suppose your input looks like:

{
  "list": [
    {"temperature":21.2,"humidity":68.9,"pressure":1003},
    {"temperature":19.7,"humidity":62.1,"pressure":1007},
    {"temperature":18.6,"humidity":59.8,"pressure":1009}
   ] 
}

If you only want to keep the temperature field, you must create the following filter:

{
  "list": [
    {"temperature":true}
   ] 
}

Let’s put this example into code. Here is how we can create the filter document:

StaticJsonDocument<64> filter;
filter["list"][0]["temperature"] = true;

Once the filter document is ready, you must wrap it with DeserializationOption::Filter, and pass it to deserializeJson():

deserializeJson(doc, input, DeserializationOption::Filter(filter));

After executing this line, doc will contains the following document:

{
  "list": [
    {"temperature":21.2},
    {"temperature":19.7},
    {"temperature":18.6}
   ] 
}

In this case, filtering reduced the memory consumption in half.

The OpenWeatherMap case-study in Mastering ArduinoJson shows how to use this technique in a real project.

Deserialization in chunks

Instead of parsing the whole JSON document in one shot, you can parse only a part of it and repeat the operation.

For example, suppose you are interested in the “characters” array from the following JSON document:

{
   "characters": [
        {"name":"Homer Simpson"},
        {"name":"Marge Simpson"},
        {"name":"Bart Simpson"},
        {"name":"Lisa Simpson"},
        // ...
    ]
}

If you try to deserialize the JSON document, it will probably not fit in the RAM of the microcontroller. However, you could easily deserialize each JSON object (i.e., each “character”), one by one.

One cool feature of ArduinoJson is that, when it parses an object from a Stream, it stops reading as soon as it sees the closing brace (}). The same is true with arrays: it stops reading as soon as it sees the closing bracket (]).

This feature allows you to parse streams in chunks: you just need to call deserializeJson() in a loop. Of course, you must skip the commas (,) between the objects.

Here is how this technique works:

  1. Jump to the beginning of the array.
  2. Call deserializeJson()
  3. Read the next character; if it’s a comma, go to 2.

For step 1, you can use Stream::find(); for step 2, you can use Stream::findUntil()

Here is how you can implement this technique:

input.find("\"characters\":[");
do {
    deserializeJson(doc, input);
    // ...
} while (input.findUntil(",","]"));

The Reddit case-study in Mastering ArduinoJson shows how to use this technique in a real project.