How to determine the capacity of the JsonDocument

ArduinoJson 6 uses a fixed memory allocation strategy. This is cool because it allows your program to be smaller, faster, and not prone to heap fragmentation. Unfortunately, it requires a little more work from your side.

When you create a JsonDocument, you must specify the memory capacity of the document. The larger the capacity is, the more complex the document can be, but the more memory it consumes.

This page explains how to size your JsonDocument correctly, depending on your use case.

If this is too much of a headache, switch to ArduinoJson 7, which doesn’t require setting the capacity.

Techniques

Here are the four techniques you can use to determine the capacity of your JsonDocument. The technique you choose depends on your knowledge of your document and your platform.

If you know what the document looks like, choose technique 1 or 2.
If you cannot predict the shape of the document, choose technique 3 or 4.

Don’t overthink this problem!

Keep things simple. If your program computes the capacity at run-time, you’re trying to be too smart!

Don’t aim for the exact memory consumption in each possible configuration. Instead, pick one size that is big enough to support all valid configurations. This RAM will only be used for a very short period anyway; if not, you have another problem.

The first technique consists in using a tool that does the job for you.

The ArduinoJson Assistant is an online tool that helps you determine the capacity of a JsonDocument. It analyzes a JSON document that you provide and computes the required capacity.

To use the ArduinoJson Assistant:

  1. Go to arduinojson.org/assistant.
  2. In step 1, choose your platform and select the appropriate conditions.
  3. In step 2, paste your JSON document.
  4. In step 3, see the required capacity for the JsonDocument.

Additionally, in step 4, the ArduinoJson Assistant generates a sample program that you can use as a starting point.

Technique 2: compute the capacity with macros

The second technique consists in writing an expression that computes the capacity.

If you know the exact layout of the document, you can precisely compute the required size. ArduinoJson provides two macros to help you do that:

  1. JSON_OBJECT_SIZE(n) returns the size of a JSON object with n members.
  2. JSON_ARRAY_SIZE(n) returns the size of a JSON array with n elements.

On top of that, you need to add the capacity required to store the strings that ArduinoJson needs to duplicate.
When deserializing, ArduinoJson copies the strings from the input, except if the input is a char* (see below).
When serializing, ArduinoJson copies all strings that are not const char* (see below).

Let’s see an example. Suppose your JSON document is:

{"values":[1,2,3]}

Then, the required capacity is:

const size_t capacity = JSON_OBJECT_SIZE(1) + JSON_ARRAY_SIZE(3);

If ArduinoJson needs to copy the strings (because you called deserializeJson() with a read-only input, for example), then the required capacity is:

const size_t capacity = JSON_OBJECT_SIZE(1) + JSON_ARRAY_SIZE(3) + 7;

The seven bytes correspond to the characters of the string "values", including the terminator.

String duplication in ArduinoJson

Depending on the type of input, the parser uses a different storage strategy for strings:

  1. If the input is mutable (char* or char[]), the parser assumes that the input buffer is persistent.
    In that case, it stores pointers to the string in the input.
    This is the “zero-copy” mode.

  2. On the other hand, if the input is read-only (const char*, String, or Stream), the parser assumes that the input is volatile.
    In that case, the parser copies the strings to the JsonDocument.

String duplication also happens when you populate a JsonDocument.
If you insert a value (or a key) of type String, std::string, or const __FlashStringHelper*, ArduinoJson stores a copy in the JsonDocument.

Don’t forget to take this into account when you compute the capacity of the JsonDocument.
The ArduinoJson Assistant shows how many extra bytes are required for the string duplications in step 3.

Since ArduinoJson 6.16, the JsonDocument stores only one copy of each string, no matter how many times it appears in the input. Therefore, you only need to count each string once.

Technique 3: set a memory budget

The third technique consists in defining a budget.

If your program must accept any kind of input (meaning you don’t have any information about it), I recommend setting a memory budget for the JSON serialization: at design time, you decide how much memory you can afford to spend on JSON serialization. This requires you to know your program and your platform well, so you know how much memory should be available.

This technique is particularly suitable for StaticJsonDocument because the capacity is known at compile time.

For example, I use this technique in the “Recursive Analyzer” case study in Mastering ArduinoJson.

Technique 4: use whatever RAM is available

The last technique consists in allocating as much memory as possible.

This technique is similar to the previous one, except that you know the capacity at run-time, so you cannot use a StaticJsonDocument.

Because DynamicJsonDocument needs a contiguous memory block, this technique doesn’t work well when the heap is fragmented.

The details depend on your platform. Here is how you could do on ESP8266:

DynamicJsonDocument doc(ESP.getMaxFreeBlockSize() - 512);

And here is for ESP32:

DynamicJsonDocument doc(ESP.getMaxAllocHeap());

In both examples, we call a function that returns the largest block of free memory. In the case of the ESP32, you can directly pass the result to the constructor of DynamicJsonDocument. In the case of the ESP8266, you must subtract a few bytes because the ESP.getMaxFreeBlockSize() doesn’t account for the allocator’s overhead.

This technique consumes a lot of RAM, so only use it for a short period: destruct the DynamicJsonDocument as soon as possible. Alternatively, you can call shrinkToFit() to reduce memory usage but beware of heap fragmentation.

For a detailed explanation, see the “Automatic Capacity” section in the “Advanced Techniques” chapter of Mastering ArduinoJson.

What if my input varies?

If your program must deserialize a JSON document that changes, you must allocate a JsonDocument large enough for the largest valid input. I insist on the term valid because your program doesn’t need to support any JSON document, but only the ones relevant to your use case.

For example, if your “wifi” array supports up to four access-point objects, create a dummy JSON document maxed out like this:

{
    "wifi": [
        {
            "SSID": "XXXXXXXXXXXX",
            "password": "XXXXXXXXXXXX",
        },
        {
            "SSID": "XXXXXXXXXXXX",
            "password": "XXXXXXXXXXXX",
        },
        {
            "SSID": "XXXXXXXXXXXX",
            "password": "XXXXXXXXXXXX",
        },
        {
            "SSID": "XXXXXXXXXXXX",
            "password": "XXXXXXXXXXXX",
        },
    ]
}

Then, pass this document to the ArduinoJson Assistant to get the required capacity. In step 3, leave the box “Deduplicate values when measuring the capacity” unchecked; otherwise, the Assistant will deduplicate the dummy "XXXXXXXXXXXX" strings.

For example, I use this technique in the “Configuration in SPIFFS” case study in Mastering ArduinoJson.

What if I don’t know my input ahead of time?

I really doubt that you don’t know anything about your input, but if that’s the case, use technique 3 (memory budget) or technique 4 (largest possible capacity).

What happens if the JsonDocument is too small?

When the JsonDocument overflows:

See also

Global warming stripes by Professor Ed Hawkins (University of Reading)