Why does the document take up so much RAM?
Every once in a while, I receive a message from an unhappy user who finds that the library consumes too much memory. Invariably, the person wants to store an array of integers in a JSON document and is shocked by the size computed by the ArduinoJson Assistant. The rationale is always the same: “My input file is 2KB long, how could it take 3KB of RAM?”
Let me illustrate the issue with an example. Here is a JSON document containing an array of integers:
[1,2,3,4]
In its serialized form, this JSON document takes 9 bytes.
Yet, the ArduinoJson Assistant tells us we need a JsonDocument with a capacity of 64 bytes!
Indeed, from this angle ArduinoJson (and all other JSON libraries) looks terrible. Let’s see another example, again with four integers:
[-9223372036854775808,-9223372036854775808,-9223372036854775808,-9223372036854775808]
Now, the serialized form is 85 bytes, but the Assistant still says we need 64 bytes. ArduinoJson certainly looks better from that angle.
As you can see, the serialized size depends on the values, whereas the size in RAM doesn’t. Therefore, comparing these two measures doesn’t give a good appreciation of the library’s memory efficiency.
What would be a good measure, then? We could compare it with the RAM a C program uses to store the same information. For example, the following array takes 16 bytes on most platforms:
int values[4];
Right, but ArduinoJson stores 64-bit integers, so a closer approximation would be:
int64_t values[4];
This array takes 32 bytes; it’s still twice less than what ArduinoJson requires!
Indeed, but programs don’t always insert the values sequentially. For example, a program could do that:
doc["array1"].add(1);
doc["array2"].add(1);
doc["array1"].add(2);
Even though 1 and 2 appear next to each other in the serialized form, they were not added together in the document. If the JsonDocument stored these values sequentially, it would have to reallocate “array1” on the third line to make room for the new value. Sure, I could implement this behavior, but it would complexify the code and significantly increase the library size.
Of course, a similar problem occurs if the program removes an element:
doc["array1"].add(1);
doc["array1"].add(2);
doc["array1"].remove(0);
If elements were stored sequentially, ArduinoJson would have to move the elements. Again, doable but not ideal.
That’s why ArduinoJson stores arrays (and objects) as a linked list. If you want to replicate that in a C++ program, it would look like so:
struct Node {
int64_t* value;
Node* next;
};
Node nodes[4];
Check by yourself, and you’ll see that sizeof(nodes) is 64.
Admittedly, 64 bytes to store 1, 2, 3, and 4 is too much, but it’s only a problem if you have many numbers. If that’s your case, you’re probably trying to store binary data; something JSON is notoriously bad for. Instead, you should consider using a binary format. You can watch my Serialization and JSON on Arduino presentation on YouTube.
Anyway, this excessive RAM usage should be very limited in time. Indeed, ArduinoJson is a serialization library; it never pretended to be a replacement for “vector
int values[4];
void readConfig(File file) {
DynamicJsonDocument doc(128);
deserializeJson(doc, file);
copyArray(doc["values"], values);
}
This code snippet demonstrates how to best use the library: the JsonDocument is short-lived, so memory consumption is less of a problem.
There are other reasons why you should follow this pattern in your program. Please read Why must I create a separate config object? Why can’t I use JsonDocument directly?
