How to reduce memory usage?

One of the biggest challenges of microcontrollers is the small quantity of RAM they offer. On this page, we’ll see how to write Arduino programs that consume less memory. Of course, the focus is on JSON serialization and deserialization, but some tips are applicable even when you don’t use ArduinoJson.

Some ESP32s come with a large external PSRAM that you can use with ArduinoJson.

Tip 1: Pass Stream directly to the deserializer

A stream (Stream or std::istream) is a volatile bytes source, so its content needs to be copied into RAM. The best way to deal with streams is to let ArduinoJson copy the content because it ignores everything it can: punctuation, spaces, comments, and duplicated strings.

To do that, simply pass the stream to deserializeJson() or deserializeMsgPack().

Good: pass the Stream directly
File file = SD.open(filename);
deserializeJson(doc, file);
Bad: pass a copy of the input
char buffer[256];
File file = SD.open(filename);
file.read(buffer, 256);
JsonDocument doc;
deserializeJson(doc, buffer);
Worse: copy the input on the heap
File file = SD.open(filename);
size_t fileSize = file.size();
std::unique_ptr<char[]> buffer(new char[fileSize]);
file.read(buffer.get(), fileSize);
JsonDocument doc;
deserializeJson(doc, buffer.get(), fileSize);
Even worse: manage heap memory manually
File file = SD.open(filename);
size_t fileSize = file.size();
char* buffer = (char*)malloc(fileSize);
file.read(buffer, fileSize);
JsonDocument doc;
deserializeJson(doc, buffer, fileSize);
free(buffer);
See also:

Tip 2: Move longs strings literals to Flash memory

Flash (or PROGMEM) strings are excellent for reducing the RAM usage of a program: contrary to regular string literals, they only use RAM when the program pulls them from the Flash.

This advice is only relevant to Harvard architectures (mainly AVR and ESP8266).
It does not apply to von Neumann architectures (such as ESP32, megaAVR, and ARM).
See the first chapter of Mastering ArduinoJson for an explanation.

Good: long strings literals in Flash
doc[F("description")] = F("Lorem ipsum dolor sit amet, consectetur adipiscing elit.");
Bad: long strings literals in RAM
doc["description"] = "Lorem ipsum dolor sit amet, consectetur adipiscing elit.";
Good: log strings in Flash
Serial.print(F("deserializeJson() failed with error code "));
Serial.println(err.f_str());
Bad: log strings in RAM
Serial.print("deserializeJson() failed with error code ");
Serial.println(err.c_str());

The compiler doesn’t deduplicate strings created by the F() macro. Don’t repeat the same string multiple times; instead, save the pointer in a variable of type const __FlashStringHelper*.

Tip 3: Filter input

You can tell deserializeJson() to filter the input to keep only the relevant parts.

This technique only makes sense if the input contains a lot of information that you are not interested in. A typical example is a web service like OpenWeatherMap, which returns many fields, but only a few are relevant to your project.

For the implementation details, see:

Tip 4: Deserialize in chunks

One neat feature of ArduinoJson is that, when it parses an object from a Stream, it stops reading when it encounters the closing }, and the same is true for arrays. Using this feature, you don’t have to deserialize the whole JSON document at once. Instead, you can parse only a part of it and repeat the operation.

This technique works great when your input contains an array of nested objects. For example, if you want to parse the huge response of a 10-day forecast of Weather Underground, you can skip the beginning until you see "forecastday": [ in the stream (use Stream::find()), and then parse the objects for each day one after the other.

For the implementation details, see:

Tip 5: Avoid global variables

Global variables are harmful on many levels; we’ll only look at memory usage.

The problem with global variables is that, by definition, they live during the whole execution of the program. In other words, they always consume RAM, whether the program uses them or not. Contrast that with local variables with a short lifespan; they only consume memory when the program needs them.

My advice is: reduce the number and the size of global variables to the strict minimum.

Tip 6: Avoid duplication of String

Remember that the String class always makes a copy of the string passed to the constructor.

For example, the following line makes a copy of “hello world”:

String s = "hello world"; // one copy

Yes, it means that “hello world” is present twice in RAM: in the global section and in the heap. If you just need to give a name to this variable, it’s better to use the actual type of the string literal, which is const char*:

const char* s = "hello world"; // no copy

Another common mistake is to declare a function that takes a parameter of type String by value:

void f(String s) {
  // ...
}

Because the function parameter is a value (not a pointer, nor a reference), each function invocation creates a new String, thereby creating another copy of the string. To avoid this useless duplication, use a const-reference:

void f(const String& s) {
  // ...
}
See also:

Tip 7: Reduce the size of variables

In situations where every byte counts, you need to make sure that every variable is as small as possible.

For example, suppose you have:

int value = 42;

If you’re sure that value will never exceed 127, you can use a char instead of an int:

char value = 42;

Depending on the platform, this can divide the size of this variable by four.

Of course, you save even more memory if it’s an array or a struct. Consider:

int values[32]; // 128 bytes on 32-bit architecture

You can reduce significantly by using char instead:

char values[32]; // 32 bytes on any architecture

This technique works for local variables, but you can also use it for function parameters. Indeed, when passed by value, function arguments are copied to the stack, so you can save some stack memory by reducing the parameters’ size.

For example, consider changing:

void f(int value);

to

void f(char value);

I know it seems to be a small gain, but remember that most functions call other functions, which call other functions… At some point, the stack becomes a big sandwich with the arguments to many functions. If you reduce the size of the arguments, you can save a lot of stack memory.

Global warming stripes by Professor Ed Hawkins (University of Reading)