How to reduce memory usage?
One of the biggest challenges of programming microcontrollers is the small quantity of RAM that they offer. For example, a classic Arduino UNO has only 2KB of RAM!
That’s why it’s imperative to use special techniques to consume less memory in our program. Some of these techniques are counter-intuitive, and most are radically different from what you usually think is right. If you are used to writing applications for computers, phones, or tablets, you need to adopt a different mindset and challenge your assumptions.
On this page, we’ll see how we can write an Arduino program that consumes less memory. Of course, the focus is on JSON serialization and deserialization, so we’ll see how we can use ArduinoJson with less RAM.
Tip 1: Avoid duplication if the input is in memory
Remember that ArduinoJson’s deserializer has two modes:
- the zero-copy mode, used when the input is writeable (
char*
) - the normal mode, used when the input is read-only (
const char*
,String
)
In the second mode, the deserializer duplicates the relevant parts of the input (basically everything except spaces and punctuation) in the JsonBuffer
.
So, to get the most efficient program, you must use the first mode: the zero-copy mode.
Good: zero-copy
char[] json = "{\"hello\":\"world\"}"; jsonBuffer.parseObject(json);
Bad: duplication
const char* json = "{\"hello\":\"world\"}"; jsonBuffer.parseObject(json);
Bad: duplication
String json = "{\"hello\":\"world\"}"; jsonBuffer.parseObject(json);
See also:
Tip 2: Pass Stream directly to the deserializer
A stream (Stream
or std::istream
) is a source of volatile bytes, so its content needs to be copied in RAM.
The best thing to do is to let ArduinoJson do the duplication as it ignores everything that s not required: punctuation, spaces, and comments.
To do that, simply pass the stream to parseArray()
or parseObject()`.
Good: pass the Stream directly
File file = SD.open(filename); jsonBuffer.parseObject(file);
Bad: pass a copy of the input
char buffer[256]; File file = SD.open(filename); file.read(buffer, 256); DynamicJsonBuffer jsonBuffer; jsonBuffer.parseObject(buffer);
See also:
Tip 3: Prefer stack to heap memory
Allocating and deallocating in the heap cause overhead and fragmentation, so the program can use much less RAM than there is actually on the device.
Heap allocation happens anytime you use malloc()
, new
, and String
.
ArduinoJson uses the stack with StaticJsonBuffer
and the heap for DynamicJsonBuffer
.
For small JsonBuffer
(let’s say under 1KB), prefer a StaticJsonBuffer
.
If you’re using a microcontroller with very limited RAM (for example, the ATmega328 of an Arduino UNO), you should not use the heap at all.
Good: only stack memory
char[] json = "{\"hello\":\"world\"}"; StaticJsonBuffer<200> jsonBuffer; jsonBuffer.parseObject(json);
Bad: only heap memory
String json = "{\"hello\":\"world\"}"; DynamicJsonBuffer jsonBuffer; jsonBuffer.parseObject(json);
Tip 4: Deserialize in chunks
One neat feature of ArduinoJson is that, when it parses an object from a Stream, it stops reading when it encounters the closing }
, and the same is true for arrays.
Using this feature, you don’t have to deserialize the whole JSON document at once. Instead, you can parse only a part of it and repeat the operation.
This technique works great when your input contains an array of nested objects. For example, if you want to parse the huge response of a 10-day forecast of Weather Underground, you can skip the beginning until you see "forecastday": [
in the stream (use Stream::find()
), and then parse the objects for each day one after the other.
As usual, don’t reuse the JsonBuffer
and declare it inside the loop.
See also:
Tip 5: Avoid duplication of Flash strings
At first sight, Flash (or PROGMEM
) strings look excellent for reducing the RAM usage of a program. Indeed, the principle of these strings is to reside in the Flash memory, which is much bigger than the RAM.
Unfortunately, Flash memory uses a different address space, so software that manipulates both RAM and Flash strings needs to temporarily copy the Flash strings into the RAM. For example, the String
class makes a copy in the heap, and ArduinoJson makes a copy in the JsonBuffer
.
Let’s compare the two following lines:
array.add("value"); // ArduinoJson stores a pointer
array.add(F("value")); // ArduinoJson duplicates the string
In the first line, because the strings are in RAM, ArduinoJson can simply store pointers to the strings. But, in the second line, because the strings are in Flash, ArduinoJson must make a copy of them in the RAM.
However, with just these two lines, there is no real problem; but look what happens if we do the same thing in a loop:
// ArduinoJson stores 10 pointers
for (int i=0; i<10; i++) {
array.add("value");
}
// ArduinoJson stores 10 copies
for (int i=0; i<10; i++) {
array.add(F("value"));
}
Here is the problem with Flash strings, as ArduinoJson is not able to see that the same string is inserted several times, it makes several copies of the Flash string.
Flash strings are a double-edged sword. If used correctly, they can save RAM, but most of the time, they make the problem worse.
My advice is the following: don’t use Flash strings for keys and values, only use them for log messages and similar strings.
Tip 6: Avoid global variables
Global variables are bad on many levels; we’ll only look at memory usage.
The problem with global variables is that, by definition, they live during the whole execution of the program. In other words, they always consume RAM, whether the program uses them or not. Contrast that with local variables that have a short lifespan; they only consume memory when the program needs them.
My advice is: reduce the number and the size of global variables to a strict minimum.
Tip 7: Avoid duplication of String
Remember that the String
class always makes a copy of the string passed to the constructor.
For example, the following line makes a copy of “hello world”:
String s = "hello world"; // one copy
Yes, it means that “hello world” is present twice in RAM: in the global section and in the heap.
If you just need to give a name to this variable, it’s better to use the actual type of the string literal, which is const char*
:
const char* s = "hello world"; // no copy
Another common mistake is to declare a function that takes a parameter of type String
by value:
void f(String s) {
}
Because the function parameter is a value (not a pointer nor a reference), each invocation of the function creates a new String
, thereby creating another copy of the string.
To avoid this useless duplication, use a const-reference:
void f(const String& s) {
}
See also:
Tip 8: Reduce the size of variables
In a situation where every byte counts, you need to make sure that every variable is as small as possible.
For example, suppose you have:
int value = 42;
Now, if you’re sure that value
will never exceed 127, you can use a char
instead of an int
:
char value = 42;
Depending on the platform, this can divide the size of this variable by four.
Of course, you save even more memory if it’s an array or a struct. Consider:
int values[32]; // 128 bytes on 32-bit architecture
You can reduce significantly by using char
instead:
char values[32]; // 32 bytes on any architecture
This technique works for local variables, but you can also use if for function parameters. Indeed, when passed by value, function arguments are copied to the stack, so you can save some stack memory by reducing the size of the parameters.
For example, consider changing:
void f(int value);
to
void f(char value);
I know it seems to be a small gain, but remember that most functions call other functions, which call other functions… At some point, the stack becomes a big sandwich with the arguments for many functions. If you reduce the size of the arguments, you can save a lot of stack memory.
Keep learning
If you need more tips or if there are some notions you don’t understand, check out Mastering ArduinoJson.
In particular, the second chapter might interest you; it covers:
- stack and heap
- pointers and references
- strings (including Flash strings and
String
)
But that’s not all. The last chapter contains several case studies to illustrate how to write clean and efficient code with ArduinoJson. For example, it shows how to parse the huge responses from OpenWeatherMap and Weather Underground with limited memory.