What is the external RAM?

The ESP32 chip contains 520KB of RAM. While it’s sufficient for most projects, others may need more memory. To increase the capacity of the microcontroller, the manufacturer can add a memory chip to the board. This external RAM chip is connected to the ESP32 via the SPI bus.

For example, the following boards embed such a chip:

Board External RAM
diymore ESP32 CAM 4 MB
Espressif ESP32-WROVER-KIT 4 MB
HiLetgo ESP32 Camera Module Fisheye 8 MB
KeeYees ESP32-CAM 4 MB
LoLin D32 Pro 8 MB
M5Stack Fire 4 MB
MakerHawk ESP32 Camera 8 MB
TTGO ESP32 Camera 8 MB
TTGO ESP32 WROVER 8 MB
uPesy ESP32 Wrover DevKit 4 MB

In theory, any SPI memory chip could be used, but in practice, it’s always the same: the ESP-PSRAM32. Because this chip uses a technology known as Pseudostatic RAM (PSRAM), we often use the name “PSRAM” when we should really say “External SPI RAM.”

Using the extended memory requires extra work from the programmer: you need to call dedicated allocation functions. Instead of the good old malloc(), you must call heap_caps_malloc(MALLOC_CAP_SPIRAM).

How to use the PSRAM with ArduinoJson?

As we just saw, to use the PSRAM, a program must use dedicated allocation functions, which means we cannot use DynamicJsonDocument. Instead, we must use another kind of JsonDocument that calls the appropriate functions.

You can create this new kind of JsonDocument by defining a custom allocator class that you pass as a template parameter to BasicJsonDocument<T>, the base class of DynamicJsonDocument. The custom allocator must implement two public member functions, allocate() and deallocate(), with the same signatures as malloc() and free().

struct SpiRamAllocator {
  void* allocate(size_t size) {
    return heap_caps_malloc(size, MALLOC_CAP_SPIRAM);
  }

  void deallocate(void* pointer) {
    heap_caps_free(pointer);
  }

  void* reallocate(void* ptr, size_t new_size) {
    return heap_caps_realloc(ptr, new_size, MALLOC_CAP_SPIRAM);
  }
};

using SpiRamJsonDocument = BasicJsonDocument<SpiRamAllocator>;

This snippets defines SpiRamJsonDocument which you can use like any other JsonDocument:

SpiRamJsonDocument doc(1048576);
deserializeJson(doc, input);

You don’t need to do anything else.

You cannot declare a global SpiRamJsonDocument because it would call heap_caps_malloc() before the PSRAM is ready to use.

Alternative solution

Alternatively, you can ask the ESP32 to include external RAM into the classic malloc() function so that a program can use both RAMs without modification. In this mode, malloc() returns memory blocks from either the internal or the external RAM, which means you can use a regular DynamicJsonDocument

To use this mode, you must configure CONFIG_SPIRAM_USE to SPIRAM_USE_MALLOC.

This setting can be set in the following file:

  • %LOCALAPPDATA%\Arduino15\packages\esp32\hardware\esp32\1.0.6\tools\sdk (on Windows)
  • ~/.arduino15/packages/esp32/hardware/esp32/1.0.6/tools/sdk/sdkconfig (on Linux and macOS)

In this file, you’ll find a block like this:

#
# SPI RAM config
#
CONFIG_SPIRAM_BOOT_INIT=
CONFIG_SPIRAM_USE_MEMMAP=
CONFIG_SPIRAM_USE_CAPS_ALLOC=y
CONFIG_SPIRAM_USE_MALLOC=
CONFIG_SPIRAM_TYPE_AUTO=y
CONFIG_SPIRAM_TYPE_ESPPSRAM32=
CONFIG_SPIRAM_TYPE_ESPPSRAM64=
CONFIG_SPIRAM_SIZE=-1
CONFIG_SPIRAM_SPEED_40M=y
CONFIG_SPIRAM_CACHE_WORKAROUND=y
CONFIG_SPIRAM_BANKSWITCH_ENABLE=y
CONFIG_SPIRAM_BANKSWITCH_RESERVE=8
CONFIG_WIFI_LWIP_ALLOCATION_FROM_SPIRAM_FIRST=
CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY=

You can set CONFIG_SPIRAM_USE_MALLOC=y to include PSRAM in malloc()’s scope.

The problem is this file is volatile - it gets replaced when one updates arduino boards.
Please let me know if you know a better (non volatile) solution.

See also

 X online