How to use external RAM on ESP32?

What is the external RAM?

The ESP32 chip contains 520KB of RAM. While it’s sufficient for most projects, others may need more memory. To increase the capacity of the microcontroller, the manufacturer can add a memory chip to the board. This external RAM chip is connected to the ESP32 via the SPI bus.

For example, the following boards embed such a chip:

In theory, any SPI memory chip could be used, but in practice, it’s always the same: the ESP-PSRAM32. Because this chip uses a technology known as Pseudostatic RAM (PSRAM), we often use the name “PSRAM” when we should really say “External SPI RAM.”

Using the extended memory requires extra work from the programmer: you need to call dedicated allocation functions. Instead of the good old malloc(), you must call heap_caps_malloc(MALLOC_CAP_SPIRAM).

How to use the PSRAM with ArduinoJson?

As we just saw, to use the PSRAM, a program must use dedicated allocation functions, which means we cannot use DynamicJsonDocument. Instead, we must use another kind of JsonDocument that calls the appropriate functions.

You can create this new kind of JsonDocument by defining a custom allocator class that you pass as a template parameter to BasicJsonDocument<T>, the base class of DynamicJsonDocument. The custom allocator must implement two public member functions, allocate() and deallocate(), with the same signatures as malloc() and free().

struct SpiRamAllocator {
  void* allocate(size_t size) {
    return heap_caps_malloc(size, MALLOC_CAP_SPIRAM);
  }

  void deallocate(void* pointer) {
    heap_caps_free(pointer);
  }

  void* reallocate(void* ptr, size_t new_size) {
    return heap_caps_realloc(ptr, new_size, MALLOC_CAP_SPIRAM);
  }
};

using SpiRamJsonDocument = BasicJsonDocument<SpiRamAllocator>;

This snippets defines SpiRamJsonDocument which you can use like any other JsonDocument:

SpiRamJsonDocument doc(1048576);
deserializeJson(doc, input);

You don’t need to do anything else.

You cannot declare a global SpiRamJsonDocument because it would call heap_caps_malloc() before the PSRAM is ready to use.

You probably don’t need SpiRamJsonDocument if you updated the Arduino core; see below.

Alternative solution

Alternatively, you can ask the ESP32 to include external RAM into the classic malloc() function so that a program can use both RAMs without modification. In this mode, malloc() returns memory blocks from either the internal or the external RAM, which means you can use a regular DynamicJsonDocument

To use this mode, you must configure CONFIG_SPIRAM_USE to SPIRAM_USE_MALLOC.

Arduino core for ESP32 version 1

This setting can be set in the following file:

• %LOCALAPPDATA%\Arduino15\packages\esp32\hardware\esp32\1.0.6\tools\sdk (on Windows)
• ~/.arduino15/packages/esp32/hardware/esp32/1.0.6/tools/sdk/sdkconfig (on Linux and macOS)

In the file sdkconfig, you’ll find a block like this:

#
# SPI RAM config
#
CONFIG_SPIRAM_BOOT_INIT=
CONFIG_SPIRAM_USE_MEMMAP=
CONFIG_SPIRAM_USE_CAPS_ALLOC=y
CONFIG_SPIRAM_USE_MALLOC=
CONFIG_SPIRAM_TYPE_AUTO=y
CONFIG_SPIRAM_TYPE_ESPPSRAM32=
CONFIG_SPIRAM_TYPE_ESPPSRAM64=
CONFIG_SPIRAM_SIZE=-1
CONFIG_SPIRAM_SPEED_40M=y
CONFIG_SPIRAM_CACHE_WORKAROUND=y
CONFIG_SPIRAM_BANKSWITCH_ENABLE=y
CONFIG_SPIRAM_BANKSWITCH_RESERVE=8
CONFIG_WIFI_LWIP_ALLOCATION_FROM_SPIRAM_FIRST=
CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY=

You can set CONFIG_SPIRAM_USE_MALLOC=y to include PSRAM in malloc()’s scope.

The problem is this file is volatile: it gets replaced when you update the Arduino core.

Arduino core for ESP32 version 2

Version 2 has separate configurations per ESP generation:

• On Windows:

  1. %LOCALAPPDATA%\Arduino15\packages\esp32\hardware\esp32\2.0.2\tools\sdk\esp32 (for ESP32)
  2. %LOCALAPPDATA%\Arduino15\packages\esp32\hardware\esp32\2.0.2\tools\sdk\esp32s2 (for ESP32-S2)
  3. %LOCALAPPDATA%\Arduino15\packages\esp32\hardware\esp32\2.0.2\tools\sdk\esp32c3 (for ESP32-C3)

• On Linux and macOS:

  1. ~/.arduino15/packages/esp32/hardware/esp32/2.0.2/tools/sdk/esp32 (for ESP32)
  2. ~/.arduino15/packages/esp32/hardware/esp32/2.0.2/tools/sdk/esp32s2 (for ESP32-S2)
  3. ~/.arduino15/packages/esp32/hardware/esp32/2.0.2/tools/sdk/esp32c3 (for ESP32-C3)

In the file sdkconfig, you’ll find a block like this:

#
# SPI RAM config
#
CONFIG_SPIRAM_TYPE_AUTO=y
# CONFIG_SPIRAM_TYPE_ESPPSRAM16 is not set
# CONFIG_SPIRAM_TYPE_ESPPSRAM32 is not set
# CONFIG_SPIRAM_TYPE_ESPPSRAM64 is not set
CONFIG_SPIRAM_SIZE=-1
CONFIG_SPIRAM_SPEED_40M=y
CONFIG_SPIRAM=y
# CONFIG_SPIRAM_BOOT_INIT is not set
# CONFIG_SPIRAM_USE_MEMMAP is not set
# CONFIG_SPIRAM_USE_CAPS_ALLOC is not set
CONFIG_SPIRAM_USE_MALLOC=y
CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=4096
CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y
CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=0
# CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY is not set
# CONFIG_SPIRAM_ALLOW_NOINIT_SEG_EXTERNAL_MEMORY is not set
CONFIG_SPIRAM_CACHE_WORKAROUND=y

As you can see, CONFIG_SPIRAM_USE_MALLOC is already set to y, so malloc() already returns blocks from the PSRAM. In other words, you don’t have to change any setting when using version 2 of the core.

See also

• BasicJsonDocument<T>
• How to use PSRAM on uPesy