How to append a JSON object to a file?

Scenario: you have a file (on SD, SPIFFS, LittleFS, or any other filesystem) that contains a list of JSON objects, and you want to add a new object to the list.

Question: What’s the best way to proceed? Should you load the complete file in memory, or is there a better solution?

There are two ways to append a JSON object to a file, depending on the format of the file.

Option 1: the file contains an array

Usually, we store a list of objects in a JSON array, like so:

[
  {"first_name":"Stan","last_name":"Marsh"},
  {"first_name":"Kyle","last_name":"Broflovski"},
  {"first_name":"Eric","last_name":"Cartman"}
]

As you can see, appending an element to the array requires inserting the new object before the closing bracket (]).

This operation is not supported by the filesystem, so our only option is to load the file in memory and serialize it back, like so:

DynamicJsonDocument doc(8192);

// Read the file
File file = SPIFFS.open("/southpath.json", "r");
deserializeJson(doc, file);
file.close();

// Append new element
JsonObject obj = doc.createNestedObject();
obj["first_name"] = "Kenny";
obj["last_name"] = "McCormick";

// Write the file
file = SPIFFS.open("/southpath.json", "w");
serializeJson(doc, file);
file.close();

Option 2: the file contains line-separated objects

If you want to optimize the append-to-file scenario, you must change the format of the file. Instead of wrapping the list in a JSON array, you must write the object one after the other. Usually, we insert a line break between the objects, following the pseudo-standard JSONLines and ndjson.

Here is an example:

{"first_name":"Stan","last_name":"Marsh"}
{"first_name":"Kyle","last_name":"Broflovski"}
{"first_name":"Eric","last_name":"Cartman"}

As you can see, appending an element is just a matter of writing the new object at the end of the file; we don’t need to load the file in memory to do that.

// Create the new element
StaticJsonDocument<256> doc;
doc["first_name"] = "Kenny";
doc["last_name"] = "McCormick";

// Append to file
File file = SPIFFS.open("/southpath.jsonl", "a");
serializeJson(doc, file);
file.println();
file.close();

How to read a JSONL file?

If you want to read this file with ArduinoJson, you simply need to call deserializeJson() repeatedly:

File file = SPIFFS.open("/southpath.jsonl", "r");

while (true) {
  StaticJsonDocument<256> doc;

  DeserializationError err = deserializeJson(doc, file);
  if (err) break;

  Serial.print(doc["first_name"].as<const char*>);
  Serial.print(" ");
  Serial.println(doc["last_name"].as<const char*>);
}

file.close();

Option 1: the file contains an array

Option 2: the file contains line-separated objects

How to read a JSONL file?

See also