Strings

Table of Contents

• Building the message
• Safely reading strings
• Generational pointers
• Inserting strings by length
• Cleaning up

Introduction

The previous guide was about reading messages. This guide will be about strings.

Strings, are just pointers. This guide will show how to safely read strings pointing to live message data using the LITE3_STR() macro.

Example Code

This guide is based on an example inside the Lite³ repository found in examples/context_api/03-strings.c:

/*
    Lite³: A JSON-Compatible Zero-Copy Serialization Format
 
    Copyright © 2025 Elias de Jong <elias@fastserial.com>
 
    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"), to deal
    in the Software without restriction, including without limitation the rights
    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    copies of the Software, and to permit persons to whom the Software is
    furnished to do so, subject to the following conditions:
 
    The above copyright notice and this permission notice shall be included in all
    copies or substantial portions of the Software.
 
    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
    SOFTWARE.
 
      __ __________________        ____
    _  ___ ___/ /___(_)_/ /_______|_  /
     _  _____/ / __/ /_  __/  _ \_/_ < 
      ___ __/ /___/ / / /_ /  __/____/ 
           /_____/_/  \__/ \___/       
*/
#include <stdio.h>
#include <string.h>
 
#include "lite3_context_api.h"
 
 
int main() {
        lite3_ctx *ctx = lite3_ctx_create();
        if (!ctx) {
                perror("Failed to create lite3_ctx *ctx");
                return 1;
        }
 
        // Build message
        if (lite3_ctx_init_obj(ctx)                                             < 0
                || lite3_ctx_set_str(ctx, 0, "name", "Marie")                   < 0
                || lite3_ctx_set_i64(ctx, 0, "age", 24)                         < 0
                || lite3_ctx_set_str(ctx, 0, "email", "marie@example.com")      < 0) {
                perror("Failed to build message");
                return 1;
        }
 
        // Remember: strings contain a pointer directly to the live data
        lite3_str email;
        if (lite3_ctx_get_str(ctx, 0, "email", &email) < 0) {
                perror("Failed to get email");
                return 1;
        }
 
        // ⚠️ Buffer mutation invalidates email!
        if (lite3_ctx_set_str(ctx, 0, "phone", "1234567890") < 0) {
                perror("Failed to set email");
                return 1;
        }
        // // ✅ Uncomment this code to see email become valid again
        // printf("Refreshing string reference...\n");
        // if (lite3_ctx_get_str(ctx, 0, "email", &email) < 0) {
        //         perror("Failed to get email");
        //         return 1;
        // }
        printf("Marie's email: %s\n\n", LITE3_STR(ctx->buf, email));
 
        const char *country = "Germany";
        size_t country_len = strlen(country);
 
        if (lite3_ctx_set_str_n(ctx, 0, "country", country, country_len) < 0) {
                perror("Failed to set country");
                return 1;
        }
 
        if (lite3_ctx_json_print(ctx, 0) < 0) { // Print Lite³ as JSON
                perror("Failed to print JSON");
                return 1;
        }
 
        lite3_ctx_destroy(ctx);
        return 0;
}

Output:

Marie's email: (null)
 
{
    "age": 24,
    "email": "marie@example.com",
    "phone": "1234567890",
    "name": "Marie",
    "country": "Germany"
}

Explanation

We will walk through the example code step-by-step, explaining the use of Lite³ library functions.

Lite³ messages are just bytes, stored contiguously inside a buffer. If you want to allocate these messages inside your own custom allocators, you can using the Buffer API. However in this guide, we will be using the Context API so that memory is managed automatically.

Note that Lite³ is a binary format, but the examples print message data as JSON to stdout for better readability.

Building the message

As explained in the first guide, we use contexts to store Lite³ buffers (see: Context API):

lite3_ctx *ctx = lite3_ctx_create();
if (!ctx) {
        perror("Failed to create lite3_ctx *ctx");
        return 1;
}

We then insert some basic fields:

// Build message
if (lite3_ctx_init_obj(ctx)                                             < 0
        || lite3_ctx_set_str(ctx, 0, "name", "Marie")                   < 0
        || lite3_ctx_set_i64(ctx, 0, "age", 24)                         < 0
        || lite3_ctx_set_str(ctx, 0, "email", "marie@example.com")      < 0) {
        perror("Failed to build message");
        return 1;
}

At this point, the message looks like this:

{
    "age": 24,
    "email": "marie@example.com",
    "name": "Marie"
}

Safely reading strings

Now we enter a curious sequence:

1. we first read the email string and store it in a variable
2. then we insert an unrelated field "phone"
3. now we try to read again from the same email string

Let's see the code first:

// Remember: strings contain a pointer directly to the live data
lite3_str email;
if (lite3_ctx_get_str(ctx, 0, "email", &email) < 0) {
        perror("Failed to get email");
        return 1;
}
 
// ⚠️ Buffer mutation invalidates email!
if (lite3_ctx_set_str(ctx, 0, "phone", "1234567890") < 0) {
        perror("Failed to set email");
        return 1;
}
// // ✅ Uncomment this code to see email become valid again
// printf("Refreshing string reference...\n");
// if (lite3_ctx_get_str(ctx, 0, "email", &email) < 0) {
//         perror("Failed to get email");
//         return 1;
// }
printf("Marie's email: %s\n\n", LITE3_STR(ctx->buf, email));

Output:

Marie's email: (null)

If we however uncomment the code snippet to overwrite the reference after the mutation has occured:

Refreshing string reference...
Marie's email: marie@example.com

Now the string is valid again, and the macro returns a direct pointer.

Note

Lite³ always internally appends a NULL character to strings, although it is not included in the length count. This makes sure that Lite³ strings are compatible with functions expecting C strings.

Generational pointers

Strings in C are typically declared as const char * or just char *. So why then do we bother with this lite3_str struct?

To understand why, imagine we were using just a simple char pointer. When we read a string field, we store the char pointer somewhere, and read it when we want. Right?

Wrong.

Because as you see in the above code sample, what happens when we insert data? Inserting data will mutate the buffer, changing its internal structure. Since this affects the underlying data, we can no longer guarantee that the pointer will be valid. If we're lucky, the string is still located in the same place. However if unlucky, we could run into a dangling pointer scenario.

With most other serialization formats, the data, once serialized, is immutable. Therefore it is trivial to hand out char pointers. Lite³ however is mutable 'by design'. Data is always simultaneously serialized and mutable. If we hand out string references, we we cannot possibly keep track of all their individual lifetimes, let alone their validity.

One solution would be to always copy string data. This way, the copy remains safe to read. However this would require extra memory allocation and violate our 'zero-copy philosophy'. Being able to read data directly in-place is an incredibly powerful optimization. But how then can we provide this ability without compromising on safety or useability?

For this, Lite³ implements a safety mechanism called 'generational pointers', also known as 'generational references'.

Basically, every pointer stores an extra 'generation' field. Pointers always point to some data source, which also contain this field. When a pointer is obtained, its generation matches that of the source. As long as its generation matches that of the source, it is considered 'valid'. If however the data source is changed or modified in any way, we increment the source generation (new data, new generation). All outstanding references to the previous generation are now outdated and therefore 'invalid'. Attempting to read such a reference will return NULL.

This becomes visible through the lite3_str struct members:

lite3_str.gen // generation of the Lite³ buffer when this struct was returned
lite3_str.len // char array length (characters, exclusive of NULL-terminator)
lite3_str.ptr // char array pointer to string inside Lite³ buffer

The generation count is not read manually, but using a macro:

printf("Marie's email: %s\n\n", LITE3_STR(ctx->buf, email));

• ctx->buf is the buffer pointer (uint8_t *), pointing to the start of the serialized message stored inside the context.
• email is an instance of lite3_str.

Every Lite³ message stores an internal generation count that is incremented on buffer mutation, typically by Object Set functions. The macro will compare the generation count of the buffer with that of the string reference. If they match, the string pointer is returned. Otherwise, it returns NULL.

This mechanism allows for safe references that will automatically invalidate when the underlying data is changed. It provides safety and peace of mind, knowing that pointers are safe to dereference. Application developers are highly encouraged to use this pattern throughout their codebases.

For those worried about the runtime performance impact of this macro, it is almost non-existant. The check is branchless and essentially compiles down to a single cmov.

Inserting strings by length

All examples until now have used lite3_ctx_get_str(). This function inserts a string into a Lite³ message. Behind the scenes, a call to strlen() is made to reserve enough space. This means the entire string must be scanned to find out its length. But what if you already know the length beforehand?

Then you can use lite3_ctx_set_str_n():

const char *country = "Germany";
size_t country_len = strlen(country);
 
if (lite3_ctx_set_str_n(ctx, 0, "country", country, country_len) < 0) {
        perror("Failed to set country");
        return 1;
}

It just takes one extra parameter: str_len. This is the length of the string, excluding NULL-terminator.

Finally after all insertions, we can see what the data looks like:

if (lite3_ctx_json_print(ctx, 0) < 0) { // Print Lite³ as JSON
        perror("Failed to print JSON");
        return 1;
}

Output:

{
    "age": 24,
    "email": "marie@example.com",
    "phone": "1234567890",
    "name": "Marie",
    "country": "Germany"
}

Cleaning up

We are good citizens, so we clean up after ourselves:

lite3_ctx_destroy(ctx);

This destroys the context, freeing all the internal buffers so that the memory is released. When you create contexts, don't forget to destroy them, or else you will be leaking memory.

Conclusion

This was the third guide showing how to use strings.

In the next guide we will start seeing nested objects: Next Guide: Nesting