Introduction
In the previous guide focused on building messages. But what is the point if we cannot read them? Therefore this guide will be about reading messages.
Lite³ is binary format without any schema definition or IDL. Serialized binary data can be read directly without parsing, transformation, or unmarshalling. The Lite³ API exposes functions for performing key-value lookups similar to 'dictionaries' in dynamic languages.
The example uses a message containing metadata of a book:
{
"pages": 272,
"title": "C Programming Language, 2nd Edition",
"reviews": null,
"language": "en",
"in_stock": true,
"price_usd": 60.3
}
We will be accessing various fields and storing them in variables. Additionally, some utility functions will be showcased.
Example Code
This guide is based on an example inside the Lite³ repository found in examples/context_api/02-reading-messages.c:
40 perror(
"Failed to create lite3_ctx *ctx");
52 perror(
"Failed to build message");
55 printf(
"buflen: %zu\n", ctx->buflen);
57 perror(
"Failed to print JSON");
70 perror(
"Failed to read message");
73 printf(
"\ntitle: %s\n",
LITE3_STR(ctx->buf, title));
74 printf(
"language: %s\n",
LITE3_STR(ctx->buf, language));
75 printf(
"price_usd: %f\n", price_usd);
76 printf(
"pages: %li\n", pages);
77 printf(
"in_stock: %s\n\n", in_stock ?
"true" :
"false");
80 printf(
"No reviews to display.\n");
83 printf(
"\nTitle field exists: %s\n",
lite3_ctx_exists(ctx, 0,
"title") ?
"true" :
"false");
84 printf(
"Price field exists: %s\n",
lite3_ctx_exists(ctx, 0,
"price_usd") ?
"true" :
"false");
85 printf(
"ISBN field exists: %s\n",
lite3_ctx_exists(ctx, 0,
"isbn") ?
"true" :
"false");
88 printf(
"\nTitle is string type: %s\n", title_type ==
LITE3_TYPE_STRING ?
"true" :
"false");
89 printf(
"Title is integer type: %s\n", title_type ==
LITE3_TYPE_I64 ?
"true" :
"false");
93 perror(
"Failed to get price_usd");
96 printf(
"\nPrice is string type: %s\n", lite3_val_is_str(price_val) ?
"true" :
"false");
97 printf(
"Price is double type: %s\n", lite3_val_is_f64(price_val) ?
"true" :
"false");
99 printf(
"price_val value: %f\n", lite3_val_f64(price_val));
103 uint32_t entry_count;
105 perror(
"Failed to get entry count");
108 printf(
"\nObject entries: %u\n", entry_count);
#define lite3_ctx_get(ctx, ofs, key, out)
Get value from object.
#define lite3_ctx_get_str(ctx, ofs, key, out)
Get string value by key.
#define lite3_ctx_get_f64(ctx, ofs, key, out)
Get floating point value by key.
#define lite3_ctx_get_bool(ctx, ofs, key, out)
Get boolean value by key.
#define lite3_ctx_get_i64(ctx, ofs, key, out)
Get integer value by key.
static int lite3_ctx_init_obj(lite3_ctx *ctx)
Initialize a Lite³ context as an object.
static int lite3_ctx_json_print(lite3_ctx *ctx, size_t ofs)
Print Lite³ buffer as JSON to stdout
static lite3_ctx * lite3_ctx_create(void)
Create context with minimum size.
void lite3_ctx_destroy(lite3_ctx *ctx)
Destroy context.
#define lite3_ctx_set_i64(ctx, ofs, key, value)
Set integer in object.
#define lite3_ctx_set_null(ctx, ofs, key)
Set null in object.
#define lite3_ctx_set_bool(ctx, ofs, key, value)
Set boolean in object.
#define lite3_ctx_set_f64(ctx, ofs, key, value)
Set floating point in object.
#define lite3_ctx_set_str(ctx, ofs, key, str)
Set string in object.
#define lite3_ctx_get_type(ctx, ofs, key)
Find value by key and return value type.
static int lite3_ctx_count(lite3_ctx *ctx, size_t ofs, uint32_t *out)
Write back the number of object entries or array elements.
#define lite3_ctx_exists(ctx, ofs, key)
Attempt to find a key.
#define lite3_ctx_is_null(ctx, ofs, key)
Find value by key and test for null type.
lite3_type
enum containing all Lite³ types
#define LITE3_STR(buf, val)
Generational pointer / safe access wrapper.
@ LITE3_TYPE_STRING
maps to 'string' type in JSON
@ LITE3_TYPE_F64
maps to 'number' type in JSON; underlying datatype: double
@ LITE3_TYPE_I64
maps to 'number' type in JSON; underlying datatype: int64_t
static size_t lite3_val_type_size(lite3_val *val)
Returns the size of the value type.
Lite³ Context API Header.
Struct holding a reference to a string inside a Lite³ buffer.
Struct representing a value inside a Lite³ buffer.
Output:
buflen: 220
{
"pages": 272,
"title": "C Programming Language, 2nd Edition",
"reviews": null,
"language": "en",
"in_stock": true,
"price_usd": 60.3
}
title: C Programming Language, 2nd Edition
language: en
price_usd: 60.300000
pages: 272
in_stock: true
No reviews to display.
Title field exists: true
Price field exists: true
ISBN field exists: false
Title is string type: true
Title is integer type: false
Price is string type: false
Price is double type: true
price_val value: 60.300000
price_val type size: 8
Object entries: 6
Explanation
We will walk through the example code step-by-step, explaining the use of Lite³ library functions.
Lite³ messages are just bytes, stored contiguously inside a buffer. If you want to allocate these messages inside your own custom allocators, you can using the Buffer API. However in this guide, we will be using the Context API so that memory is managed automatically.
Note that Lite³ is a binary format, but the examples print message data as JSON to stdout for better readability.
Building the message
As explained in the first guide, we use contexts to store Lite³ buffers (see: Context API):
if (!ctx) {
perror("Failed to create lite3_ctx *ctx");
return 1;
}
We then insert the book metadata:
perror("Failed to build message");
return 1;
}
printf("buflen: %zu\n", ctx->buflen);
perror("Failed to print JSON");
return 1;
}
- Note
- Function are grouped inside a single if-statement for error handling. Otherwise, each function would need a separate if-statement.
lite3_ctx_json_print() is used to convert Lite³ binary data to JSON, since raw bytes are not very friendly to the human eye.
The output shows us what this data looks like:
buflen: 220
{
"pages": 272,
"title": "C Programming Language, 2nd Edition",
"reviews": null,
"language": "en",
"in_stock": true,
"price_usd": 60.3
}
We can see that this message takes up a total of 220 bytes.
Reading values
Now that we have a serialized message, we can start reading individual values from it. For this, we use Object Get functions:
double price_usd;
int64_t pages;
bool in_stock;
perror("Failed to read message");
return 1;
}
printf(
"\ntitle: %s\n",
LITE3_STR(ctx->buf, title));
printf(
"language: %s\n",
LITE3_STR(ctx->buf, language));
printf("price_usd: %f\n", price_usd);
printf("pages: %li\n", pages);
printf("in_stock: %s\n\n", in_stock ? "true" : "false");
Output:
title: C Programming Language, 2nd Edition
language: en
price_usd: 60.300000
pages: 272
in_stock: true
Get functions still return int for error handling. The actual values are read via *out parameters:
ctx is the context that stores our message. In the previous guide we explained the 2nd 'zero' parameter. Basically, if we read from the root-level object/array, we use ofs == 0. This will change when we start using Nesting. The 3rd parameter is the key to lookup and the 4th is the *out write-back parameter (the actual value we're reading).
Note that we are reading these values directly from the serialized binary data in true zero-copy style 😎. Most serialization formats require parsing and/or intermediate datastructures to support this kind of API.
- Note
- Reading values from Lite³ is done through write-back parameters since pointer dereferences at unaligned addresses are not portable to non-x86 platforms like ARM. Guaranteeing alignment inside serialized data is difficult, since it would almost always require wasteful padding bytes.
Strings
When reading strings, what is this lite3_str struct for? Why not just char *?
Actually, this struct still contains a char pointer and can be accessed directly using title.ptr or language.ptr. However, this kind of direct access is highly discouraged. Instead the pointer is returned through a macro:
printf(
"\ntitle: %s\n",
LITE3_STR(ctx->buf, title));
printf(
"language: %s\n",
LITE3_STR(ctx->buf, language));
One thing to remember about strings, is that they are just pointers. One problem of storing these pointers is that if we insert more data into ctx, then this will modify the structure of the message. If afterwards we dereference one of these stored pointers, it could trigger a dangling pointer scenario. If we're lucky, the pointer is still valid. But otherwise, we get undefined behavior.
The LITE3_STR() macro only returns a direct pointer if we can guarantee its validity. Otherwise, it returns NULL. This will be explained more in detail in the next guide (strings).
Null
Besides strings, the other types are fairly straightforward except for LITE3_TYPE_NULL, which maps directly to null in JSON.
If you think about it, a function lite3_ctx_get_null() doesn't really make sense. If we are already expecting null, then why would be read it? So that's why this function doesn't exist. Instead, we check for a null value like this:
printf("No reviews to display.\n");
}
Output:
If we would attempt to read a null value using something like lite3_ctx_get_str(ctx, 0, "reviews", &reviews), then this would simply fail, returning -1 with errno == EINVAL.
- Note
- Object Get functions will error if the found value does not match the expected type. This runtime type-checking is an important safety feature of Lite³. Only the generic
lite3_get() requires manual type checking using lite3_val functions.
In the C programming language, NULL is a special pointer value 'pointing to nowhere'. But this is totally unrelated.
Field exists?
Sometimes we don't want to read a value, we just want to know if a key exists inside an object. Perhaps it is an optional key.
This is where Utility Functions come in:
printf(
"\nTitle field exists: %s\n",
lite3_ctx_exists(ctx, 0,
"title") ?
"true" :
"false");
printf(
"Price field exists: %s\n",
lite3_ctx_exists(ctx, 0,
"price_usd") ?
"true" :
"false");
printf(
"ISBN field exists: %s\n",
lite3_ctx_exists(ctx, 0,
"isbn") ?
"true" :
"false");
Output:
Title field exists: true
Price field exists: true
ISBN field exists: false
If the key cannot be found for another reason, i.e. because message is invalid, then the function also returns false.
Get type
Sometimes we may want to get the type of a value:
printf(
"\nTitle is string type: %s\n", title_type ==
LITE3_TYPE_STRING ?
"true" :
"false");
printf(
"Title is integer type: %s\n", title_type ==
LITE3_TYPE_I64 ?
"true" :
"false");
Output:
Title is string type: true
Title is integer type: false
Lite³ types are defined in enum lite3_type.
Get opaque value
Unlike other Object Get functions, lite3_get() does not get a specific type. Instead, it produces a generic lite3_val pointer, which points to a value inside the Lite³ buffer. This can be useful in cases where you don't know the exact type of a value beforehand. For instance:
- ambiguous types
- values that may also be
null
- values returned by Iterators
See lite3_val functions.
perror("Failed to get price_usd");
return 1;
}
printf("\nPrice is string type: %s\n", lite3_val_is_str(price_val) ? "true" : "false");
printf("Price is double type: %s\n", lite3_val_is_f64(price_val) ? "true" : "false");
printf("price_val value: %f\n", lite3_val_f64(price_val));
}
Output:
Price is string type: false
Price is double type: true
price_val value: 60.300000
price_val type size: 8
(doubles are 8 bytes)
- Warning
- Any value read as lite3_val must be read as the correct type, or the underlying data will be misinterpreted, potentially leading to undefined behavior. Always type check before reading lite3_val. There is no runtime safety provided by the library here, because applications are already expected to type check for opaque values.
Object entry count
Every object also stores how many entries it contains. This can be read like so:
uint32_t entry_count;
perror("Failed to get entry count");
return 1;
}
printf("\nObject entries: %u\n", entry_count);
Output:
- Note
- Indexes and counts are typed
(uint32_t), while buffer offsets and lengths are (size_t).
Cleaning up
We are good citizens, so we clean up after ourselves:
This destroys the context, freeing all the internal buffers so that the memory is released. When you create contexts, don't forget to destroy them, or else you will be leaking memory.
Conclusion
This was the second guide showing how to read messages.
In the next guide will cover strings and how to handle them properly:
Next Guide: Strings