userver: Formats (JSON, YAML, BSON, ...)
Loading...
Searching...
No Matches
Formats (JSON, YAML, BSON, ...)

Userver provides classes for reading, working with, and serializing various data formats. Classes for different formats have an almost identical interface. There are common points for customizing parsers and serializers of custom types.

formats::*::Value

Classes formats::json::Value, formats::bson::Value, yaml_config::YamlConfig and formats::yaml::Value are intended for non-modifying work with formats (in other words, for reading data).

Usage Example:

// #include <userver/formats/json.hpp>
"key1": 1,
"key2": {"key3":"val"}
})");
const auto key1 = json["key1"].As<int>();
ASSERT_EQ(key1, 1);
const auto key3 = json["key2"]["key3"].As<std::string>();
ASSERT_EQ(key3, "val");

Customization of formats::*::Value::As<T>()

In order for formats::*::Value to be able to represent data as a C++ type, you should write a special function Parse for that C++ type. Parse should be located in the namespace of the type or may be located in the formats::common namespace if the type comes from third-party library that you have no control of:

namespace my_namespace {
struct MyKeyValue {
std::string field1;
int field2;
};
// The function must be declared in the namespace of your type
MyKeyValue Parse(const formats::json::Value& json,
return MyKeyValue{
json["field1"].As<std::string>(""),
json["field2"].As<int>(1), // return `1` if "field2" is missing
};
}
TEST(FormatsJson, ExampleUsageMyStruct) {
"my_value": {
"field1": "one",
"field2": 1
}
})");
auto data = json["my_value"].As<MyKeyValue>();
EXPECT_EQ(data.field1, "one");
EXPECT_EQ(data.field2, 1);
}
} // namespace my_namespace

You can write a single parser for all formats, just make it a template:

USERVER_NAMESPACE_BEGIN
namespace my_namespace {
struct MyKeyValue {
std::string field1;
int field2;
};
// The function must be declared in the namespace of your type
template <class Value>
MyKeyValue Parse(const Value& data, formats::parse::To<MyKeyValue>) {
return MyKeyValue{
data["field1"].template As<std::string>(),
data["field2"].template As<int>(-1), // return `1` if "field2" is missing
};
}
TEST(CommonFormats, Parse) {
// json
"my_value": {
"field1": "one",
"field2": 1
}
})");
auto data_json = json["my_value"].As<MyKeyValue>();
EXPECT_EQ(data_json.field1, "one");
EXPECT_EQ(data_json.field2, 1);
// yaml
my_value:
field1: "one"
field2: 1
)");
auto data_yaml = yaml["my_value"].As<MyKeyValue>();
EXPECT_EQ(data_yaml.field1, "one");
EXPECT_EQ(data_yaml.field2, 1);
}
} // namespace my_namespace

Inline helpers formats::*::Make*

To build objects of trivial types some of the formats provide inline helpers, like formats::json::MakeArray(), formats::json::MakeObject():

const auto kDoc = formats::json::MakeObject(
"key1", 1,
"key2", "val",
"key3", formats::json::MakeObject("sub", -1),
"key4", formats::json::MakeArray(1, 2, 3),
"key5", 10.5,
"key6", false
);

Or formats::bson::MakeDoc(), formats::bson::MakeArray():

const auto doc = formats::bson::MakeDoc(
"key_a", formats::bson::MakeArray(0, 1, 2), //
"key_d", formats::bson::MakeDoc("one", 1, "two", 2), //
"key_n", nullptr //
);
auto umap = doc["key_d"].As<std::unordered_map<std::string, int>>();
EXPECT_EQ(1, umap["one"]);
EXPECT_EQ(2, umap["two"]);

Those inline helper functions usually work slightly faster than formats::*::ValueBuilder. However, if you need a std::string with JSON the fastest way would be to use the Streaming Serialization. Inline helpers could not be customized for used provided types, unlike other format building types. Inline helpers could produce broken value on bad input because they skip some of the checks, for example a key uniqueness check.

formats::*::ValueBuilder

Classes formats::json::ValueBuilder, formats::bson::ValueBuilder and formats::yaml::ValueBuilder are designed for building objects of a given format.

Usage Example:

// #include <userver/formats/json.hpp>
builder["key1"] = 1;
builder["key2"]["key3"] = "val";
ASSERT_EQ(json["key1"].As<int>(), 1);
ASSERT_EQ(json["key2"]["key3"].As<std::string>(), "val");

Customization of formats::*::ValueBuilder

In order for formats::*::ValueBuilder to be able to represent a C++ type in the specified format, you should write a special function Serialize for that C++ type. Serialize should be located in the namespace of the type or may be located in the formats::common namespace if the type comes from third-party library that you have no control of:

namespace my_namespace {
struct MyKeyValue {
std::string field1;
int field2;
};
// The function must be declared in the namespace of your type
formats::json::Value Serialize(const MyKeyValue& data,
builder["field1"] = data.field1;
builder["field2"] = data.field2;
return builder.ExtractValue();
}
TEST(JsonValueBuilder, ExampleCustomization) {
MyKeyValue object = {"val", 1};
builder["example"] = object;
auto json = builder.ExtractValue();
ASSERT_EQ(json["example"]["field1"].As<std::string>(), "val");
ASSERT_EQ(json["example"]["field2"].As<int>(), 1);
}
TEST(JsonValueBuilder, StringViewRemove) {
const std::string str = "ab";
builder["a"] = 1;
builder[str] = 2;
builder.Remove(std::string_view(str.data(), 1));
EXPECT_EQ(1, builder.GetSize());
const auto value = builder.ExtractValue();
EXPECT_EQ(2, value[str].As<int>());
}
TEST(JsonValueBuilder, StringViewHasMember) {
const std::string str = "ab";
main_builder[str] = 2;
EXPECT_EQ(false, main_builder.HasMember("a"));
EXPECT_EQ(false, main_builder.HasMember(std::string_view(str.data(), 1)));
EXPECT_EQ(true, main_builder.HasMember(std::string_view(str.data(), 2)));
}
TEST(JsonValueBuilder, StringViewEmplaceNocheck) {
const std::string str = "ab";
main_builder.EmplaceNocheck(std::string_view(str.data(), 1), 1);
main_builder.EmplaceNocheck(std::string_view(str.data(), 2), 2);
main_builder.EmplaceNocheck(std::string_view(std::string(1024, 'a') + 'b'),
1025);
const auto value = main_builder.ExtractValue();
EXPECT_EQ(1, value["a"].As<int>());
EXPECT_EQ(2, value["ab"].As<int>());
EXPECT_EQ(1025, value[std::string(1024, 'a') + 'b'].As<int>());
}
} // namespace my_namespace

You can write a single serializer for all formats, for make it a template:

USERVER_NAMESPACE_BEGIN
namespace my_namespace {
struct MyKeyValue {
std::string field1;
int field2;
};
// The function must be declared in the namespace of your type
template <class Value>
Value Serialize(const MyKeyValue& data, formats::serialize::To<Value>) {
typename Value::Builder builder;
builder["field1"] = data.field1;
builder["field2"] = data.field2;
return builder.ExtractValue();
}
TEST(CommonFormats, Serialize) {
MyKeyValue obj = {"val", 1};
// json
builderJson["example"] = obj;
auto json = builderJson.ExtractValue();
EXPECT_EQ(json["example"]["field1"].As<std::string>(), "val");
EXPECT_EQ(json["example"]["field2"].As<int>(), 1);
// yaml
builderYaml["example"] = obj;
auto yaml = builderYaml.ExtractValue();
EXPECT_EQ(yaml["example"]["field1"].As<std::string>(), "val");
EXPECT_EQ(yaml["example"]["field2"].As<int>(), 1);
}
} // namespace my_namespace

Streaming Serialization

For runtime-critical code, it is possible to use streaming serializers. They allow you to serialize several times faster than formats::json::ValueBuilder, but should be used carefully because may produce broken format.

At the moment, stream serialization is implemented only for JSON via the formats::json::StringBuilder.

In order for stream serialization to work with your data type, you need to define the WriteToStream function in the namespace of your type:

namespace my_namespace {
struct MyKeyValue {
std::string field1;
int field2;
};
// The function must be declared in the namespace of your type
inline void WriteToStream(const MyKeyValue& data,
// A class that adds '{' in the constructor and '}' in the destructor to JSON
sw.Key("field1");
// Use the WriteToStream functions for values, don't work with StringBuilder
// directly
WriteToStream(data.field1, sw);
sw.Key("field2");
WriteToStream(data.field2, sw);
}
TEST(JsonStringBuilder, ExampleUsage) {
StringBuilder sb;
MyKeyValue data = {"one", 1};
WriteToStream(data, sb);
ASSERT_EQ(sb.GetString(), "{\"field1\":\"one\",\"field2\":1}");
}
} // namespace my_namespace

Note that you may get invalid JSON, since:

Test your serializers!