Unit Tests and Benchmarks

Table of Contents

• Unit tests (googletest)
• Getting started
• Coroutine environment
• Exception assertions
• Mocked time
• Mocked dynamic config
• Benchmarks (google-benchmark)
• Getting started
• Coroutine environment
• Mocked dynamic config

Unit tests (googletest)

Getting started

All userver test helpers live in utest CMake target:

target_link_libraries(your-test-target PRIVATE utest)

To include gtest and userver-specific macros, do:

#include <userver/utest/utest.hpp>

The header provides alternative gtest-like macros that run tests in a coroutine environment:

• UTEST(test_suite_name, test_name)
• UTEST_MT(test_suite_name, test_name, thread_count)
• UTEST_F(test_suite_name, test_name)
• UTEST_F_MT(test_suite_name, test_name, thread_count)
• UTEST_P(test_suite_name, test_name)
• UTEST_P_MT(test_suite_name, test_name, thread_count)
• UTEST_DEATH(test_suite_name, test_name)
• TYPED_UTEST(test_suite_name, test_name)
• TYPED_UTEST_MT(test_suite_name, test_name, thread_count)
• TYPED_UTEST_P(test_suite_name, test_name)
• TYPED_UTEST_P_MT(test_suite_name, test_name, thread_count)
• TYPED_UTEST_SUITE(test_suite_name, types)
• INSTANTIATE_UTEST_SUITE_P(prefix, test_suite_name, ...)
• REGISTER_TYPED_UTEST_SUITE_P(test_suite_name, ...)
• INSTANTIATE_TYPED_UTEST_SUITE_P(prefix, test_suite_name, types)
• TYPED_UTEST_SUITE_P(test_suite_name)

As usual, gmock is available in <gmock/gmock.h>.

See also utest::PrintTestName for info on how to simplify writing parametrized tests and official gtest documentation.

Coroutine environment

To test code that uses coroutine environment (e.g. creates tasks or uses synchronization primitives), use UTEST instead of TEST in the test header:

UTEST(Semaphore, PassAcrossCoroutines) {
  engine::Semaphore s{1};
  std::shared_lock guard{s};
 
  auto task = utils::Async("test", [guard = std::move(guard)] {});
  task.WaitFor(utest::kMaxTestWaitTime);
  EXPECT_TRUE(task.IsFinished());
}

There are U-versions for all gtest macros that declare tests and test groups: UTEST_F, UTEST_P, INSTANTIATE_UTEST_SUITE_P etc. Test fixture's constructor, destructor, SetUp and TearDown are executed in the same coroutine environment as the test body.

By default, a single-threaded TaskProcessor is used. It is usually enough, because userver engine enables concurrency even on 1 thread. However, some tests require actual parallelism, e.g. torture tests that search for potential data races. In such cases, use _MT macro versions:

UTEST_MT(SemaphoreLock, LockMoveCopyOwning, 2) {
  engine::Semaphore sem{1};
  engine::SemaphoreLock lock(sem);
  ASSERT_TRUE(lock.OwnsLock());
 
  engine::SemaphoreLock move_here{std::move(lock)};
  // NOLINTNEXTLINE(bugprone-use-after-move,clang-analyzer-cplusplus.Move)
  EXPECT_FALSE(lock.OwnsLock());
  EXPECT_TRUE(move_here.OwnsLock());
}

The specified thread count is available in U-tests as GetThreadCount() method.

For DEATH-tests (when testing aborts or assertion fails) use UTEST_DEATH. It configures gtest-DEATH-checks to work in multithreaded environment. Also it disables using of ev_default_loop and catching of SIGCHLD signal to work with gtest's waitpid() calls.

Exception assertions

Standard gtest exception assertions provide poor error messages. Their equivalents with proper diagnostics are available in <userver/utest/utest.hpp>:

• UEXPECT_THROW_MSG(statement, exception_type, message_substring)
• UASSERT_THROW_MSG(statement, exception_type, message_substring)
• UEXPECT_THROW(statement, exception_type)
• UASSERT_THROW(statement, exception_type)
• UEXPECT_NO_THROW(statement)
• UASSERT_NO_THROW(statement)

Example usage:

Mocked time

• To mock time, use utils::datetime::Now() and utils::datetime::SteadyNow() from <userver/utils/datetime.hpp> instead of std::chrono::system_clock::now() and std::chrono::steady_clock::now(), respectively
• Control the mocked time in tests using <userver/utils/mock_now.hpp>

class Timer final {
 public:
  // Starts the next loop and returns time elapsed on the previous loop
  std::chrono::system_clock::duration NextLoop() {
    const auto now = utils::datetime::Now();
    const auto elapsed = now - loop_start_;
    loop_start_ = now;
    return elapsed;
  }
 
 private:
  std::chrono::system_clock::time_point loop_start_{utils::datetime::Now()};
};
 
TEST(MockNow, Timer) {
  using utils::datetime::Stringtime;
 
  utils::datetime::MockNowSet(Stringtime("2000-01-01T00:00:00+0000"));
  Timer timer;
 
  utils::datetime::MockSleep(9001s);  // does NOT sleep, just sets the new time
  EXPECT_EQ(timer.NextLoop(), 9001s);
 
  utils::datetime::MockNowSet(Stringtime("2000-01-02T00:00:00+0000"));
  EXPECT_EQ(timer.NextLoop(), 24h - 9001s);
}

Mocked dynamic config

You can fill dynamic config with custom config values using dynamic_config::StorageMock.

class DummyClient;
 
std::string DummyFunction(const dynamic_config::Snapshot& config) {
  return config[kDummyConfig].bar;
}
 
UTEST(DynamicConfig, Snippet) {
  // The 'StorageMock' will only contain the specified configs, and nothing more
  dynamic_config::StorageMock storage{
      {kDummyConfig, {42, "what"}},
      {kIntConfig, 5},
  };
 
  EXPECT_EQ(DummyFunction(storage.GetSnapshot()), "what");
 
  // 'DummyClient' stores 'dynamic_config::Source' for access to latest configs
  DummyClient client{storage.GetSource()};
  UEXPECT_NO_THROW(client.DoStuff());
 
  storage.Extend({{kDummyConfig, {-10000, "invalid"}}});
  UEXPECT_THROW(client.DoStuff(), std::runtime_error);
}

If you don't want to specify all configs used by the tested code, you can use default dynamic config.

To use default dynamic config values in tests, add DEFAULT_DYNAMIC_CONFIG_FILENAME preprocessor definition to your test CMake target, specifying the path of a YAML file with dynamic_config::DocsMap contents.

Default dynamic config values can be accessed using <dynamic_config/test_helpers.hpp>:

• dynamic_config::GetDefaultSnapshot()
• dynamic_config::GetDefaultSource()
• dynamic_config::MakeDefaultStorage(overrides)

// Some production code
void MyHelper(const dynamic_config::Snapshot&);
 
class MyClient final {
  explicit MyClient(dynamic_config::Source);
};
 
// Tests
TEST(Stuff, DefaultConfig) {
  MyHelper(dynamic_config::GetDefaultSnapshot());
  MyClient client{dynamic_config::GetDefaultSource()};
}
 
TEST(Stuff, CustomConfig) {
  const auto config_storage = dynamic_config::MakeDefaultStorage({
      {kThisConfig, this_value},
      {kThatConfig, that_value},
  });
  MyHelper(config_storage.GetSnapshot());
  MyClient client{config_storage.GetSource()};
}

Benchmarks (google-benchmark)

Getting started

All userver benchmark helpers live in ubench CMake target:

target_link_libraries(your-bench-target PRIVATE ubench)

As usual, google-benchmark is available in <benchmark/benchmark.h>.

See also official google-benchmark documentation.

Coroutine environment

Use engine::RunStandalone to run parts of your benchmark in a coroutine environment:

void semaphore_lock(benchmark::State& state) {
  engine::RunStandalone([&]() {
    std::size_t i = 0;
    engine::Semaphore sem{std::numeric_limits<std::size_t>::max()};
 
    for (auto _ : state) {
      sem.lock_shared();
      ++i;
    }
 
    for (std::size_t j = 0; j < i; ++j) {
      sem.unlock_shared();
    }
  });
}
BENCHMARK(semaphore_lock);

Mocked dynamic config

See the equivalent utest section.

Default dynamic configs are available in <userver/dynamic_config/benchmark_helpers.hpp>.

---

Go backPeriodics and DistLocksUp nextFunctional service tests (testsuite)