# Unit Testing¶

This section describes how the Kodo unit tests work, how to run them and how to extend them.

## Overview¶

The purpose of the Kodo unit tests is to assert that the various Kodo features work across a number of different platforms and compilers. Having an extensive unit test suite makes it easier for developers to ensure that their changes work as expected and do not cause regressions in unexpected areas of the library.

The goal is that all features in Kodo should have a corresponding test case. Often a test case is added while when implementing a new feature. This makes it possible to assert that the new functionality works as expected as early as possible.

The unit tests are implemented using the Google C++ Testing Framework (gtest) which defines a bunch of helpers useful when writing tests. You can find more information on gtest on their homepage.

## Running the tests¶

One of the first things you might want to try is to run the Kodo unit tests on your own machine. There are two ways to do this:

### Run the test binary manually¶

The test binary is built using the waf build scripts shipped as part of the Kodo source code. You can read more about how you can get the source code and how to build it in the Getting Started section.

Once the code is built, the test binary will be located in a subfolder of the build folder that depends on your platform:

Linux
build/linux/test/kodo_tests
Mac OSX
build/darwin/test/kodo_tests
Windows
build/win32/test/kodo_tests.exe

If you are cross-compiling with an mkspec (as described in the Cross-compilation and Tool Options section), then the resulting binary will be located in:

mkspec
build/[mkspec]/test/kodo_tests

Note

Running the unit tests may take a long time on mobile and embedded platforms, since we test with an extensive set of parameters. You can lower the complexity and speed up the tests if you invoke the test binary with the embedded profile:

./kodo_tests --profile=embedded


### Run the test as part of the build¶

In some cases it is convenient to run the test binary as part of a build. This can be done by passing the following option to waf:

python waf --run_tests


When adding a new feature to Kodo it is a good idea to also add a corresponding unit test. The source code for the different unit tests are placed in the test/src folder of the Kodo project.

All files with a .cpp file extension in the test/src will automatically be included in the test executable produced when building Kodo with waf.

In general we follow these guidelines regarding the unit tests:

1. Every class should have a corresponding unit test cpp file.
2. Remember to place the test file as described in Namespaces and directories

The purpose of this is to make it easy to find the unit test for a specific class. In some cases it makes sense to have multiple classes tested in the same file. In those cases we still make a placeholder cpp file referring to the actual cpp file where the test can be found. An example of this can be seen for some of the codecs e.g. the class full_rlnc_encoder located in src/kodo/rlnc/full_rlnc_encoder.hpp is tested in full_rlnc_codes.cpp but the place-holder still exists.

The placeholder file in this cases (test/src/test_full_rlnc_encoder.cpp) looks like the following:

  1 2 3 4 5 6 7 8 9 10 // Copyright Steinwurf ApS 2011. // Distributed under the "STEINWURF RESEARCH LICENSE 1.0". // See accompanying file LICENSE.rst or // http://www.steinwurf.com/licensing #include /// @file test_full_vector_encoder.cpp The unit tests for the /// full_vector_encoder stack are defined in /// test_full_vector_codes.cpp 

Once the .cpp test file has been created we can start to implement the unit test code. This is done with the help of the gtest framework.

### Dealing with type aliases¶

In some cases we have headers containing only type-aliases such as using statements. We currently do not require that these are explicitly unit tested.

Regardless of whether a unit test is implemented or not we still leave a .cpp file in the test folder.

An example is src/kodo/partial_mutable_shallow_storage_layers.hpp which only contains a single using declaration. It does not have any explicit unit tests but a place holder file is still created (test/src/test_mutable_partial_shallow_symbol_storage.cpp):

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 // Copyright Steinwurf ApS 2015. // Distributed under the "STEINWURF RESEARCH LICENSE 1.0". // See accompanying file LICENSE.rst or // http://www.steinwurf.com/licensing /// @file test_mutable_partial_shallow_symbol_storage.cpp Unit tests for the /// mutable_partial_shallow_symbol_storage class #include #include #include #include #include #include #include namespace kodo_core { // Put dummy layers and tests classes in an anonymous namespace // to avoid violations of ODF (one-definition-rule) in other // translation units namespace { /// Helper layer class dummy_layer { public: template void construct(Factory& the_factory) { (void) the_factory; m_construct(); } template void initialize(Factory& the_factory) { m_initialize(); symbol_size.set_return(the_factory.symbol_size()); } stub::function m_initialize; stub::function m_construct; stub::function block_size; stub::function symbol_size; stub::function symbols; stub::function set_mutable_symbol; stub::function set_mutable_symbols; }; // Helper stack class dummy_stack : public mutable_partial_shallow_symbol_storage { }; // Helper factory class dummy_factory { public: stub::function max_symbol_size; stub::function symbol_size; }; } } // Test that the stack functions properly when the partial symbol is // not needed TEST(test_mutable_partial_shallow_symbol_storage, no_partial) { kodo_core::dummy_factory factory; factory.max_symbol_size.set_return(10U); uint32_t symbol_size = 5U; factory.symbol_size.set_return(symbol_size); kodo_core::dummy_stack stack; kodo_core::dummy_layer& layer = stack; stack.construct(factory); stack.initialize(factory); // This is initialize in the dummy layer using the factory EXPECT_EQ(stack.symbol_size(), 5U); // Check state is correct and that initialize and construct calls // were correctly forwarded EXPECT_TRUE(layer.m_construct.expect_calls().with().to_bool()); EXPECT_TRUE(layer.m_initialize.expect_calls().with().to_bool()); EXPECT_FALSE(stack.has_partial_symbol()); // Set the remaining needed state in the stack uint32_t symbols = 5U; layer.symbols.set_return(symbols); uint32_t block_size = 25U; layer.block_size.set_return(block_size); // Make a buffer that perfectly fits 5 symbols of size 5 bytes std::vector data(block_size); stack.set_mutable_symbols(storage::storage(data)); EXPECT_FALSE(stack.has_partial_symbol()); EXPECT_EQ(0U, layer.set_mutable_symbol.calls()); EXPECT_TRUE(layer.set_mutable_symbols.expect_calls() .with(storage::storage(data)) .to_bool()); } // Test that the stack functions properly when the partial symbol is // needed TEST(test_mutable_partial_shallow_symbol_storage, partial) { kodo_core::dummy_factory factory; factory.max_symbol_size.set_return(10U); uint32_t symbol_size = 5U; factory.symbol_size.set_return(symbol_size); kodo_core::dummy_stack stack; kodo_core::dummy_layer& layer = stack; stack.construct(factory); stack.initialize(factory); // This is initialize in the dummy layer using the factory EXPECT_EQ(stack.symbol_size(), 5U); // Check state is correct and that initialize and construct calls // were correctly forwarded EXPECT_TRUE(layer.m_construct.expect_calls().with().to_bool()); EXPECT_TRUE(layer.m_initialize.expect_calls().with().to_bool()); EXPECT_FALSE(stack.has_partial_symbol()); // Set the remaining needed state in the stack uint32_t symbols = 5U; layer.symbols.set_return(symbols); uint32_t block_size = 25U; layer.block_size.set_return(block_size); // Make a buffer that is 23 bytes. That means we will have 4 // symbols of 5 bytes and one symbol which only be 3 bytes. std::vector data(block_size - 2U, 'a'); stack.set_mutable_symbols(storage::storage(data)); EXPECT_TRUE(stack.has_partial_symbol()); // To check that the calls made to the set_mutable_symbols functions // are what we expect we need a custom predicate function to // compare the arguments. The reason is that we need to use the // is_same function to compare the storage::mutable_storage // objects. is_same compares that the pointers point to the same // memory as opposed to just checking whether the content is equal using parameter = std::tuple; auto compare = [](const parameter& a, const parameter& b) -> bool { if (std::get<0>(a) != std::get<0>(b)) return false; if (storage::is_same(std::get<1>(a), std::get<1>(b))) return true; return false; }; EXPECT_EQ(layer.set_mutable_symbol.calls(), 5U); // The actual calls that were made auto zero = std::make_tuple(0, storage::mutable_storage(&data[0], 5U)); auto one = std::make_tuple(1, storage::mutable_storage(&data[5], 5U)); auto two = std::make_tuple(2, storage::mutable_storage(&data[10], 5U)); auto three = std::make_tuple(3, storage::mutable_storage(&data[15], 5U)); auto four = std::make_tuple(4, storage::mutable_storage(&data[20], 5U)); EXPECT_TRUE(compare(layer.set_mutable_symbol.call_arguments(0), zero)); EXPECT_TRUE(compare(layer.set_mutable_symbol.call_arguments(1), one)); EXPECT_TRUE(compare(layer.set_mutable_symbol.call_arguments(2), two)); EXPECT_TRUE(compare(layer.set_mutable_symbol.call_arguments(3), three)); EXPECT_FALSE(compare(layer.set_mutable_symbol.call_arguments(4), four)); } 

## Example unit test¶

The Kodo library is build using the parameterized-inheritance/ mixin-layers C++ design technique. When unit testing a layer we try to isolate it as much as possible. To do this we typically introduce dummy layers with the sole purpose of satisfying the layer’s dependencies. To see this in action let’s look at one of the existing unit tests.

The storage_bytes_used layer is used when we want add functionality allowing us to keep track of how many useful bytes an encoder or decoder contains.

Note

In general the amount of data which can be encoded or decoded will be determined by the number of symbols we are coding and the size of every symbol in bytes (we call this the block size). However, in practical applications we sometimes do not have enough data to fill an entire block. In those cases we can add the storage_bytes_used layer to embed in every encoder and decoder the ability to store the number of actual data bytes.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 // Copyright Steinwurf ApS 2011. // Distributed under the "STEINWURF RESEARCH LICENSE 1.0". // See accompanying file LICENSE.rst or // http://www.steinwurf.com/licensing #pragma once #include #include namespace kodo_core { /// @ingroup storage_info_layers /// /// @brief Provides access to the number of useful bytes used out /// of the total size of the encoders or decoders storage. template class storage_bytes_used : public SuperCoder { public: /// Constructor storage_bytes_used() : m_bytes_used(0) {} /// @copydoc layer::initialize(Factory&) template void initialize(Factory& the_factory) { SuperCoder::initialize(the_factory); m_bytes_used = 0; } /// @copydoc layer::set_bytes_used(uint32_t) void set_bytes_used(uint32_t bytes_used) { assert(bytes_used > 0); assert(bytes_used <= SuperCoder::block_size()); m_bytes_used = bytes_used; } /// @copydoc layer::bytes_used() const uint32_t bytes_used() const { return m_bytes_used; } protected: /// The number of bytes used uint32_t m_bytes_used; }; } 

As seen, the layer depends on two functions being provided by the SuperCoder:

1. SuperCoder::initialize(the_factory)
2. SuperCoder::block_size()

Using our Doxygen documentation, it is possible to look up the purpose of the two undefined functions.

In this case we want to check that the state is correctly updated when calling set_bytes_used and that the state is correctly reset when calling initialize. The following unit test code was implemented in test/src/test_storage_bytes_used.cpp to test this:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 // Copyright Steinwurf ApS 2011. // Distributed under the "STEINWURF RESEARCH LICENSE 1.0". // See accompanying file LICENSE.rst or // http://www.steinwurf.com/licensing /// @file test_storage_bytes_used.cpp Unit tests for the /// storage_bytes_used class #include #include #include #include namespace kodo_core { // Put dummy layers and tests classes in an anonymous namespace // to avoid violations of ODF (one-definition-rule) in other // translation units namespace { /// Helper layer class dummy_layer { public: template void initialize(Factory& the_factory) { (void) the_factory; m_initialize(); } uint32_t block_size() const { return m_block_size(); } stub::function m_initialize; stub::function m_block_size; }; // Helper stack class dummy_stack : public storage_bytes_used { }; // Helper factory class dummy_factory { }; } } TEST(test_storage_bytes_used, api) { kodo_core::dummy_factory factory; kodo_core::dummy_stack stack; stack.m_block_size.set_return(10U); stack.initialize(factory); stack.set_bytes_used(9U); EXPECT_EQ(stack.m_initialize.calls(), 1U); EXPECT_TRUE(stack.m_block_size.expect_calls().with().to_bool()); EXPECT_EQ(stack.bytes_used(), 9U); // Initialize again and check that the state is reset stack.initialize(factory); EXPECT_EQ(stack.m_initialize.calls(), 2U); EXPECT_TRUE(stack.m_block_size.expect_calls().with().to_bool()); EXPECT_EQ(stack.bytes_used(), 0U); } 

In the above test code we use one test helper which allows use to easily add testing stubs to the unit test.

http://en.wikipedia.org/wiki/Test_stub

The library is called stub and is freely available under the BSD license:

https://github.com/steinwurf/stub

## Naming the test case¶

When we define a test using gtest we use the TEST(test_case_name, test_name) macro to define and name a test function. In Kodo we use the following naming guideline:

1. The test_case_name should be name according to its placement in the test/src directory. If the file is place in the root of the test/src folder e.g. test/src/test_my_fancy_code.cpp we name the test_case_name as test_my_fancy_code. Similarly if the file is placed in a subdirectory e.g. object/test_new_code.cpp we will specify the test_case_name as object_test_new_code. This should make it easy to find the source code of a failing unit test. Remember to place the test file as described in Namespaces and directories
2. The test_name is up to the developer but should be as descriptive of the purpose of the unit test as possible.