Testing...Testing...1,2,3

Who tests the unit tests? – Anon

Ever since I was introduced to the concept of Test Driven Development (TDD) and
Unit Testing back in 2006, I fell in love with it. So much that it got me fired
from my first project and later on to get an outstanding employee award in the
second project.

Most of that testing was done in Microsoft .Net (nUnit) and C++ (using custom
build framework at the time) and more recently with jUnit and it’s relatives in
Java during my short tryst with Android development. But those were real easy
compared to how one tests with ATF.

One of the reasons I probably found this a bit overwhelming is because of my
lack of exposure to a large system with collections of multiple interdependent
and interconnecting components which deals with the most basic fundamentals of
running an operating system. Application level testing like an Android
Application or .Net one for that matter assumes you have a base operating system
up and running be it Windows / Linux / BSD or some variant and your tests
generally fall within the limits of the application which you are testing which
are pretty much well defined business logic from a design specification of the
application.

Unit testing / Functional testing for an operating system and it’s most
fundamental components like uvm (the virtual memory manager) was clearly out of
scope to me when I was first given the task of this project. Cherry (cherry@)
approach of exposing the uvm_physseg.c to the userland was definitely one step
towards doing unit / functional testing, but this was not enough. What else is
needed to make the circle of testing complete?

After reading through the code in uvm_physseg.c I came across the following
facts to differentiate between unit testing and functional testing in
the ATF environment.

• Functions exposed directly were subjected to testing.
• Static functions were not tested.
• Test nomenclature was simple, you give the name of the function as the name of
  your test. For example if you were testing the function uvm_physseg_delete()
  then the test name would be uvm_physseg_delete.
• Unit Test nomenclature had the word unit prepended to the function
  name. In the above example if you were writing the tests for
  uvm_phsseg_delete() then the test name would be unit_uvm_physseg_delete.

The differentiation between a unit and functional test was simple, in a Unit
test you setup the variables / parameters you need to pass into the function (if
any) or the function uses / requires ahead of the function call and then invoke
the function and test for conditions. In a functional test you call the function
to be tested in the natural way it is supposed to be invoked, meaning you setup
all the preconditions upto the funtion call like you would expect to happen if
the system were running in actual production. This means functional test cases
are allowed to have multiple calls to other dependent function calls before the
invocation of the function which is being tested.

An example showing the unit test variant vs the functional test variant of the
same function witin uvm_physseg.c

ATF_TC(unit_uvm_physseg_start);
ATF_TC_HEAD(unit_uvm_physseg_start, tc)
{
	atf_tc_set_md_var(tc, "descr", "unit test for working of uvm_physseg_start()");
}
ATF_TC_BODY(unit_uvm_physseg_start, tc)
{
	struct uvm_physseg *seg; /* A test segment */

	/* Allocate valid memory for each of the segments */
	seg = malloc(sizeof(struct uvm_physseg));

	ATF_REQUIRE(seg != NULL);

	seg->start = START_TESTS_START;

	ATF_CHECK_EQ(seg->start, uvm_physseg_start(seg));
}

ATF_TC(uvm_physseg_start);
ATF_TC_HEAD(uvm_physseg_start, tc)
{
	atf_tc_set_md_var(tc, "descr", "test for checking the start address of a valid segment");
}
ATF_TC_BODY(uvm_physseg_start, tc)
{
	/* A test Segment */
	struct uvm_physseg *seg;

	sfn_t start, end, avail_start, avail_end;

	/* props[] check - single item */
	struct uvm_physseg_prop props[UVM_PHYSSEG_TYPE_COUNT] = {
		{
			UVM_PHYSSEG_DMA,
			{
				UVM_PHYSSEG_DMA_CAT1
			}
		}
	};

	start = avail_start = VALID_START_1;
	end = avail_end = VALID_END_1;

	seg = uvm_physseg_plug(start, end,
	    avail_start, avail_end, props);

	ATF_REQUIRE(seg != NULL);

	ATF_REQUIRE(seg->pgs != NULL);

	ATF_CHECK_EQ(VALID_START_1, uvm_physseg_start(seg));
}