Testing...Testing...1,2,3

Who tests the unit tests? – Anon

Ever since I was introduced to the concept of Test Driven Development (TDD) and Unit Testing back in 2006, I fell in love with it. So much that it got me fired from my first project and later on to get an outstanding employee award in the second project.

Most of that testing was done in Microsoft .Net (nUnit) and C++ (using custom build framework at the time) and more recently with jUnit and it’s relatives in Java during my short tryst with Android development. But those were real easy compared to how one tests with ATF.

One of the reasons I probably found this a bit overwhelming is because of my lack of exposure to a large system with collections of multiple interdependent and interconnecting components which deals with the most basic fundamentals of running an operating system. Application level testing like an Android Application or .Net one for that matter assumes you have a base operating system up and running be it Windows / Linux / BSD or some variant and your tests generally fall within the limits of the application which you are testing which are pretty much well defined business logic from a design specification of the application.

Unit testing / Functional testing for an operating system and it’s most fundamental components like uvm (the virtual memory manager) was clearly out of scope to me when I was first given the task of this project. Cherry (cherry@) approach of exposing the uvm_physseg.c to the userland was definitely one step towards doing unit / functional testing, but this was not enough. What else is needed to make the circle of testing complete?

After reading through the code in uvm_physseg.c I came across the following facts to differentiate between unit testing and functional testing in the ATF environment.

• Functions exposed directly were subjected to testing.
• Static functions were not tested.
• Test nomenclature was simple, you give the name of the function as the name of your test. For example if you were testing the function uvm_physseg_delete() then the test name would be uvm_physseg_delete.
• Unit Test nomenclature had the word unit prepended to the function name. In the above example if you were writing the tests for uvm_phsseg_delete() then the test name would be unit_uvm_physseg_delete.

The differentiation between a unit and functional test was simple, in a Unit test you setup the variables / parameters you need to pass into the function (if any) or the function uses / requires ahead of the function call and then invoke the function and test for conditions. In a functional test you call the function to be tested in the natural way it is supposed to be invoked, meaning you setup all the preconditions upto the funtion call like you would expect to happen if the system were running in actual production. This means functional test cases are allowed to have multiple calls to other dependent function calls before the invocation of the function which is being tested.

An example showing the unit test variant vs the functional test variant of the same function witin uvm_physseg.c

ATF_TC(unit_uvm_physseg_start);
ATF_TC_HEAD(unit_uvm_physseg_start, tc)
{
	atf_tc_set_md_var(tc, "descr", "unit test for working of uvm_physseg_start()");
}
ATF_TC_BODY(unit_uvm_physseg_start, tc)
{
	struct uvm_physseg *seg; /* A test segment */

	/* Allocate valid memory for each of the segments */
	seg = malloc(sizeof(struct uvm_physseg));

	ATF_REQUIRE(seg != NULL);

	seg->start = START_TESTS_START;

	ATF_CHECK_EQ(seg->start, uvm_physseg_start(seg));
}

ATF_TC(uvm_physseg_start);
ATF_TC_HEAD(uvm_physseg_start, tc)
{
	atf_tc_set_md_var(tc, "descr", "test for checking the start address of a valid segment");
}
ATF_TC_BODY(uvm_physseg_start, tc)
{
	/* A test Segment */
	struct uvm_physseg *seg;

	sfn_t start, end, avail_start, avail_end;

	/* props[] check - single item */
	struct uvm_physseg_prop props[UVM_PHYSSEG_TYPE_COUNT] = {
		{
			UVM_PHYSSEG_DMA,
			{
				UVM_PHYSSEG_DMA_CAT1
			}
		}
	};

	start = avail_start = VALID_START_1;
	end = avail_end = VALID_END_1;

	seg = uvm_physseg_plug(start, end,
	    avail_start, avail_end, props);

	ATF_REQUIRE(seg != NULL);

	ATF_REQUIRE(seg->pgs != NULL);

	ATF_CHECK_EQ(VALID_START_1, uvm_physseg_start(seg));
}