What happens in userspace stays in userspace! – Tod McQuillin

This post is more about a testing strategy that was adopted by Cherry (cherry@)
to test Kernel Space code in Userspace which I would like to formalize in this
blog post.

So basically the question is how exactly do you take a part of the segregated
API from uvm and then test that specific section alone using ATF. And by testing
I mean writing Unit Tests or Functional Tests without actually testing how the
changes affect the whole system (i.e. Integration Testing) for example via Rump.

Cherry, when he did the design for the new API applied some object orientation
techniques to the code structure which helped me to write the tests quite
easily. Here are some of the notable points

  • Keep structures that you do not wish to expose globally to the users in a .c
    file. This attempts to let know the future consumers of the API to make sure
    that they do not directly modify structures that are otherwise not exposed in
    the .h file. Note: Of course it does not prevent a user from doing otherwise,
    but then again we expect the users who would want to modify / re-write APIs like
    this to be sufficiently disciplined to not do Rambo style coding without a valid
    reason.

  • The .h file nicely exposes all the “valid” operations that can be done on
    the various opaque structures that is used in this API. In terms of Object
    Orientied concepts these would be getters / setters etc along with the
    functionality the API can provide.

Now with these in place, there were couple more tricks done by Cherry that enabled
the testing of the API in ATF userspace.

  • Include the .c file as part of the ATF Test.

    This makes the entire API exposed to User land, and enables isolated testing of the
    various functions exposed in the .h file.

    However just including the .c file is just the start of things, after this comes
    the hard part of making the Test compile.

  • Segregate the machine-dependent (MD) part and the machine-independent (MI) part

    This has a neat trick of having all the MI part being implemented in the corresponding
    header (.h) file but, all the MD part going in to #include<machine/uvm_physseg.h>
    header which is included from the corresponding sys/arch/<machine>/include
    directory.

    A separate machine/uvm_physseg.h is maintained in the ATF tests folder to prevent
    build breakage.

  • Stub / Re-Implement Kernel API calls

    This is important, since many of the includes in uvm for example uses functions
    that is only accessible when running within the Kernel for example kmem_alloc()
    or kmem_free() and these cannot be called from witin userland, so one way we
    imeplemented this was to have a separate include that will have these functions
    into the MD header file so that they do not get included in the ATF tests causing
    build errors. Then re-implement these functions with wrappers that do similar things
    in the userland for example malloc() or free(). Of course this is only a work
    around, but it does provide sufficient isolation for unit / functional testing.

  • Stub / Re-Implement Dependent API calls

    Very much like above, the file we are including may be dependent on other APIs
    that the system uses for example uvm_physseg.c depends on certain calls from
    uvm_page.c now the question here is just stub them or do we need to re-implement
    the whole function, my personal take on this is stub them and if your testing
    depends on return values that are returned by these calls, then you may need a
    minimalstic re-implementation just enough so that your tests can get through them.

So basically that is, that is how we managed to get isolation while doing these
unit / functional tests.