It always bugged me a bit that the uvm_physseg_handle_immutable test failed
when it was run under non hotplug environment. Of course this was the “expected”
outcome, but rather than showing a hard fail on the test, was there a way
for me to indicate that the test is expected to fail under these conditions.

And as always my impatient and overlooking nature got the best of me. I had over
looked the sections in the ATF Tutorial page[1] that describes this exact
situation, there is a function doing exactly this.

atf_expect_fail(const char format, …): Tells the atf runtime that the
code following this call is expected to raise one or more failures (be it with
atf_tc_fail, ATF_REQUIRE_, etc.). Use this to mark a block of code that is
known to be broken (e.g. a test that reproduces a known bug). Use the string
parameter to provide an explanation about why the code is broken; if possible,
provide a PR number. Lastly, to terminate the “expected failure” code block
and reset the runtime to the default functionality, use the atf_expect_pass()
function.

And the variation of the above

atf_expect_signal(int signo, const char *format, …): Same as atf_expect_fail
but expects the test case to receive a specific signal. Provide -1 to
indicate any signal.

These two functions were a boon, now in light of this new information, I was
able to re-write the the original test with the following changes

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

diff --git a/tests/sys/uvm/t_uvm_physmem.c b/tests/sys/uvm/t_uvm_physmem.c
index 6567955..8b2c675 100644
--- a/tests/sys/uvm/t_uvm_physmem.c
+++ b/tests/sys/uvm/t_uvm_physmem.c
@@ -189,7 +189,6 @@ uvmpdpol_reinit(void)

 /* end - Provide Kernel API equivalents */

-
 #include "uvm/uvm_physmem.c"

 #include <atf-c.h>
@@ -653,10 +652,13 @@ ATF_TC_BODY(uvm_physmem_handle_immutable, tc)
        /* Fetch Previous, we inserted a lower value */
        upm = uvm_physmem_get_prev(upm);

+#if !defined(UVM_HOTPLUG)
        /*
         * This test is going to fail for the Array Implementation but is
         * expected to pass in the RB Tree implementation.
         */
+       atf_tc_expect_fail("Mutable handle in static array impl.");
+#endif
        ATF_CHECK(UVM_PHYSMEM_TYPE_INVALID_EMPTY != upm);
        ATF_CHECK_EQ(VALID_START_PFN_1, uvm_physmem_get_start(upm));
        ATF_CHECK_EQ(VALID_END_PFN_1, uvm_physmem_get_end(upm));

Now the test can fail as “expected” but not be counted as a hard fail, but as
a failure within the expected test environment.

References

  1. https://wiki.netbsd.org/tutorials/atf/