Writing Ports and Packages

After my recent encounter with latex-beamer to create presentations, I came
across an utility called dspdfviewer[1] which is very useful when displaying the
slides + speaker notes created by latex-beamer in a dual screen setup. I
primarily use Windows as my daily driver operating system, so for me fetching
dspdfviewer executable for windows was pretty much easy. But after my
presentation in AsiaBSDCon 2017, some of the people in the audience was
interested in the tool I used to do my presentation. It was when I was
explaining about dspdfviewer, we discovered that dspdfviewer lacked a package in
NetBSD pkgsrc. After exiting the conference room, I was talking with trouble
(philip@) who told me that dspdfviewer was missing from ports tree in FreeBSD
too. And that is how this story begins.

Writing the FreeBSD Port

Once I got back to Bangalore, I was working on writing the FreeBSD ports script
for dspdfviewer. The script for dspdfvewer was slightly different from the
irc/irssi-icb port since it uses cmake for generating the build files. Add
to that, there were multiple dependencies on dspdfviewer, mainly boost-libs
and libpoppler-qt5 which in turn had other dependencies. After some
flippingaround with the Porter’s Handbook and some trial and error I was able to
figure out how to get dspdfviewer built using the FreeBSD Ports system. And
overall the difficulty was not too much thanks to a whole section[2] dedicated
to using cmake in the Porter’s Handbook.

I put the package under graphics/dspdfviewer, since I found other similar PDF
reader applications put there. After the usual round of make install clean
tests in trouble’s server and a run through portlint, I wrapped it up nicely
and put it up in github[3].

Writing the NetBSD pkgsrc

It is interesting to note that, NetBSD’s pkgsrc was forked off from the ports
tree in FreeBSD back in the olden days. Since this was my first time writing
pkgsrc scripts, I was expecting to be on familiar grounds since I managed to
write up couple ports for FreeBSD. And this was going to be the bane of my
existance for the next couple of days.

There was a slight catch in writing of the pkgsrc scripts for dspdfviewer. I did
not have direct access to a NetBSD machine, so I installed pkgsrc in the FreeBSD
server that I had access to. This process was probably the easiest. And things
started to get difficult from here.

Once I had the basic pkgsrc setup and the related tools setup, as with Ports, my
initial steps were to go through the “pkgsrc guide”[4] and follow instructions
from there. Following the guide I tried to use url2pkg to create a boilerplate
Makefile for dspdfviewer. But this was not too satisfactory since the original
program is hosted in github. So searching through the pkgsrc guide, I came
across a section[5] which specifically dealt with fetching packages from
github. So after some tinkering around with variables and moving them around in
the file, I was able to do a successful fetch of the source file from github.

Fetching the file was one thing, but getting it to build was a different story
altogether. Since there was a dependency on cmake for dspdfviewer, I was on
the lookout for the cmake section equivalent from the Porter’s Handbook, but
alas all I could find was one “paragraph” that was dedicated to cmake which
was specifically in section 17.11[6]. But this was not too helpful, I could not
exactly determine why the build was failing since cmake was not running in
verbose mode. After asking around in the NetBSD ICB (coming to think of it, I
should have asked this question in #pkgsrc in irc) and not getting any helpful
answers, it was time to grep around the code base to check for the string
“verbose”. Some of the results showed me BUILD_MAKE_FLAGS+= VERBOSE=1 which I
just added to the Makefile and voila, I have verbose cmake output.

After going through the build logs (which I have not shown here for the sake of
brevity), I figured out that cmake was not running the correct folder. In
order to understand what is happening here, we should look at how dspdfviewer is
expected to be build from source. The cmake needs to be run in a folder called
“build” which should reside within dspdfviewer’s source tree. Now the FreeBSD
ports was aware of how do this because I could set the BUILD_WRKSRC and
CONFIGURE_WRKSRC to point to ${WRKSRC}/build which will cause the ports
build process to make a directory called “build” within the source directory and
start executing the commands there. But I could not find an equivalent
configuration variable in pkgsrc. After some more time of probing the
documentation, specifically the build process section. I found out about hooks
within the system that divides the build process into various “pre-“, “post-“
and “do-“ stages. Using this new found knowledge, I was able to create the
“build” directory using the pre-configure stage. In the dspdfviewer project,
running cmake is the configure stage. And I add following lines towards the
end of the Makefile.

pre-configure:
        mkdir -p ${WRKSRC}/build

Finally got a “build” directory inside the source directory, yet the build was
failing. Further inspection revealed that, cmake was not being run from within
the “build” directory, Sigh!! So after more probing around I found that
do-configure is the place to manually run the cmake and wrote up the
following.

do-configure:
        cd ${CONFIGURE_DIR} && cmake .. ${CMAKE_ARGS}

And fingers crossed, I ran the build again, only to fail in a dependency build
specifically the package x11/qt5-qtbase, looking at the build log I could see
that the linker found an undefined reference to kinfo_getproc looking at the
man pages, I figured out that this was due to a missing flag -lutil. Adding
this flag seems to have fixed the build for x11/qt5-qtbase so I wrote up a
PR #53281[7] in NetBSD Website. By the time I am writing this post the bug was
accepted and marked as fixed.

Anyway continuing from this point, this time it worked upto the linking point of
dspdfviewer and failed to link the boost-libs to the dspdfviewer
executable. And then I figured out with the help of cherry (cherry@) that I
forgot to include the boost-libs at the bottom of the Makefile. DOH!!

.include "../../devel/boost-libs/buildlink3.mk"

Time again to do the build, break, repeat cycle. But this time the
build succeeded. Then came up the clean up duty, running bmake install clean
making sure nothing is breaking down. After some clean up fixes based on the
output of pkglint the equivalent of portlint from FreeBSD, I put the package
under print/dspdfviewer, wrapped it all up nicely and pushed it into
github[8].

Some breakages

Since the FreeBSD ports version of dspdfviewer was ready, trouble helped put it
up in the ports tree. There were a couple of bugs that came up after the commit.

• manpage location - dspdfviewer by default installs man pages to
  share/man/man1 while this may be right for some operating systems, in BSD
  land this is put in just man/man1 so I had to patch the CMakeLists.txt
  which was a simple enough job.

• Build breakage in llvm-4.0 - This was more of a challenging task, so from
  llvm-4.0 a new warning is generated for “undefined function template”
  -Wundefined-func-template but this did not exist prior to this specific
  llvm-4.0 version. Which meant that the build would not break in older
  versions, but would break in any version starting from this
  onwards. Considering this a new change was made in the script to account for
  this condition.

  .if ${OPSYS} == FreeBSD && ${OSVERSION} >= 1200023
  CFLAGS+=        -Wno-error=undefined-func-template
  .endif
  

Conclusion

Once again thanks to trouble for helping me commit the ports version of
dspdfviewer, also for helping me with doing fixes in the scripts and extensive
build testing via poudriere.

The pkgsrc version of dspdfviewer has not been committed, this is because I have
not had a chance to test the scripts in a proper NetBSD machine. I should
probably talk about this to someone in the pkgsrc IRC channel.

References

1. dspdfviewer Hompeage - http://dspdfviewer.danny-edel.de/
2. FreeBSD Porter’s Handbook (Building Mechanisms) - https://www.freebsd.org/doc/en/books/porters-handbook/building.html
3. dspdfviewer FreeBSD ports script - https://github.com/fraggerfox/dspdfviewer-port
4. The pkgsrc Guide - https://www.netbsd.org/docs/pkgsrc/index.html
5. The pkgsrc Guide (Github hosted packages) - http://www.jp.netbsd.org/docs/pkgsrc/pkgsrc.html#build.fetch.github
6. The pkgsrc Guide (The configure phase) - http://www.jp.netbsd.org/docs/pkgsrc/pkgsrc.html#build.configure
7. NetBSD Problem Report #52181 - http://gnats.netbsd.org/52181
8. dspdfviewer NetBSD pkgsrc script - https://github.com/fraggerfox/dspdfviewer-pkgsrc
9. dsdpfviewer in Freshports - https://www.freshports.org/graphics/dspdfviewer