[RFC] Technical proposal for fakeroot removal

Sunday, 1 September 2019

Hello,

As work towards the minimal viable product of APK Foundry nears
completion, I have been thinking of large scale trybuilds that can be
done in order to fully test its functionality. I think one such trybuild
that would also serve a functional purpose would be to begin work on
removing fakeroot(1) from the packaging process.

Most of the utility of using fakeroot in abuild currently is in order to
assign arbitrary user or group ownership to files before they are added
to the final .apk file. It does so by using an LD_PRELOAD hack to shadow
the libc's relevant functions, and uses a daemon to keep track of the
fake ownership of files.

As of this writing, there appears to be:

* 24 packages that use the $pkgusers and $pkggroups directives.
* 3 packages that only use the $pkggroups directive.
* 0 packages that only use the $pkgusers directive.
* 12 packages that use chown(1) or chgrp(1) in the APKBUILD, 8 of which
  are not included in any of the above.

And an unknown amount of packages that perform their own chown or chgrp
during the vendor's installation process (this may later be estimated by
examining all available apks and checking for user, group != root).

In regards to a replacement, I think in absence of fakeroot we can
instead declaratively specify which files need special ownership
requirements through the use of libarchive's mtree(5) functionality. The
mtree file format is a BSD invention that describes the contents of a
tar file and allows for arbitrary specification of user and group
ownership as well as permissions, file type, etc.

By generating a de facto mtree for the package contents with
--uname root --gname root and allowing the packager to override this
mtree where necessary, I think the need for fakeroot can be mostly
eliminated.

This of course would have the side effect of tying abuild to the
libarchive implementation of bsdtar at runtime, unless we were to ship
our own utilities (built from libarchive I presume) to handle the
mtrees.

abuild would have an opaque shell function or similar that allows
specifying the file type, user ownership, group ownership, and
permissions for a file in the package. This function would then write to
a file in the control directory that contains the generated mtree
override(s), and merges that with the default root:root mtree before
using the merged mtree to build the data.tar.gz file.

For example, system/abuild itself needs to install /var/cache/distfiles
with abuild group ownership. The APKBUILD would have something like the
following instead of a chgrp:

...
 amtree -g abuild "$pkgdir"/var/cache/distfiles 
The amtree function would translate this into an mtree entry and add it
to the $controldir/.mtree override file:

...
 var/cache/distfiles gname=abuild uname=root mode=775 type=dir 
After merging the overrides with the default root:root mtree, libarchive
then supports reading an mtree file as a description from which to build
a tar file, like so:

...
 tar -cf data.tar.gz --uname=root --gname=root @$controldir/.mtree

I specifically think that the mtrees would have to be built on the fly
(not statically / version controlled) because some packages will of
course have filenames that differ between architectures. Thus if you
need an mtree for each (architecture + subpackage) combination, things
can quickly balloon.

To make this a gradual process we can also add some option to $options
if necessary.

Thoughts?

Max

2021

2020

2019

2018