[RFC] Technical proposal for fakeroot removal
by Max Rees
Hello,
As work towards the minimal viable product of APK Foundry nears
completion, I have been thinking of large scale trybuilds that can be
done in order to fully test its functionality. I think one such trybuild
that would also serve a functional purpose would be to begin work on
removing fakeroot(1) from the packaging process.
Most of the utility of using fakeroot in abuild currently is in order to
assign arbitrary user or group ownership to files before they are added
to the final .apk file. It does so by using an LD_PRELOAD hack to shadow
the libc's relevant functions, and uses a daemon to keep track of the
fake ownership of files.
As of this writing, there appears to be:
* 24 packages that use the $pkgusers and $pkggroups directives.
* 3 packages that only use the $pkggroups directive.
* 0 packages that only use the $pkgusers directive.
* 12 packages that use chown(1) or chgrp(1) in the APKBUILD, 8 of which
are not included in any of the above.
And an unknown amount of packages that perform their own chown or chgrp
during the vendor's installation process (this may later be estimated by
examining all available apks and checking for user, group != root).
In regards to a replacement, I think in absence of fakeroot we can
instead declaratively specify which files need special ownership
requirements through the use of libarchive's mtree(5) functionality. The
mtree file format is a BSD invention that describes the contents of a
tar file and allows for arbitrary specification of user and group
ownership as well as permissions, file type, etc.
By generating a de facto mtree for the package contents with
--uname root --gname root and allowing the packager to override this
mtree where necessary, I think the need for fakeroot can be mostly
eliminated.
This of course would have the side effect of tying abuild to the
libarchive implementation of bsdtar at runtime, unless we were to ship
our own utilities (built from libarchive I presume) to handle the
mtrees.
abuild would have an opaque shell function or similar that allows
specifying the file type, user ownership, group ownership, and
permissions for a file in the package. This function would then write to
a file in the control directory that contains the generated mtree
override(s), and merges that with the default root:root mtree before
using the merged mtree to build the data.tar.gz file.
For example, system/abuild itself needs to install /var/cache/distfiles
with abuild group ownership. The APKBUILD would have something like the
following instead of a chgrp:
> amtree -g abuild "$pkgdir"/var/cache/distfiles
The amtree function would translate this into an mtree entry and add it
to the $controldir/.mtree override file:
> var/cache/distfiles gname=abuild uname=root mode=775 type=dir
After merging the overrides with the default root:root mtree, libarchive
then supports reading an mtree file as a description from which to build
a tar file, like so:
> tar -cf data.tar.gz --uname=root --gname=root @$controldir/.mtree
I specifically think that the mtrees would have to be built on the fly
(not statically / version controlled) because some packages will of
course have filenames that differ between architectures. Thus if you
need an mtree for each (architecture + subpackage) combination, things
can quickly balloon.
To make this a gradual process we can also add some option to $options
if necessary.
Thoughts?
Max
10 months, 3 weeks
Handling cross-architecture installations with package scripts
by A. Wilcox
Hello all,
I'm trying to figure out the best way to handle cross-architecture
installations. Right now, I can create a pretty much bootable RPi 3
image with KDE using this command:
apk --arch aarch64 --root /path/to/sdcard -X
https://distfiles.adelielinux.org/adelie/1.0/system -X
https://distfiles.adelielinux.org/adelie/1.0/user --initdb add
adelie-base-posix kde x11
However, some packages fail to install properly if the host computer
isn't aarch64. Notably, this includes D-Bus, which needs to generate a
machine ID during installation:
== user/dbus/dbus.post-install ==
#!/bin/sh
exec dbus-uuidgen --ensure
This doesn't work because dbus-uuidgen is in the SD card root. It is an
ARM binary, not a PowerPC binary as my host system expects. So we end
up with: Exec format error.
It's possible to pass --no-scripts to apk to avoid the attempted
execution of these scripts, which would then prevent world from being
marked broken on the SD card. This would silently break D-Bus, though.
Alternatively, one could set up a oneshot service to run 'apk fix' on
first system boot. My concern is that it would have to pull the apks
off the network, which could be slow or even non-existent.
As you can see, neither of these options are very appealing.
Does anyone have any ideas on how we could handle this usage? One idea
I had was to add a --scripts-only flag to apk fix, to avoid redownload
and reinstall of the package and simply re-run any scripts that have
failed execution.
I suppose some scripts may need to be reworked to handle being called
multiple times, but this shouldn't be a large issue and is probably a
good idea from a correctness perspective anyway.
However, I'm open to other ideas. Let's discuss!
Best,
--arw
--
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org
1 year, 3 months