On Sep 05 08:55 PM, A. Wilcox wrote:
> On 09/05/18 15:46, Max Rees wrote:
> > I believe that as it stands right now (I cannot check it at the moment), my
> > abuildd-enqueue implementation does not distinguish between new commits added
> > to an existing MR, a force push over an existing MR, or a completely new MR.
> > Thus all added/changed (NB: *not* deleted) packages for a MR would be rebuilt
> > on each change to the MR.
>
> That is probably for the best, as a MR is going to be a single unit of
> work. I can understand that if, say, XFCE or elementary was added as a
> MR, there might be a lot of busywork if a single package is changed.
> But there could be non-obvious (or subtly non-declared) dependencies
> between them all, which is why I think a clean rebuild for every MR
> event is not only acceptable, but desireable.
>
> It is, after all, just "machine time" right?
This is actually something I had not considered with respect to changing
dependencies between packages, so indeed it would be best to always
rebuild the whole MR. Duly noted.
> > If there are other questions or considerations you'd like to make, please let
> > me know.
>
> If a build server goes down for maintenance, is that architecture "lost"
> - i.e. will it require manual intervention / building of what was pushed
> while that server was down?
I was also thinking about this earlier. I think the current design would
allow for a build server to go down at any time, and once it comes back
up (really, once abuildd-agentd comes back up on the build server) it
can just reconnect to the MQTT broker and receive any jobs that it
missed while offline. Based on my currently limited understanding of
MQTT, this can be accomplished by using persistent sessions and a
QOS >= 1. I guess that begs the question of what happens when the MQTT
broker goes down - I will have to research what the brokers do in such
a case, i.e. whether they save those persistent sessions to disk on
shutdown. Otherwise we would need to add something to the SQL or just
deal with such cases manually.
> > Another feature I thought of during the same period today is that it
> > should be possible to prioritize jobs based on type (push, MR) and branch, if
> > applicable. Things such as being able to skip a build that would be caused by a
> > particular commit (I have seen things like having "[ci-skip]" in the commit
> > title before) are more difficult in the present implementation, but I'm sure it
> > can be done at some point.
>
> Definitely prioritise 1-STABLE, then current, then feature branches,
> then MRs from contributors, then MRs from the general public. At least,
> that's how I would put it in a hierarchy.
For STABLE, will there be corresponding branch(es) for it? I assume
current will correspond to master. In this case we can set priority
levels for each project based on type (push, MR, manual - more on that
later), branch, and user. I currently have a setting that allows the
project to specify which branches it should watch for push events on.
MRs from contributors - how best to implement that? We could start
adding privileged GitLab API calls but I am slightly opposed to that. So
far I have not needed any privileged APIs, just whatever GitLab sends me
in the webhook payload. I already have access control implemented based
on the pushing user so that you can implement allow/deny lists. This can
probably be extended to the MRs along with adding priorities - I doubt
GitLab allows colons in usernames, so we could do something like
username:20 or whatever. Similarly with branches. We could also do
something like glob matching so that you could have "stable/*" with
priority 1, "master" with priority 5, "feature/*" with priority 10, etc.
Something I also haven't mentioned yet is there is also functionality
already that allows users to manually request builds for a MR by adding
a comment with a keyword to the MR (the default is "[build]"). This has
its own separate access control, but it ultimately generates an MR
event.
Max
Received on Thu Sep 06 2018 - 02:23:12 UTC