Rare project developers

[Alexander Gall <>]

On Thu, 5 Jan 2023 13:48:48 +0100, mc36 <> said:

> i see that you're having hard time with the regular profiles update... :)
> maybe my exceptional commit from yesterday fvcked up your local copy?

> btw, you already mentioned that to fully automate this, with the current
> source tree, one needs to be extra cautious... so here comes my question:

> would it be hard for you to store the optimizations in a different repo?

> back in the days frederic and i created one just for this purpose:
> https://bitbucket.software.geant.org/projects/RARE/repos/rare-profiles/browse
> it's empty now but if we move 
> https://bitbucket.software.geant.org/projects/RARE/repos/rare/browse/profiles
> there then one can easier automate the whole thing imho without, having to 
> care?

That would make things more complicated, I think. In the end, what
needs to happen is that changes in the p4 code can only be pushed to
the rare-nix repo once the profiles have been optimized for the new
code, no matter where the profiles are stored.

What I'm currently implementing is to call a webhook whenever a new
commit is pushed to the rare repo.  The hook launches a VM in the
SWITCH cloud (with 16 cores to speed things up but it needs to be
shut down when it's not used for cost reasons :).

Inside the VM, the following happens

   * Git clone of the rare repo for the new commit
   * Git clone of the latest commit of rare-nix
   * Run the optimizier with the SDE version and compiler flags for
     that version of rare-nix
   * If any of the profiles has changed, push the changes to a new
     branch of the rare repo and create a PR

The PR could be merged automatically but I prefer to do it
manually. After that, rare-nix can be updated with the reference to
this merged commit and Hydra will be happy.

A separate repo for the profiles doesn't really make a difference
except that the rare-nix repo would then have to pull in that repo as
well.

I don't see a simple way to make this whole thing any easier :(

One problem is to notice when the optimizer fails or doesn't change
any of the values. In either case no PR will be generated but there
won't be any explicit feedback (there will be access to a log file via
a URL, though).  I'll have to figure out some kind of push message for
this.

-- 
Alex