Skip to Content.
Sympa Menu

rare-dev - Re: [rare-dev] Automated profile optimizer

Subject: Rare project developers

List archive

Re: [rare-dev] Automated profile optimizer


Chronological Thread 
    < Chronological >      < Thread >
  • From: Alexander Gall <>
  • To: <>
  • Cc: mc36 <>, Frédéric LOUI <>
  • Subject: Re: [rare-dev] Automated profile optimizer
  • Date: Tue, 7 Feb 2023 09:22:41 +0100
On Tue, 7 Feb 2023 09:19:53 +0100, Alexander Gall <> said:

> On Tue, 7 Feb 2023 09:03:02 +0100, mc36 <> said:
>> hi,
>> i've commited today morning to the bitbucket
>> https://bitbucket.software.geant.org/projects/RARE/repos/rare/commits/cc7cfe6a8f3b78e6f64b884c2001daa7b82fff1e
>> then i updated the optimize_profiles tag but seemingly nothing happened...
>> could you please give me hints about what i missed here?

> The webhook terminates with a 502 error if you trigger the process
> with the tag. I don't understand why. I just did the same and it
> worked (the script still failed because the VM took too long to start,
> maybe I should increase that timeout).

>> ps: is it possible to have a /rerun or something command to the slack bot
>> for such cases?

> Just remove and re-add the tag. I have enabled some debugging to see
> why it fails when you do this.

Also, you can test the webhook via bibucket. Go to settings/webhook,
then select "edit". There is a button "Test Connection", which should
get a 200 response with the body

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>RARE Profile Optimizer</title>
</head>
<body>
<pre>
Test successful
</pre>
</body>
</html>

--
Alex

> --
> Alex

>> thanks,
>> cs


>> On 1/16/23 15:34, mc36 wrote:
>>> you're unbelievable, thank you so much! :)
>>>
>>> On 1/16/23 13:55, Alexander Gall wrote:
>>>> Hi csaba
>>>>
>>>> I have implemented the mechanism for automated profile optimization I
>>>> described earlier. You can use it simply by pointing the Git tag
>>>> "optimize_profiles" to the commit of the RARE repo for which you want
>>>> to perform the optimization. Progress is reported live on the Slack
>>>> channel https://wp6t1-wp-rare.slack.com/archives/C04JM1LND63. I took
>>>> the liberty to invite you (and Fr d ric) to the channel. I know you're
>>>> not a Slack person, but you only have to check that channel after you
>>>> initiated an optimizer run :)
>>>>
>>>> The optimization finishes either with
>>>>
>>>> Status: Success
>>>>
>>>> or
>>>>
>>>> Status: Failure
>>>>
>>>> In both cases, the output of the optimization process is uploaded to
>>>> the channel. You can request the output for a running optimizer with
>>>> the "/progress" command in the channel at any time.
>>>>
>>>> The "Success" status means that the optimizer was able to find working
>>>> values for all of the profiles, where as "Failure" either means that
>>>> at least one profile could not be optimized or something else went
>>>> wrong. The log file should say what exactly the problem was in that
>>>> case.
>>>>
>>>> For "Success" there are two possible outcomes. If none of the
>>>> optimized values actually changed (compared to the tagged commit),
>>>> nothing else happens and the log ends with something like
>>>>
>>>> No changes detected
>>>>
>>>> This means that the RARE commit tagged with "optimize_profiles" can be
>>>> safely added to the RARE-NIX master branch (in rare/repo.nix).
>>>>
>>>> If some of the optimized values actually changed, the optimizer adds a
>>>> new branch named "optimizer_<sde-version>_<random>" in the RARE repo
>>>> and creates a PR to merge it to the master branch. One of us can then
>>>> review the generated code and merge it. It is recommended to use the
>>>> option to automatically delete the branch after the merge (the
>>>> <random> part of the branch name is there to avoid a conflict in a
>>>> future run of the optimizer in case one of the old branches is still
>>>> around).
>>>>
>>>> Then, finally, the commit that results from that merge can be added to
>>>> the RARE-NIX repo (rare/repo.nix).
>>>>
>>>> I have added the tag this morning to
>>>> https://bitbucket.software.geant.org/projects/RARE/repos/rare/commits/554626230089032b5065d6b6749115dca3642597
>>>>
>>>> Here is what happened when I did that:
>>>>
>>>> - The webhook "RARE P4 Profile Optimizer" associated with the RARE
>>>> repo was triggered
>>>>
>>>> - The webhook sent a POST request to
>>>> http://hydra.nix.net.switch.ch/cgi-bin/profile-optimizer.cgi
>>>>
>>>> - The CGI script checked that the event was the addition of the tag
>>>> "optimize_profiles" (any other event is ignored)
>>>>
>>>> - The script then activated a VM in the SWITCH cloud and executed a
>>>> helper script
>>>>
>>>> - The helper script cloned the RARE repo and checked out the commit to
>>>> which the "optimize_profiles" tag points and executed the
>>>> profiles/optimize.sh script
>>>>
>>>> - The optimize.sh script checked out the latest commit of the RARE-NIX
>>>> repo and ran the optimizer in the context of that commit (i.e. it
>>>> used the SDE version and p4c compiler options as required by that
>>>> version of RARE-NIX)
>>>>
>>>> - The optimizer run was successful, i.e. it found a working
>>>> configuration for all profiles
>>>>
>>>> - The optimized profiles did *not* differ from those already present
>>>> in the RARE repo tagged with "optimize_profiles"
>>>>
>>>> I.e. this was an instance of a successful run that did not produce a
>>>> new set of optimized values and hence no branch and PR.
>>>>
>>>> Please don't move the "optimize_profiles" tag while the optimizer is
>>>> running. The system can't deal well with multiple concurrent optimizer
>>>> runs :( It can probably also fail in many other ways, it's all a bit
>>>> hackish...
>>>>

Archive powered by MHonArc 2.6.19.

Top of Page