An open discussion list for topics related to the eduGAIN interfederation service.

[Peter Schober <peter.schober AT univie.ac.at>]

* Leif Johansson <leifj AT sunet.se> [2021-04-20 09:16]:
> Recently I had occasion use the edugain connectivity check and I was
> seeing a large degree of mismatch between the test results and
> reality in the sense that several IdPs who are marked green still
> were unable to pick up a new service I recently deployed.

:(

Though depening on just how recently that's to be expected, of course:
The path from local registrar to MDS may be short (1h delay max,
AFAIR) but how fast/often other federations ingest the MDS feed,
resign and republish may vary widely, I think. Same with the "last
mile" of end entities refreshing their metadata from their local
federation's feeds, I guess.

> Looking at the specs for the connectivity check [1] I found that tests are
> only performed using two test SPs. Since we're not using signed 
> authentication
> requests it would be possible to randomize checks over a larger set of SPs.
> 
> I worry that IdPs are happy when they have passed the check which doesn't
> help new services.

You mean folks are actively gaming the system in the sense that they
perform manual steps to load metadata only for the 2 test SPs (in
order to satisfy the test) instead of not doing anything special and
simply load the right (eduGAIN-enabled) metadata (in order to actually
interoperate with others)?

I kind of thought this to be possible (though still unlikely, as you'd
only be lying to yourself) with the "eduGAIN Attribute Release Check",
https://release-check.edugain.org/ since that's only about attribute
release and adding a rule to release attributes to 2 or 3 test SPs
seems easier to me than adding metadata for those SPs to your IDP.

So I think the eduGAIN Connectivity Check is less likely to become
misused that way.

> Would it be possible to volunteer services for inclusion into the
> set of SPs used for testing?

Let's see what Tomasz and Maja come up with, but "randomize checks
over a larger set of SPs" as you suggest would make such trickery
harder to pull off for both test services.
(Of course it should /always/ be significantly simpler to just Do The
Right Thing™ we're testing for instead of gaming the system but maybe
there are some real-world incentives for such trickery, e.g. the
federation operator nagging the IDP tech-c to fix some issue and
instead they trick the system to shut up their fed op?)

One downside I'd see with this approach is that it would artificially
create the impression that some SPs (of the set tested with) are more
often requested to be used with a gievn IDP than they in fact are,
judging from the authn requests reaching IDPs. But then most people
are unlikely to pay that close attention to their logs to even
notice and the client IP address in the logs would also give away that
those authn requests are not "real"/actual requests from end users...

-peter