The EquivalenceOracle interface does not seem to allow for guaranteed determinism

[stateMachinist]

findCounterExample expects inputs in the form of a Collection. However, this type does not guarantee an order over entries. This can be an issue when trying to implement reproducible sampling from the alphabet (e.g. as RandomWp does here; even though it samples from an ordered alphabet, the order may not be deterministic depending on the original collection type). This means that test word selection may in some cases depend on unpredictable runtime context. That may not be much of a problem in practice, but it does seem like a potential trip hazard to me. Then again, maybe I am expecting too many guardrails. What do you think?

Edit: On second thought, enforcing an ordered collection at this interface would only shift the problem. Formally, an alphabet is just a set, and any ordering over symbols requires a concretization from the abstract state machine formalism. I still think it may be a little bit cleaner when this concretization is forced to happen before learning begins (i.e. when setting up the experiment). But that's a debatable position, relating to our previous semantic discussion about the difference between algorithms and implementations.

[mtf90]

I think enforcing determinism (e.g., via requiring a List) would be an unnecessary restriction for what the component needs to accomplish. For the correctness of the learning process, equivalence oracles must only provide a counterexample, not necessarily the counterexample (determined by whatever metric).

However, this does not conflict with your wish for being able to determinize the process. I just see it as an implementation detail of, e.g., which combination of alphabet/collection, oracle, etc. you use. As such, I do agree that possibly involved glue-code (such as the Experiment class, etc.) should re-use user-provided inputs wherever possible and not impose custom constraints, e.g., by wrapping collections in lists, sets, etc. For example, in the RandomWp case, I think using CollectionUtil.randomAccessList in this line would be much cleaner. Sure, the oracle needs an ordered/random access datastructure at this point, but in case the inputs were a IntRange or CharRange, wrapping them in an ArrayList would introduce performance overhead, etc.

[stateMachinist]

I understand your perspective. I guess it is not the responsibility of the tester to guarantee deterministic behavior in every scenario. Rather, it is sufficient to enable deterministic behavior upon proper use (i.e. when called with an ordered collection), which the current interface already allows.