inefficient test detection

[staabm]

I wonder whether this extension could also rank tests by their efficency, meaning not just a ranking by the runtime involved, but a separate ranking for e.g. memory usage.

wdyt?

[localheinz]

Sounds interesting, I'll consider this!

[mvorisek]

Measuring memory usage is quite complicated, as test might fill caches, load files/classes, resulting in very different results on changed order or tests rerun/repeat.

In atk4/data repo we test memory leaks by https://github.com/atk4/data/blob/5.2.0/.github/workflows/test-unit.yml#L149 . We run tests multiple times and test if there is zero memory increase when tests are rerun. We rerun them many times, as some data structures (array, string..) in php are resized in steps (4, 8...) so this way we can detect even the smallest memory leaks like self::$x[] = true;.