JUnitLoop includes tests that are not in any source folder

[willhains]

I have a few tests that are technically JUnit tests, but cannot be run within Eclipse due to environment dependencies. I want to exclude them from JUnitLoop.

I tried adding @Category(Runtime.class) to them, but it seems JUnit still calls their constructor, which in this case is enough to cause the environment dependency issue and crash JUnit, so none of my other tests get run.

So now I have tried moving those test cases to a folder which is not a source folder (i.e. not included in the build path of the project). However it seems JUnitLoop still includes them in the tests_to_run / LoopTestSuite.java. And since those classes are not on the build path, LoopTestSuite.java fails to compile.

JUnitLoop should only search for JUnit test classes in the source folder(s) of each project.

[mirkoseifert]

As I understand there are two issues.

First, the @Category(Runtime.class) annotation does not prevent JUnit from creating an instance of your test class. This is due to the fact that we use a test runner which is bundled with JUnit (org.junit.experimental.categories.Categories). This runner or JUnit itself is causing the call to the constructor. I think you can move the code from the contructor to a setup method to avoid this issue. The alternative would be to use a custom test runner, but that may cause trouble in other scenarios.

Second, tests which are not longer contained in a source folder are still executed. Since JUnitLoop listens to compilation events to determine where to find test classes, classes which are not compiled are not added to the test suite that is executed by JUnitLoop. However, since your class was compiled before it is possible that JUnitLoop does not recognize that the class is not part of a source folder anymore. This is a bug, which needs to be fixed. In the meantime you may delete the project 'JUnitLoop' from your workspace. It will automatically be regenerated by JUnitLoop and not contain the test class anymore.

[willhains]

Ah, I see! Very interesting. I will try deleting the JUnitLoop project and report back here.

With regards to the first issue (constructor calls despite @Category(Runtime.class)), unfortunately I can't modify the constructor code as it comes from a framework. (It's Play Framework's FunctionalTest if you're interested.) Anyway, I guess there's nothing you can do about this in JUnitLoop -- probably should be raised as a bug on JUnit itself though...

[willhains]

Yes, deleting the JUnitLoop project after removing those folders from the build path seems to work around the second issue. Thanks!

[mirkoseifert]

I agree regarding the issue with the constructor call.