Repository navigation
Dataprovider caches results even for dynamically created data / results (e.g. using test method as parameter) #105
Description
Hello , would you help to check the issue.
I write a sample in https://github.com/weiro-9-w7/junit-data-provider
when you DataProviderExternelFileTest testcase, FrameworkMethod get annotation is the other method annotation.
@RunWith(DataProviderRunner.class)
public class DataProviderExternelFileTest {
@Test
@UseDataProvider(value = "loadFromExternalFile", location = LoadFromExternalFile.class)
@ExternalFile(format = ExternalFile.Format.JSON, value = "persons.json", clazz=Person.class, isArray=true)
public void should_return_array_when_is_array_is_true(List<Person> person) {
// Expect:
assertEquals(person.get(0).getName(), "zhangsan");
assertEquals(person.get(0).getAge(), 18);
}
@Test
@UseDataProvider(value = "loadFromExternalFile", location = LoadFromExternalFile.class)
@ExternalFile(format = ExternalFile.Format.JSON, value = "person.json", clazz=Person.class)
public void should_return_object_when_is_array_is_false(Person person) {
// Expect:
assertEquals(person.getName(), "zhangsan");
assertEquals(person.getAge(), 18);
}
}
public class LoadFromExternalFile {
@dataProvider
public static Object[][] loadFromExternalFile(FrameworkMethod testMethod) throws IOException {
ExternalFile externalFile = testMethod.getAnnotation(ExternalFile.class);
String testDataFile = externalFile.value();
File file = new File(LoadFromExternalFile.class.getResource("/").getPath() + testDataFile);
String content = FileUtils.readFileToString(file, "utf8");
if(externalFile.isArray()){
return new Object[][]{{JSON.parseArray(content, externalFile.clazz())}};
}else{
return new Object[][]{{JSON.parseObject(content, externalFile.clazz())}};
}
}
}
when you run should_return_array_when_is_array_is_true method, but ExternalFile externalFile = testMethod.getAnnotation(ExternalFile.class); externalFile is should_return_object_when_is_array_is_false
Activity
weiro-9-w7 commentedon Jan 23, 2018
aaschmid commentedon Jan 23, 2018
Hi @weiro-9-w7,
I will have a look as soon as I can. How urgent is the topic for you?
Cheers,
Andreas
weiro-9-w7 commentedon Jan 24, 2018
Hi @aaschmid ,
I'm glad you can give feedback to me in time. We prepare to use Junit-data-provider on the legacy systems . The function of different methods to use different annoation data is very important to us. So we hope to help solve it as soon as possible.
aaschmid commentedon Jan 24, 2018
Hi @weiro-9-w7,
to be honest, I don't get your problem. Everything is working fine on my machine, see this screenshot:
Test
should_return_array_when_is_array_is_truegot an array and the other didn't get one, as expected.Is there another issue, I haven't understood yet?
weiro-9-w7 commentedon Jan 25, 2018
Hi @aaschmid ,
yes, you can't get any problems because I move one method to other class.
When one class have two methods and use annotation, issues will happen.
Would you run test for DataProviderExternelFileMixTest. you will found problem.
weiro-9-w7 commentedon Jan 25, 2018
weiro-9-w7 commentedon Jan 25, 2018
aaschmid commentedon Jan 25, 2018
Hi @weiro-9-w7,
problem is that the dataprovider results are cached per test class such that your dataprovider method is only called once for the first test case in this class. As JUnit4 always evaluates all test cases before executing it, it does not matter if you only execute one test case or all test cases in that class.
This caching mechanism was introduced by feature request, see #93. The caching, though, only occurs per test class in JUnit4. In JUnit5 the caching even occurs on per test run ...
I guess I should distinguish if a dataprovider takes a further argument (do not cache result) or not (cache result). Another possibility could be another setting option on
@DataProviderannotation.I will try to provide a fix with a new version on the weekend.
Cheers,
Andreas
weiro-9-w7 commentedon Jan 26, 2018
@aaschmid
thanks
1 remaining item
aaschmid commentedon Jan 28, 2018
Hi @weiro-9-w7,
I fixed the issue by introducing a
cacheoption on@DataProvider, see commit 143948f (or above). I just need to stabilize the test cases (very fragile currently). You can test it manually by cloning the repo and creating your own version locally.Any feedback welcome.
Cheers,
Andreas
introduce option on @dataProvider if its result should be cached (#105)
[-]Annotation Insanity -- in IntelliJ IDEA run[/-][+]Dataprovider caches results even for dynamically created data / results (e.g. using test method as parameter)[/+]aaschmid commentedon Jan 29, 2018
Works with new version v2.2, see
weiro-9-w7 commentedon Feb 5, 2018
Hi @aaschmid , verified, thanks!
shiv20934 commentedon Apr 5, 2021
Getting the below issue after using v2.2
java.lang.ArrayStoreException: sun.reflect.annotation.TypeNotPresentExceptionProxy
at sun.reflect.annotation.AnnotationParser.parseClassArray(Unknown Source)
at sun.reflect.annotation.AnnotationParser.parseArray(Unknown Source)
at sun.reflect.annotation.AnnotationParser.parseMemberValue(Unknown Source)
at java.lang.reflect.Method.getDefaultValue(Unknown Source)
at sun.reflect.annotation.AnnotationType.(Unknown Source)
at sun.reflect.annotation.AnnotationType.getInstance(Unknown Source)
at sun.reflect.annotation.AnnotationParser.parseAnnotation2(Unknown Source)
at sun.reflect.annotation.AnnotationParser.parseAnnotations2(Unknown Source)
at sun.reflect.annotation.AnnotationParser.parseAnnotations(Unknown Source)
at java.lang.reflect.Executable.declaredAnnotations(Unknown Source)
at java.lang.reflect.Executable.declaredAnnotations(Unknown Source)
at java.lang.reflect.Executable.getDeclaredAnnotations(Unknown Source)
at java.lang.reflect.Method.getDeclaredAnnotations(Unknown Source)
at java.lang.reflect.AccessibleObject.getAnnotations(Unknown Source)
at org.junit.runners.model.FrameworkMethod.getAnnotations(FrameworkMethod.java:189)
at org.junit.runners.model.TestClass.addToAnnotationLists(TestClass.java:58)
at org.junit.runners.model.TestClass.(TestClass.java:46)
at org.junit.runners.ParentRunner.(ParentRunner.java:75)
at org.junit.runners.BlockJUnit4ClassRunner.(BlockJUnit4ClassRunner.java:57)
at com.tngtech.java.junit.dataprovider.DataProviderRunner.(DataProviderRunner.java:79)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:29)
at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:21)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:26)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createUnfilteredTest(JUnit4TestLoader.java:90)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createTest(JUnit4TestLoader.java:76)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.loadTests(JUnit4TestLoader.java:49)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:526)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:770)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:464)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
aaschmid commentedon Apr 5, 2021
@shiv20934: Can you provide more information and / or use v2.8 instead which might not have that issue anymore?