Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Introduce object creation using builders for fuzzers#2583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
Markoutte wants to merge11 commits intomain
base:main
Choose a base branch
Loading
frompelevin/builders-providers

Conversation

@Markoutte
Copy link
Collaborator

Description

We introduce newBuilderObjectValueProvider for Java that finds any method, that can create a value of given type. Together withAbstractObjectValueProvider it works only when ObjectValueProvider and others fails to build a value.

The provider searches any method from any class that can return object of this type or any type that can be inferenced from the given one. It doesn't matter if this method static or not, but if this method is not static, that we suppose that we have a builder class for it. As a builder class we try to find other methods that returns an instance of this builder. This logic added because of default way to create such builder, where any set or add method returns the instance of builder to support chain calling. This builder should work to find methods for factories and singletones as well.

To support this provider, we introduce new routine in fuzzer's API calledModify (the name maybe is not final). It is added forSeed.Recurisve and works as follows: it accepts a sequence of routines that changes or creates new object with given this instance and fuzzed values. For example, we firstly create builder's object and update it with builder method, then finally we call the method, that creates an object using this builder. This sequence is always called when creating an object and never changed its ordering.

In addition, you can use this method, apply some method to the instance and return the instance object itself, for example, to always close aClosable object.

As a result,ContestEstimator covers 53% of guava project code, before it covers only 43%.

How to test

Automated tests

All fuzzer tests should pass.

Manual tests

You can try to generate tests for this little project:

publicinterfaceMuzzi {intgetA();}
publicclassRealMuzziimplementsMuzzi {privatefinalinta;privateRealMuzzi(inta) {this.a =a;    }@OverridepublicintgetA() {returna;    }publicstaticclassBuilder {privateinta;publicBuildersetA(inta) {this.a =a;returnthis;        }publicMuzzibuild() {returnnewRealMuzzi(a);        }    }}
publicclassMuzziTest {publicintsize(Muzzimuzzi) {if (muzzi.getA() <0) {return -1;        }elseif (muzzi.getA() >0) {return1;        }return0;    }}

Self-check list

Check off the item if the statement is true. Hint: [x] is a marked item.

Please do not delete the list or its items.

  • I've set the properlabels for my PR (at least, for category and component).
  • PRtitle anddescription are clear and intelligible.
  • I've added enoughcomments to my code, particularly in hard-to-understand areas.
  • The functionality I've repaired, changed or added is covered withautomated tests.
  • Manual tests have been provided optionally.
  • Thedocumentation for the functionality I've been working on is up-to-date.

@MarkoutteMarkoutte added ctg-enhancementNew feature, improvement or change request comp-fuzzingIssue is related to the fuzzing labelsSep 8, 2023
@IlyaMuravjov
Copy link
Collaborator

Now it finds bridge methodorg.springframework.format.number.money.CurrencyUnitFormatter.print(Object, Locale): String and latter crashes withClassNotFoundException: javax.money.CurrencyUnit when trying to create a value forCurrencyUnitFormatter.

I think the codebase is not ready to deal with arbitrary classes that can be found on the classpath and the best solution for now is to make sure that we only createFuzzedType instances from classes that only mention classes that are on the classpath.

In other words someValueProviders expect that usingfuzzedType.classId.allMethods,fuzzedType.classId.allConstructors, andfuzzedType.classId.allDeclaredFieldIds won't throw an exception.

The easiest way to achieve that, that I can see is to makerunJavaFuzzing passClassDependenciesCheckingValueProvider(delegate = ValueProvider.of(providers) torunFuzzing whereClassDependenciesCheckingValueProvider.generate(description, type):

  • returnssequenceOf(Seed.Simple(nullFuzzedValue(classClassId)) iftype.classId.allMethods,type.classId.allConstructors, ortype.classId.allDeclaredFieldIds throws any exceptions
  • otherwise it returnsdelegate.generate(description, type), preferably filtering out routines that accept classes, mentioning classes that are not on the classpath

@Markoutte
Copy link
CollaboratorAuthor

@IlyaMuravjov I think that there're a many of other problems that we won't find by this review. Let's create separate issues with reproducers and stack traces for them and fix them. The main goal of this PR is to add this builder for Java, not for Spring. And I'd like to keep Java part as simple as possible without additional wrappers and delegators.

…iders# Conflicts:#utbot-spring-framework/src/main/kotlin/org/utbot/framework/context/spring/SpringApplicationContextImpl.kt
@IlyaMuravjov
Copy link
Collaborator

@Markoutte
I've pushed a commit that adds a decorator letting Spring test generation to work withBuilderObjectValueProvider without crashing withClassNotFoundException. However, this problem is not Spring-specific, so I've also made an issue#2594 showing how it can be reproduced without Spring.

I mergedmain to resolve conflicts instead of rebasing ontomain, because I didn't want to force push into your branch.

Also, this PR reveled a problem in instrumnted-process that marks executions as timed out even if the thing that actually timed out is the argument creation prior to method under test invocation, see#2595.

Next, it seems thatScene.v().classes doesn't contain all classes on the classpath, rather it only includes:

  • classes that are reachable from classes located in the.class files (as opposed to.jar files)
  • some classes from the JDK
  • classes that are reachable fromorg.utbot.framework.util.classesToLoad

Meaning that for large projectsBuilderObjectValueProvider can get significantly slower and the result of53% stated in the PR description depends on the project under test being provided as.jar file (similarly to how it's done in theContestEstimator) and not.class files (similarly to how it's done when UtBot is used via Intellij plugin).

Finally, I tried to check by how much providingguava in form of.class files instead of.jar file affects performance, but when I tried generating tests for the following method, the fuzzer crashed with the following exception.

publicstaticbooleanisNull(LocalDateTimelocalDateTime) {if (localDateTime ==null)returntrue;returnfalse;}
23:32:56.036 | INFO  | JavaLanguage              | Fuzzing is stopped because of an errorjava.lang.ClassNotFoundException: com.google.common.io.TempFileCreator$JavaNioCreator$lambda_userPermissions_3__261at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]at java.lang.ClassLoader.loadClass(ClassLoader.java:587) ~[?:?]at org.utbot.common.FallbackClassLoader.loadClass(FallbackClassLoader.kt:41) ~[utbot-core-2023.09-SNAPSHOT.jar:?]at java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[?:?]at org.utbot.framework.plugin.api.util.IdUtilKt.getJClass(IdUtil.kt:105) ~[utbot-framework-api-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.providers.BuilderObjectValueProvider$generate$1.invokeSuspend(Objects.kt:237) ~[utbot-java-fuzzing-2023.09-SNAPSHOT.jar:?]at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlin.sequences.SequenceBuilderIterator.hasNext(SequenceBuilder.kt:129) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at org.utbot.fuzzing.ValueProvider$Combined$generate$1.invokeSuspend(Providers.kt:231) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlin.sequences.SequenceBuilderIterator.hasNext(SequenceBuilder.kt:129) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at org.utbot.fuzzing.ValueProvider$Combined$generate$1.invokeSuspend(Providers.kt:231) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlin.sequences.SequenceBuilderIterator.hasNext(SequenceBuilder.kt:129) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:307) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:303) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlin.sequences.SequencesKt___SequencesKt.toCollection(_Sequences.kt:787) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlin.sequences.SequencesKt___SequencesKt.toMutableList(_Sequences.kt:817) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlin.sequences.SequencesKt___SequencesKt.toList(_Sequences.kt:808) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at org.utbot.fuzzing.FuzzingApi$produce$seeds$1.apply(Api.kt:427) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi$produce$seeds$1.apply(Api.kt:426) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at java.util.HashMap.computeIfAbsent(HashMap.java:1220) ~[?:?]at org.utbot.fuzzing.FuzzingApi.produce(Api.kt:426) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.fuzz(Api.kt:399) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.reduce(Api.kt:484) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.produce(Api.kt:438) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.fuzz(Api.kt:399) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.reduce(Api.kt:519) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.produce(Api.kt:437) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.fuzz(Api.kt:399) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.reduce(Api.kt:519) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.produce(Api.kt:437) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.fuzz(Api.kt:399) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.reduce(Api.kt:539) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.produce(Api.kt:437) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.fuzz(Api.kt:399) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.fuzz$fuzzOne(Api.kt:331) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.fuzz(Api.kt:350) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi.access$fuzz(Api.kt:1) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at org.utbot.fuzzing.FuzzingApi$fuzz$2.invokeSuspend(Api.kt) ~[utbot-fuzzing-2023.09-SNAPSHOT.jar:?]at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:?]at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:?]at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:?]at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:?]at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:?]at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:?]at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:?]at org.utbot.common.ConcurrencyKt.runBlockingWithCancellationPredicate(Concurrency.kt:38) ~[utbot-core-2023.09-SNAPSHOT.jar:?]at org.utbot.framework.plugin.api.TestCaseGenerator$generate$2$3.invoke(TestCaseGenerator.kt:204) ~[utbot-framework-2023.09-SNAPSHOT.jar:?]at org.utbot.framework.plugin.api.TestCaseGenerator$generate$2$3.invoke(TestCaseGenerator.kt:203) ~[utbot-framework-2023.09-SNAPSHOT.jar:?]at org.utbot.common.ConcurrencyKt.runIgnoringCancellationException(Concurrency.kt:47) ~[utbot-core-2023.09-SNAPSHOT.jar:?]at org.utbot.framework.plugin.api.TestCaseGenerator.generate(TestCaseGenerator.kt:203) ~[utbot-framework-2023.09-SNAPSHOT.jar:?]at org.utbot.framework.process.EngineProcessMainKt$setup$4.invoke(EngineProcessMain.kt:127) ~[utbot-framework-2023.09-SNAPSHOT.jar:?]at org.utbot.framework.process.EngineProcessMainKt$setup$4.invoke(EngineProcessMain.kt:112) ~[utbot-framework-2023.09-SNAPSHOT.jar:?]at org.utbot.rd.IdleWatchdog$measureTimeForActiveCall$1$2$1.invoke(ClientProcessUtil.kt:115) ~[utbot-rd-2023.09-SNAPSHOT.jar:?]at org.utbot.rd.IdleWatchdog.wrapActive(ClientProcessUtil.kt:88) ~[utbot-rd-2023.09-SNAPSHOT.jar:?]at org.utbot.rd.IdleWatchdog$measureTimeForActiveCall$1.invoke(ClientProcessUtil.kt:114) ~[utbot-rd-2023.09-SNAPSHOT.jar:?]at com.jetbrains.rd.framework.IRdEndpoint$set$1.invoke(TaskInterfaces.kt:182) ~[rd-framework-2023.1.2.jar:?]at com.jetbrains.rd.framework.IRdEndpoint$set$1.invoke(TaskInterfaces.kt:182) ~[rd-framework-2023.1.2.jar:?]at com.jetbrains.rd.framework.impl.RdCall.onWireReceived(RdTask.kt:362) ~[rd-framework-2023.1.2.jar:?]at com.jetbrains.rd.framework.MessageBroker$invoke$2$2.invoke(MessageBroker.kt:57) ~[rd-framework-2023.1.2.jar:?]at com.jetbrains.rd.framework.MessageBroker$invoke$2$2.invoke(MessageBroker.kt:56) ~[rd-framework-2023.1.2.jar:?]at com.jetbrains.rd.framework.impl.ProtocolContexts.readMessageContextAndInvoke(ProtocolContexts.kt:148) ~[rd-framework-2023.1.2.jar:?]at com.jetbrains.rd.framework.MessageBroker$invoke$2.invoke(MessageBroker.kt:56) ~[rd-framework-2023.1.2.jar:?]at com.jetbrains.rd.framework.MessageBroker$invoke$2.invoke(MessageBroker.kt:54) ~[rd-framework-2023.1.2.jar:?]at com.jetbrains.rd.util.threading.SingleThreadSchedulerBase.queue$lambda-3(SingleThreadScheduler.kt:41) ~[rd-core-2023.1.2.jar:?]at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]at java.lang.Thread.run(Thread.java:833) [?:?]

EgorkaKulikov pushed a commit that referenced this pull requestSep 19, 2023
…ing (#2561)* Support mock strategies and type replacement in Spring unit test fuzzing* Avoid mocking replaced types* Only use `InjectMockValueProvider` for `thisInstance`* Disallow non-mocks when mock can be used* Move all value provider decorators to one package* Improve adding of `ReplacedFuzzedTypeFlag`* Make `map` and `except` apply transformer/filter to ALL value providers* Avoid using `anyObjectValueProvider` that is removed in#2583* Make `AnyObjectValueProvider` not extend any interfaces, remove unrelated comment* Fix compilation after rebase
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@egiptipavelegiptipavelAwaiting requested review from egiptipavel

@IlyaMuravjovIlyaMuravjovAwaiting requested review from IlyaMuravjov

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

comp-fuzzingIssue is related to the fuzzingctg-enhancementNew feature, improvement or change request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Markoutte@IlyaMuravjov
[8]ページ先頭
©2009-2025 Movatter.jp