NotificationsYou must be signed in to change notification settings
Fork101
Star653

feat: add OpenSearch distance operations and selectors#1335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Open

mmiguerodriguez wants to merge22 commits intomaster

base:master

Choose a base branch

fromfeature/os-distance-operations

Open

feat: add OpenSearch distance operations and selectors#1335

mmiguerodriguez wants to merge22 commits intomasterfromfeature/os-distance-operations

Conversation

Copy link

Collaborator

mmiguerodriguez commentedOct 2, 2025

No description provided.

mmiguerodriguez added9 commits

September 23, 2025 19:55

feat: add OpenSearch distance operations and selectors

14c53c2

Merge branch 'master' into feature/os-distance-operations

df6ffe6

feature: added new integration test

a356951

Merge remote-tracking branch 'origin/master' into feature/os-distance…

9da8da8

…-operations

remove invalid assertions

54f1eac

fix tests

23b7b20

more test fixes...

b529331

fix test by using default constructor

b461a0f

Merge branch 'master' into feature/os-distance-operations

bec8eb7

mmiguerodriguez marked this pull request as ready for review

October 28, 2025 21:52

jgaleotti requested changes

Oct 29, 2025

View reviewed changes

Copy link

Collaborator

jgaleotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

please add test cases for the classes OpenSearchHeuristicsCalculator, ParameterExtractor and OpenSearchQueryHelper

...ler/src/main/java/org/evomaster/client/java/controller/opensearch/OpenSearchQueryParser.java

		importorg.evomaster.client.java.controller.opensearch.selectors.MatchSelector;
		importorg.evomaster.client.java.controller.opensearch.selectors.QuerySelector;

		publicclassOpenSearchQueryParser {

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

We need tests for OpenSearchQueryParser to check that the JSON is correctly parsed into its corresponding OpenSearch operation.

.../src/main/java/org/evomaster/client/java/controller/opensearch/utils/ParameterExtractor.java

		/**
		* Utility class to extract common parameters and reduce code duplication in selectors.
		*/
		publicclassParameterExtractor {

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

add test cases for this class

...c/main/java/org/evomaster/client/java/controller/opensearch/utils/OpenSearchQueryHelper.java

		/**
		* Extracts the case_insensitive parameter from a term query object.
		*/
		publicstaticBooleanextractCaseInsensitive(Objectquery,Stringstructure) {

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

add test cases for this class

mmiguerodriguez marked this pull request as draft

October 29, 2025 18:36

mmiguerodriguez marked this pull request as ready for review

October 29, 2025 18:36

mmiguerodriguez added3 commits

November 6, 2025 15:23

tests: add PR recommendations

41fae1f

Merge remote-tracking branch 'origin/master' into feature/os-distance…

ae37f6a

…-operations

Merge remote-tracking branch 'origin/master' into feature/os-distance…

44452ff

…-operations

jgaleotti requested a review fromarcuri82

November 14, 2025 13:56

arcuri82 requested changes

Nov 18, 2025

View reviewed changes

Copy link

Collaborator

arcuri82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

sorry for late check, but was on vacation most of last week

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java


		if (operationinstanceofMatchOperation) {
		returncalculateDistanceForMatch((MatchOperation)operation,doc);
		}

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

shouldn't these be a chain of if/else, with a final having a warn log if no match was present? (eg to see if we miss any case).

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

		// Not enough matches - distance based on how many more matches needed
		intshortfall =minimumRequired -matchCount;
		// Return a distance that reflects both the shortfall and the quality of non-matching terms
		returnshortfall *10.0 + (totalDistance /expectedTerms.size());

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

what ifexpectedTerms.size() is 0?

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java


		try {
		// Create pattern with case insensitive flag if needed
		java.util.regex.Patternpattern;

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

are the regex in OpenSearch having the same syntax of Java regex? clarify here in some comments, with possible some links

Copy link

CollaboratorAuthor

mmiguerodriguezNov 18, 2025•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

OpenSearch uses Lucene regular expression syntax, which is similar but NOT identical to Java regex.

https://opensearch.org/docs/latest/query-dsl/term/regexp/
https://lucene.apache.org/core/8_0_0/core/org/apache/lucene/util/automaton/RegExp.html

@jgaleotti Do you consider adding Java Regexp for this heuristic calculation as acceptable?

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java Outdated


		// Helper methods for advanced operations

		privateintcalculateEditDistance(Strings1,Strings2,booleanallowTranspositions) {

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

geneneric helper functions on strings should not be here, but rather inclient-java/distance-heuristics/src/main/java/org/evomaster/client/java/distance/heuristics/DistanceHelper.java

...-tests/spring/spring-rest-opensearch/src/main/java/com/opensearch/age/OpenSearchAgeRest.java Outdated

		// }
		@GetMapping("gte/{gte}")
		publicList<Age>findGteAge(@PathVariable("gte")Integergte)throwsIOException {
		returnageRepo.findGteAge(gte);

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

all these endpoints should return 404 if no data is found

...st/java/org/evomaster/e2etests/spring/rest/opensearch/students/OpenSearchStudentsEMTest.java

		Solution<RestIndividual>solution =initAndRun(args);

		// assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/students/{q}", null);
		assertHasAtLeastOne(solution,HttpVerb.GET,200,"/students/{lastName}",null);

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

add assertions for both 404 and 200 cases

...tests/spring-rest-opensearch/src/main/java/com/opensearch/queries/OpenSearchQueriesRest.java Outdated

		// Term selector tests
		@GetMapping("category/{category}")
		publicList<Product>findByCategory(@PathVariableStringcategory)throwsIOException {
		returnproductRepository.findByCategory(category);

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

do return 404 if nothing is found in these endpoints, as it will simplify writing assertions/checks in the tests

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java


		Solution<RestIndividual> solution = initAndRun(args);

		assertHasAtLeastOne(solution, HttpVerb.GET, 200, "/queries/category/{category}", null);

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

add assertions on 404

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

		}

		@Test
		public void testRangeQueries() throws Throwable {

Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

why are these E2E repeated with same configurations? can have single tests with all the assertions.
otherwise, if you only want to test specific endpoints, you have to passargs parameters to specify them, eg, with--endpointPrefix

...spring/spring-rest-opensearch/src/main/java/com/opensearch/queries/OpenSearchQueriesApp.java

		@@ -0,0 +1,21 @@
		package com.opensearch.queries;

Copy link

Collaborator

arcuri82Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

foldere2e-tests does not exist any more (we did a major refactoring of folder structures recently)

Copy link

CollaboratorAuthor

mmiguerodriguezNov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Changing that.

mmiguerodriguez added10 commits

November 18, 2025 10:03

Merge remote-tracking branch 'origin/master' into feature/os-distance…

f47e055

…-operations

refactor: use if/else chain with warn log for unsupported ops

dfe913b

fix: handle empty expectedTerms to prevent division by zero

5b7e4cd

docs: clarify OpenSearch regex vs Java regex syntax differences

152a361

refactor: move generic string helpers to DistanceHelper

b82ab2d

fix: return 404 when no data found in age endpoints

7b612ad

fix: return 404 in students endpoints and add 404 assertions in tests

e17daa6

fix: return 404 in queries endpoints and add 404 assertions

a874b55

refactor: move queries module to new folder structure

51ab79a

style: remove redundant javadoc comments

4bf9147

Copy link

CollaboratorAuthor

mmiguerodriguez commentedNov 18, 2025•
edited
Loading

@arcuri82 is there any known issue with theevomaster-e2e-tests-spring-rest-rsa module? Most of the times I try building the project I'm having issues with that...

[INFO] evomaster-e2e-tests-spring-rest-rsa ................ FAILURE [  0.150 s][INFO] evomaster-e2e-tests-spring-rpc-grpc ................ SKIPPED[INFO] evomaster-e2e-tests-spring-rpc-thrift .............. SKIPPED[INFO] evomaster-core-integration-tests ................... SKIPPED[INFO] evomaster-core-driver-it ........................... SKIPPED[INFO] evomaster-core-graphql-it .......................... SKIPPED[INFO] evomaster-core-it .................................. SKIPPED[INFO] evomaster-test-old-libraries ....................... SKIPPED[INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------

Is there any way to prevent those errors?

Copy link

Collaborator

arcuri82 commentedNov 20, 2025

@mmiguerodriguez hi. there is no known current issue aboutevomaster-e2e-tests-spring-rest-rsa. what is the error you are getting?

Copy link

CollaboratorAuthor

mmiguerodriguez commentedNov 20, 2025•
edited
Loading

I'm not completely sure what was the issue, perhaps different Java version used by Maven and IntelliJ builds...
I've modified my IntelliJ to use Java 17 and maven too. Now it's fixed.

I'll be reviewing why there's some errors on the build/tests. I've already made some of the adjustements.

Copy link

Collaborator

arcuri82 commentedNov 20, 2025

@mmiguerodriguez ah, yes. we moved to JDK 17 recently... and might move to 21 in the near future

Copy link

CollaboratorAuthor

mmiguerodriguez commentedNov 20, 2025

We'll need to adjustfor_developers.md since it may be outdated.

Copy link

Collaborator

arcuri82 commentedNov 20, 2025

@mmiguerodriguez good point! i ll do now

Labels

None yet

Movatterモバイル変換

feat: add OpenSearch distance operations and selectors#1335

Are you sure you want to change the base?

feat: add OpenSearch distance operations and selectors#1335

Uh oh!

Conversation

mmiguerodriguez commentedOct 2, 2025

Uh oh!

jgaleotti left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arcuri82 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmiguerodriguezNov 18, 2025• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmiguerodriguez commentedNov 18, 2025• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Uh oh!

arcuri82 commentedNov 20, 2025

Uh oh!

mmiguerodriguez commentedNov 20, 2025• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Uh oh!

arcuri82 commentedNov 20, 2025

Uh oh!

mmiguerodriguez commentedNov 20, 2025

Uh oh!

arcuri82 commentedNov 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mmiguerodriguezNov 18, 2025•
edited
Loading

mmiguerodriguez commentedNov 18, 2025•
edited
Loading

mmiguerodriguez commentedNov 20, 2025•
edited
Loading