Imagine that you have to consume an API to retrieve data from people of your company. Now imagine that all these data don’t follow any pattern and the API can return not only people, but robots, “phantom” accounts and all the source of irrelevant information. There are no rules: no flag to identify if the data belongs to a person or to some other creature and from time to time you can discover another variation that would classify the data as invalid.

Well, that happened. The validation could be achieved with “regex”, but it would be hard coded and the customer would always depend on the change in the code and new deploys.

Aha!

The most efficient and clean way found to do that it in Java was to create a table to save the rules that would configure a record as invalid, read and convert them to Predicates and dynamically validate each part of the API’s return to classify an object as valid or invalid.

Show me the code

This behaviour was reproduced on the project available on my GitHub, usingJava 11 and Spring Boot.

Object representation

The external API’s data is represented by the classPersonDTO.
The rules that define aPersonDTO as invalid are represented and persisted through entity ExclusionRule where:

• fieldName is the attribute onPersonDTO that will be checked.
• operator is an operator AND or OR.
• comparator is a comparator EQUALS or CONTAINS.
• ruleValues are the values separated by comma that would make thefieldName invalid.

Interpret rules

The resourcedata.sql will initialize some rules for the purpose of this test:

INSERTINTOexclusion_rule(field_name,comparator,operator,rule_values)VALUES('name','CONTAINS','OR','1,2,3,4,5,6,7,8,9,0');INSERTINTOexclusion_rule(field_name,comparator,operator,rule_values)VALUES('email','CONTAINS','OR','@exclude.me,1');INSERTINTOexclusion_rule(field_name,comparator,operator,rule_values)VALUES('internalCode','CONTAINS','AND','a,b');INSERTINTOexclusion_rule(field_name,comparator,operator,rule_values)VALUES('location','EQUALS','OR','jupiter,mars');

The rules above can be interpreted as:

• If the attributename onPersonDTO object contains 1, 2, 3, 4, 5, 6, 7, 8, 9 or 0, the object is invalid.
• If the attributeemail onPersonDTO object contains “@exclude.me” or “1”, the object is invalid.
• If the attributeinternalCode onPersonDTO object contains “a” and “b”, the object is invalid.
• If the attributelocation onPersonDTO object is equals to “jupiter” or “mars”, the object is invalid.

Using Predicates

For each possible combination of operators and comparators a validation class was created (RuleContainsAnd,RuleContainsOr andRuleEqualsOr). By implementing the interfacePredicate<T> those classes can be used to validate an object through the simple and elegant call oftest(myFieldValue) . It is only necessary to overwritetest method and define a custom rule.

publicclassRuleEqualsOrimplementsPredicate<String>{privateList<String>exclusionRulesLst;publicRuleEqualsOr(finalList<String>exclusionRulesLst){this.exclusionRulesLst=exclusionRulesLst;}@Overridepublicbooleantest(finalStringfieldValue){returnthis.exclusionRulesLst.stream().anyMatch(fieldValue::equals);}}

ClassExclusionRuleService is the responsible to retrieve saved rules, transform them to its correspondingPredicate and keep them in a list.

/**   * Retrieve all rules from the database and process it.   *   * @return   */privateMap<String,Predicate<String>>decodeAllRules(){// @formatter:offreturnthis.validationRuleRepository.findAll().stream().map(this::deconeOneRule).collect(Collectors.toMap(PairDTO::getRule,PairDTO::getPredicate));// @formatter:on}/**   * According to the rule configuration, create a Predicate.   *   * @param validationRule   * @return   */privatePairDTOdeconeOneRule(finalExclusionRulevalidationRule){PairDTOpairDTO=null;List<String>values=newArrayList<>();if(validationRule.getRuleValues().contains(",")){values=Arrays.asList(validationRule.getRuleValues().split(","));}else{values.add(validationRule.getRuleValues());}if(validationRule.getComparator()==ComparatorEnum.EQUALS&&validationRule.getOperator()==OperatorEnum.OR){pairDTO=newPairDTO(validationRule.getFieldName(),newRuleEqualsOr(values));}else{if(validationRule.getOperator()==OperatorEnum.OR){pairDTO=newPairDTO(validationRule.getFieldName(),newRuleContainsOr(values));}else{pairDTO=newPairDTO(validationRule.getFieldName(),newRuleContainsAnd(values));}}returnpairDTO;}

Where the magic lives

Now that all the validation “bed” is done, it is possible to use methodsfilterAllValid andisInvalid to receive an object or a list and pass them toisInvalidTestPredicate. On this last method we get the field of the classPersonDTO that matches the defined onExclusionRule and its value using Reflections.

It is important to be aware that the heavy use of Reflections can cause performance issues, but on this particular situation I’ve considered that some performance could be sacrificed to achieve the flexibility of the validation.

The magic happens when the methodtestis called. No additional test is required.

/**   * Retrieve the person's object fields by reflection and test its validity.   *   * @param person   * @param entry   * @return   */privateBooleanisInvalidTestPredicate(finalPersonDTOperson,finalEntry<String,Predicate<String>>entry){finalFieldfield=this.reflectionService.getFieldByName(person,entry.getKey());finalStringfieldValue=String.valueOf(this.reflectionService.getFieldValue(person,field));returnentry.getValue().test(fieldValue);}/**   * Verify if a person is invalid if it fails on any determined rule.   *   * @param person   * @return   */publicBooleanisInvalid(finalPersonDTOperson){returnexclusionRulesLst.entrySet().stream().anyMatch(e->this.isInvalidTestPredicate(person,e));}/**   * Get only valid objects from a list   *   * @param personDTOLst   * @return   */publicList<PersonDTO>filterAllValid(finalList<PersonDTO>personDTOLst){// @formatter:offreturnpersonDTOLst.stream().filter(person->!this.isInvalid(person)).collect(Collectors.toList());// @formatter:on}

Test me

On classExclusionRulesServiceTests we can check if the rules are being properly applied to the fields of a PersonDTO object.

@TestpublicvoidfilterAllValidPersonLstNameContainsOr_ok(){finalPersonDTOperson=newPersonDTO();person.setName("Daniane P. Gomes");person.setEmail("danianepg@gmail.com");person.setInternalCode("DPG001");person.setCompany("ACME");person.setLocation("BR");finalPersonDTOperson2=newPersonDTO();person2.setName("Dobberius Louis The Free Elf");person2.setEmail("dobby@free.com");person2.setInternalCode("DLTFE");person2.setCompany("Self Employed");person2.setLocation("HG");finalList<PersonDTO>personLst=newArrayList<>();personLst.add(person);personLst.add(person2);finalList<PersonDTO>personValidLst=this.exclusionRuleService.filterAllValid(personLst);assertEquals(personValidLst.size(),2);}

Conclusion

While consuming an external API we can receive data that is not properly structured. To check its relevance in a clean way we can:

• Create a repository of rules and represent them asPredicate<T>
• Convert the API response data to aPersonDTO object
• Check if each attribute ofPersonDTO is valid only by calling the methodtest

Originally posted onmy Medium page.