RFC: Moving towards validators in fields

Flow
1

Many years ago (June 2016 to be exact), Binder was designed as one of the main new features for Vaadin 8.0. One of the central ideas then was to do all the tricky things related to conversion and validation only once in Binder so that individual field component implementations would only have to be concerned with providing a value through HasValue and nothing else.

Since then, we have introduced field components with more sophisticated client-side restrictions that also need to be verified through server-side logic for security reasons. This means that part of the validation is configured through the component while Binder is still aware of the validation status through getDefaultValidator().

It seems inevitable that some aspects of validation have to be specific to the type of the field component which currently means that you are forced to split up your validation logic into two separate places: some in the field configuration and some in the binder configuration. The only practical way of avoiding this duality seems to be to move validation out of the binder and into the fields so that all validation could again be defined in the same place - the field. This RFC investigates all the things that may be affected by this.

The basic idea is thus that all validation rules would be defined for the field rather than for a Binder binding.

IntegerField participants = new IntegerField();
participants.setMin(1, "At least one participant is required");
participants.addValidator(value -> value != 13, "Can't have bad luck");
participants.setRequired("Must define the number of participants");

This structure means that each field implementation would have to be concerned with validation even if the field doesn’t have any inherent validation needs. This is probably not a big problem in practice since there’s already so many other details needed for a proper HasValue implementation that you anyways have to use e.g. AbstractField or CustomField.

This seemingly simple change does, however, have some knock-on effects that lead to trade-offs that need to be considered before we can choose a definite strategy.

Validating converted values

One central idea behind the current Binder design is that validators and converters can be mixed into an arbitrary chain of processing logic. This allows expressing validations rules in terms of domain-specific data types rather than only validating the basic value from the field itself. It doesn’t seem feasible to introduce conversion as a part of a field’s state since you would then have to make a distinction between the basic TextField<String> and a converted TextField<MyType>.

Viable options include:

  • Keep the current structure where a Binder binding is a chain of validators and converters while recommending users to define validation on the field whenever possible.
  • Remove (or deprecate) validator support from Binder bindings and instead only support a single converter that might in turn be a chain of simpler converters and validators. We could then also add a shorthand API for creating a binding with a single converter: binder.bind(someField, someConverter.withValidator(convertedValueValidator).chain(anotherConverter), getter, setter).
  • Remove (or deprecate) both validator and converter support from Binder and instead introduce a concept for creating a converted HasValue from a field component. ConvertedField<MyType> converted = myField.converted(someConverter). This field could then have additional validators added and be used directly with Binder: binder.bind(converted, getter, setter);

Bean validation

Bean validation (JSR 380) is fundamentally working on a bean level which means that Binder would still on some level also deal with validation. The current implementation applies property-level validators for the appropriate binding and it could in the future either keep doing the same or configure the field by adding a validator to the field. This would also give the opportunity that different field implementations could be aware of common validation annotations so that e.g. a @Min(1) annotation on a bean property would not only apply the corresponding server-side validation logic but also automatically apply field.setMin(1) so that the field’s UI would reflect the constraint. It might seem problematic to update the state of a field instance as a side effect from creating a binding but that’s pattern is already used e.g. when binding.asRequired() delegates to field.setRequiredIndicatorVisible(true).

Cross-field validation

The concept for cross-field validation would not change but it might become easier for users to understand: put the validator on the field on which the error message should be shown and then add a change listener to the other field to trigger the first field’s validators to run again even though the field’s own value has not changed. No such method exists today but would have to be added.

I18n objects

Form fields are currently (as of Vaadin 24.5) using i18n objects to configure error messages for the field’s own validation logic. This causes a redundancy in combination with the possibility of defining an error message as e.g. field.setMin(1, "Error message"). The i18n object might be shared between multiple field instances of the same type which means that a method like setMin would have to keep track of the error message separately rather than writing it to the current i18n object (if any).

In the short term, we could just let a message configured through a validation method override the corresponding message from the i18n object. This effectively means that the i18n object only defines default values.

In the long run, we should probably move away from i18n objects but instead make the components configure themselves based on getTranslation. This would also require a way of overriding the translation key that a specific field instance should use for a specific error condition.

Anything else?

Are these all the aspects that would be impacted by making validation a feature of field components instead of a feature of form binding? Would the approaches considered here work or are there additional things that need to be taken into account?

2 Likes
2

Short answer. The only practical way would be that everything is in the binder. Don’t break all Flow applications again.

Edit:

Keep the current structure where a Binder binding is a chain of validators and converters while recommending users to define validation on the field whenever possible.

This options made me a little bit less angry - thanks for considering / listing it as first viable option. You can’t imagine how massive the Binder is integrated into business application in real world applications.

1 Like
3

How would you practically make Binder APIs aware of component-specific validation functionaltiy like min and max limits from IntegerField and DatePicker, pattern restrictions for TextField and EmailField, and so on?

That’s where deprecations enter the picture. I don’t see any fundamental reason in this case to remove any specific API. There’s just the ever-present desire to keep our own continuous maintenance burden at a low level that means that anything deprecated might eventually be removed.

There needs to be a way of moving ahead when you realize that your old assumptions might no longer lead to the best approach for new users. But there’s always different alternatives for how to do it. In this case, it’s probably a question between doing nothing, supplementing current Binder APIs, or creating a completely new form binder with a new name.

It can be extrapolated based on the volume of forum discussion and issue tracker tickets. Binder is certainly in the “absolutely essential” category :wink:.

4

It is not problematic to me. In fact, I would like to see more of this delegation of the Binder to the fields. I would like the existing Binder API to be retained, but the validators (and possibly the converters) move to being stored in the fields instead of in the Binder. So the existing fluent Binder API becomes just a shortcut for the fields’ API, allowing developers to use either one. Because converters (and validators within them) have to execute in a prescribed order, an additional method fields could have would be something like setConvertedValue and getConvertedValue that would fire the converters in the right order (instead of having a ConvertedField class). The Binder still needs to be able to have “form-level” validation, of course.

(Hopefully I did not miss the point with this.)

1 Like
5

Aren’t those already available for the Binder and validated with the default validator or do you mean: How should the Binder know about their existence without the fields supplying the validators; e.g. how should somebody configure a binder that automatically applies the given min/max values?


forField(participants)
  .asRequired("Must define the number of participants"
  .withMin(1, "At least one participant is required")

IntegerField has Integer as type… so withMin could just delegate to a HasMinValue<T> … if that isn’t present in the field it can either throw or do nothing. The same could be applied to Dates or other types. HasPattern might be used as alternative for the text based fields.

Might be OT but IMHO: Adding validations to the fields was a mistake to begin with - they have created so many problems already. Fields should have some “visual” guidance about their max values or range - but every validation should have been done with the binder. See for example the problem for date fields that the server side has no way of getting a bad client side string input and transforming it because the field already requires a value that matches their formatting.

1 Like
6

The problem is how to define the return type of forField and asRequired so that some fields have withMin as an option, some fields have withPattern, some fields have both, and some fields have neither. It’s still simple enough with only two different cases (min and pattern) that lead to four different permutations but the number of permutations will grow exponentially with the number of individual cases.

The other challenge is that the Binder API could only support the cases that Vaadin explicitly supports but you would have to fall back to API on the field class if you create a field type that has some other type of built-in validation logic. We want to avoid designs that turn 3rd party components into 2nd class citizens.

7

While I understand the permutation problem on high lvl… I personally would say: you have at most 2 or 3 permutations:

  • range typed fields
  • string based fields
  • custom fields

And even that would be a nice to have and no must have Imho… not all problems can be fixed at compile time… just throw an unsupported exception if people do stupid things - they do it once.

For example asRequired was available for years and it took the checkbox component until 24.4 to support it :wink: and neither the binder throw nor did something broke… except the missing validation… which could have been a simple warning log “Field was called with asX… but does not support it. Consider fixing it.”

8

This is an area where it is very hard to achieve optimum that caters everything. There are requirements in the application that drive this to different directions.

Conceptually i.e. from software architecture point of view the data validation belongs to the data model. And that has been the north star idea of the Binder. Data model constraints need to be satisfied so that it can carry on with the business process.

But, it is performant to carry out the simple validations on the client side, as that gives the fastest feedback to the users. Here good examples are simple numerical range constraints, string lengths, required constraint etc. There is lot of demand to avoid extra server round trips for these. And these work well on presentation format. The client side validation support exists in the fields for this reason. Although there is events submitted to the server to keep the state in check.

But more generally, there are also complex validations, that need to be performed with data model data types instead of presentation format. And there are even more costly validations, that require back end calls to be performed. Some of these validations can be performed in server side implementation of the Field. But there are also cases which go beyond that.

One definitive problem we have with the Binder is that it has grown quite a bit since it was introduced in Vaadin 8.0. Yep, that was almost decade ago. And by the way, in Vaadin 7 we had validations and conversions API’s in the Field. So I would actually recommend you to play with smple Vaadin 7.7.17 application and use FieldGroup and make the comparison between the two. We ditched the idea of Field owning the validation for a reason (I hope … My memory is not serving me the details of that discourse anymore). Binder has gone thru lot of evolution during these years. The number of API’s has over doubled. And the fun fact is that we both forward ported and backported features between Vaadin 8 and the latest versions like Vaadin 24. So they have the same API’s even today. This growth has been organic, as our customers have requested good and well justified features to be added there. In the hindsight I would name some methods differently and organize it slightly differently in order to be more consistent and better DX. If we just would have the crystal ball you know. But nevertheless if you are able to dvelve thru the API jungle, the Binder delivers you pretty much.

My 5cents is that we should consider alternatives every now and then and gather ideas from the community how would you do it now. Especially if you have experience both the Vaadin 7 era Field based validation and Vaadin 8 - 24 era Binder based solution. If we go back to Field centered model, what were the Vaadin 7 shortcomings we should not replicate. What we would like to keep from the Binder. Should we still develop Binder further or introduce a new tool and leave Binder as is for considerable period of time as a compatibility solution. Do we need a revolution or just couple of more simple convenience API’s in Binder.

9

As Knoobie already mentioned. There must be a way to get invalid values in the server.
DatePicker is a huge pain

10

Yes, it would be useful addition to DatePicker / DateTimePicker …

For some reason the unparasble value is not included in unparsable-change events details, although it would be very simple change

Even now you can read the value from the client but double underscore indicates it as “private”.

11

Yes, that is the current mechanism that is quite new addition. Btw, Vaadin 24.5 adds finally API to customize the error message shown in the field upon client side validation error.

12

This is an intriguing idea … Currently it is undoubtly a bit confusing that you can set client side validators via Field’s API’s and other validators via Binder although Binder now reacts to client side validations too. Moreover there is overlap as you can do server side validation of say min/max value as well. If the developer is not careful it can lead to confusing errors if you use different min/max in server side and client side. Which one should have the precedence. My intuition says that if you set server side validation / or use JSR 380, it should actually disable any client side validation that could potentially conflict. But now there is nothing like that.

13

@Tatu2 Would you create an issue? My client would be happy to sponsor it.

14

This is somewhat popular approach. There is even more sophistication to this by forcing the input to certain format using mask or other mechanism. These two add-ons are rather popular.

It has been occasionally discussed whether something like this should be part of the official component set. Autoformatting / format enforcing is closely linked to the validation as it removes some need for input validation.

Closely related to this is the input pattern validation, which again can be set in the field or using Binder based Regexp validator.

There is also overlap with converters as it is possible to use Converters with Binder such a way that it results a bit similar effect as using TextField Formatter. This does not work if the TextField is in eager value change mode, and naturally involves server round trip.

15

One a bit more sophisticated API proposal is here: Server-side date parsing · Issue #2466 · vaadin/flow-components · GitHub

Naturally it would require the unparsable value to be included in unparsable-change event, and DatePicker could build on top of that.

This topic in DatePicker context is partially overlaping with custom formating of the dates, which is possible via I18n API with some limitations. The limitations could be reduced by adding locales to client side as proposed by this PR, which was rejected due extra package. Now this would not be an issue anymore as Flow loads client side resources progressively: fix: Provide locale with createCustomFormatBasedFormatterAndParser by TatuLund · Pull Request #2670 · vaadin/flow-components · GitHub

16

Usually the topic of validation is about a bean / business object of multiple properties. Or even more complex structures like business object consisting of nested beans. Just noting here that nested usage of Binder is possible by wrapping a sub-form into custom field.

However sometimes applications do have single fields which do not consist a form in the traditional sense. And some sort of input validation is required with those too. In that kind use case Binder feels overkill and direct validation API is more approachable. The current client side validation features can cater pretty much in this use case. If not enough, I wrote a small utility in the past to help this (and also to make migration of certain kind of Vaadin 7 apps easier)

17

Any idea on how that could be made to work with generics so that getConvertedType could be declared to return something more specific than Object?

18

Can’t agree more with the message! That’s exactly why I’m disabling all fields validation within my custom Binder to ensure everything is only configured in the Binder and the field is purely for the visual.

19

I would also love to have this kind of field. I’ve created my own based on Cleave.js and is heavily in used in our applications.

20

I think there’s an even more fundamental reason which is that the UI component should automatically and instantly react based on its validation rules. A very powerful example of this is the step buttons in e.g. IntegerField. You wouldn’t want to write application logic to selectively enable and disable the + and - buttons depending on the current value. Furthermore, the logic needs to run in the browser so that it will stop at the right number if the user clicks rapidly. A more snappy feeling and less network traffic are also nice improvements but not as fundamental as the other reasons.

My memory is not perfect either but I would say that the big thing there was to move away from the Property and Item concepts that were fundamentally stuck in the era before Java had generics or lambdas. This does also skew any comparison between FieldGroup and Binder towards those aspects rather than the differences between where validation and conversion resides.

When removing Property support from the fields, we were basically asking how far we could go with simplifying without causing any issues. At that time, we didn’t have anything similar to IntegerField step buttons in the framework so we didn’t even consider the duality that they cause. Now we have those and are starting to realize that we might have gone too far with simplifying the field abstraction.

Binder does three different things:

  • Form state management
  • Copying values to and from domain objects
  • Configuration of validation and conversion

Out of those, most of the evolution has been related to state management and copying whereas the validation & conversion aspect has been mostly untouched. My suggestion now is basically an incremental adjustment on that side.

I can agree that the current Binder functionality for state management and copying is close to a local maximum but I do also have a hunch that there would be even more potential in a radically different approach. The fundamental problem is that Binder is too monolithic which means that if you e.g. have some special needs for state management, then you cannot build your own solution for that while still reuse everything related to validation, conversion and copying. The amount of more or less directly copied code in your FieldBinder add-on is a manifestation of this problem.

I have some wild ideas on how we could address those things with the help of my dear full-stack signals concept but that’s definitely a new class introduced in parallel to Binder and also a topic for another RFC after we have gained some general experience with using signals from Flow. The key question is maybe how an adjustment to validation and conversion configuration would fit into that future.