AI Integration | Flow Reference | Vaadin Docs
says to post here with feedback, so here you go 
This was auto-generated, then mildly hand-edited based on the testing I’ve been doing.
Feedback on the Vaadin AI Grid preview (vaadin-ai-components-flow)
- Version evaluated: 25.2.5, verified against
flow-componentsmain(2026-08-17 — identical on every point below) - Context: natural-language querying over a 594-table Aurora MySQL warehouse, Bedrock Converse via Spring AI 2.0.0, non-streaming
- Basis: ~90 measured 12-question eval runs across five Bedrock models, plus source reading of the add-on and its Spring AI dependency
1. Summary
We built a production feature on GridAIController and eventually had to replace it. The rendering, column generation, grouping and lazy loading are good, and we still use GridRenderer and DatabaseProviderAITools unchanged. What we could not use was the tool surface: its workflow instructions, its state tool, its error handling and its validation strategy. None of those can be adjusted from outside the package.
We also had to subclass SpringAILLMProvider (to get the turn off the UI thread) and BedrockProxyChatModel (to recover discarded response metadata, and to repair tool arguments Spring AI cannot re-parse). §3 covers those and may matter more than the grid findings, because they affect every AIController.
Items marked [dialect] are MySQL/JDBC-specific; we propose them as pluggable strategies, not core changes. Several findings are Spring AI’s rather than Vaadin’s (§2.1, §2.2, §4) — we report them because every Bedrock user of this add-on hits them, the add-on’s own tool definitions trigger §2.2, and Vaadin is better placed to raise them upstream.
2. Defects with measured impact
2.1 Bedrock tool arguments are serialized in a form Spring AI cannot re-parse
BedrockProxyChatModel line 628 builds arguments with toolUseContentBlock.toolUse().input().toString(). Document.toString() does not escape control characters. Line 320 then parses that same string strictly via jsonHelper.fromJsonToMap. Any model emitting multi-line SQL — most of them, since SQL is conventionally formatted across lines — produces JSON Spring AI can write but not read. The turn dies on the follow-up request, so the symptom is a turn that silently stops rather than an error where the fault is.
We measured it by installing a repair layer that parses leniently and re-serialises strictly, counting repairs per 12-question run:
| Model | Repairs |
|---|---|
qwen.qwen3-32b-v1:0 |
700 – 3,300 |
qwen.qwen3-next-80b-a3b |
7 – 1,700 |
openai.gpt-oss-20b-1:0 |
20 – 264 |
nvidia.nemotron-super-3-120b |
5,403 |
Recommendation: raise with the Spring AI team — serialise via a real JSON writer at 628, or parse tolerantly at 320. Meanwhile the add-on docs should warn that multi-line tool arguments are unsafe on the Bedrock Converse path, because a user’s first assumption will be that the AI components are at fault.
2.2 Parameterless tools break the Bedrock request replay
DatabaseProviderAITools.getDatabaseSchema returns null from getParametersSchema(); GridAITools.getGridState declares only an optional gridId. Models disagree on what to send for a no-argument call. qwen3-32b sends {} and survives. gpt-oss-20b sends an empty string, and Bedrock rejects the next request:
ValidationException: The format of the value at messages.1.content.0.toolUse.input is invalid.
Provide a json object for the field and try again. (Status Code: 400)
So the model’s first schema fetch poisons the turn. Across an earlier 72-run measurement this accounted for essentially every gpt-oss-20b failure. Our workaround substitutes a schema with one optional ignored property, so there is always something to put in the object:
{ "type": "object",
"properties": { "reason": { "type": "string",
"description": "Optional note on why this tool is being called. Ignored by the tool." } } }
Recommendation: never ship a ToolSpec whose getParametersSchema() returns null or "properties": {}. One line each in DatabaseProviderAITools and GridAITools; costs a few tokens and removes a whole class of provider-specific failure.
2.3 GridAITools discards the one detail the model needs
} catch (ValidationException e) {
return "Error updating grid data: " + e.getMessage();
} catch (Exception e) {
return "Error updating grid data.";
}
ValidationException is a private nested class, so application code cannot construct it. Every exception an application’s Callbacks.updateData throws therefore becomes the fixed string. Our provider knew exactly what was wrong — “Column ‘readm_30_rate’ does not exist in table ‘hospital_atlas_names’” — and the model was told only that something failed. The observed result is a loop: retry the same query, or re-fetch a 20K-token schema hunting for an unidentifiable mistake. One run logged 58 identical resubmissions in a single turn.
Recommendation, in order of preference: (1) make the exception type public, e.g. ToolValidationException in the provider package, so applications can signal “this message is safe to relay”; (2) let Callbacks.updateData return Optional<String> or a small result record, making rejection an ordinary return value; (3) at minimum add an opt-in setRelayToolErrors(boolean). The generic default is defensible for unexpected exceptions and we would keep it — the problem is there is no supported way to say “I validated this deliberately and the reason is safe.”
2.4 The workflow instructions steer models away from pre-flight validation
INSTRUCTIONS_TEXT says:
WORKFLOW:
Complete the user's request in a SINGLE response by calling all needed tools.
1. Call get_grid_state() to see what's already configured
2. Call get_database_schema() to learn the exact table and column names
3. Call update_grid_data() with a SQL SELECT query using only columns from the schema
IMPORTANT:
- Call get_grid_state() and update_grid_data() in the SAME response
- Do NOT stop after get_grid_state()
We added a validate_sql tool that checks a candidate query against the live catalogue before commit; it measurably improves answer quality. But the add-on describes a three-step workflow that omits it and adds “Complete the user’s request in a SINGLE response”, which models read as go straight from schema to commit. Our system prompt says the opposite. Models merged the two badly and some turns skipped validation entirely, committing SQL that then failed inside deferred rendering, where §2.3 discarded the reason.
This is the change that forced the replacement: INSTRUCTIONS_TEXT is a private constant consumed by a tool built inside getTools(), with no override point.
Recommendation: treat the workflow text as configuration — setInstructions(String) and, more usefully, addInstructions(String). An application contributing a tool needs to tell the model where that tool fits without restating Vaadin’s own guidance and keeping it in sync across upgrades.
2.5 Validation by probe execution costs a full query run, and duplicates the renderer
public void updateData(String gridId, String query) {
// Validate eagerly so invalid SQL propagates back to the LLM as a tool error it can fix.
databaseProvider.executeQuery("SELECT * FROM (" + query + ") AS _v LIMIT 1");
GridEntry.getOrCreate(grid, gridId).setPendingQuery(query);
}
Three problems. The stated purpose is defeated by §2.3’s catch block — the message never reaches the model. LIMIT 1 does not make the probe cheap: it bounds rows returned, not rows processed, so a derived table, CTE or aggregate is materialised in full first. And GridRenderer.renderGrid then runs wrapWithLimit(query, 1) for column discovery, making the probe a duplicate of a query about to run anyway. A successful commit executes the model’s query twice before the lazy provider fetches page one.
The wrapper also breaks the add-on’s own SQL rules — update_grid_data’s description forbids SELECT * and LIMIT, and the probe uses both. Harmless to the database, but an application auditing executeQuery calls sees statements that violate the documented contract.
Recommendation: drop the probe. If a pre-commit hook is wanted, make it a strategy:
public interface QueryPreflight extends Serializable {
/** @return null when acceptable, else a message safe to return to the model. */
String check(String query);
}
Default no-op. That gives applications a supported seam for catalogue checks, dialect checks, row-limit policy or column-qualification rules, without the add-on guessing what is expensive in a given database.
2.6 [dialect: MySQL] LIMIT 0 gives the same metadata at near-zero cost
For column discovery GridRenderer needs only ResultSetMetaData — labels and types — which it currently takes from a LIMIT 1 result set. MySQL short-circuits LIMIT 0: it parses, resolves names, builds the result-set descriptor and returns without executing the plan or reading a row. The metadata is identical. The sample row is used only to decide whether to show the empty state, which the first lazy page reports just as well.
This is dialect-specific and should not go in core — PostgreSQL, SQL Server and Oracle each have their own cheapest probe (LIMIT 0, TOP 0, WHERE 1=0, or PreparedStatement.getMetaData() without execution, which is the portable option where the driver supports it).
Recommendation: extract the probe into a strategy:
public interface ResultMetadataProbe extends Serializable {
/** Column labels and types, ideally without executing. */
List<ColumnDescriptor> describe(String query);
}
Ship a JDBC default that tries PreparedStatement.getMetaData() and falls back to today’s LIMIT 1 wrapper; let dialects override. We use EXPLAIN <query> in our validation path for the same reason — it parses, resolves and plans without reading rows, catching syntax errors, unknown functions and ONLY_FULL_GROUP_BY violations for one round trip. Also MySQL-specific, also belongs behind an interface.
3. SpringAILLMProvider and AIOrchestrator
3.1 The whole turn runs on the UI thread holding the session lock
In non-streaming mode SpringAILLMProvider wraps the blocking ChatClient.call() in Flux.create, whose body executes on the subscribing thread. AIOrchestrator subscribes inline from the MessageInput submit listener. So every LLM round trip and every tool callback — for us, minutes across a dozen tool calls — runs on the Vaadin UI thread holding the VaadinSession lock. Nothing reaches the browser until it finishes: a progress indicator set immediately before the call is flushed only after the work it was meant to cover is over. The UI appears frozen for the whole turn, then jumps to the result.
Our fix is one line:
@Override
public Flux<String> stream(LLMRequest request) {
return super.stream(request).subscribeOn(Schedulers.boundedElastic());
}
subscribeOn, not publishOn: the thing to move is the subscription, since that is what invokes the blocking body inside Flux.create.
Recommendation: apply subscribeOn(Schedulers.boundedElastic()) inside SpringAILLMProvider.stream for the non-streaming path, or expose the scheduler as a constructor parameter. Two lines, large user-visible effect, and every application on that path currently has to discover it independently.
The caveat, worth documenting either way. Moving the turn off the UI thread means tool callbacks lose Vaadin’s thread locals. Ours resolved the user session from VaadinSession.getCurrent() to initialise session temp tables, and an absent session skipped initialisation without complaint, querying empty tables — a silent wrong answer, not an error. We removed the dependency by snapshotting state on the UI thread in AIController.onRequest() and having tools read the snapshot. That pattern generalises, and suggests an addition to the AIController contract: document explicitly that onRequest() is where to capture anything thread-local, because tool callbacks may not run on the UI thread. Nothing currently warns that VaadinSession.getCurrent() inside a tool is unsafe, and the failure is silent.
3.2 Every ChatResponse is discarded, so turns fail invisibly
Spring AI runs the entire tool loop inside ChatClient.call(). SpringAILLMProvider reduces the result to call().content() — a bare String — so every ChatResponse, intermediate and final, is thrown away before application code sees it. With it goes the finish reason (end_turn, max_tokens, stop_sequence, content_filtered), token usage, and whether tool calls were still pending when generation stopped.
This is why silent turn termination took so long to diagnose. A turn that hit the output ceiling mid-tool-call looked identical to one that finished normally: no exception, no error on the ResponseListener, no cause. The information needed to tell them apart existed in the ChatResponse and was discarded one layer above us. With no supported seam to reach it (§3.3), we subclassed BedrockProxyChatModel and logged finish reasons there.
Recommendation: surface response metadata — (1) a listener fired per ChatResponse including intermediates; (2) extend ResponseListener so onResponse receives finish reason and usage rather than only an optional Throwable; (3) at minimum, log a warning inside SpringAILLMProvider when a response finishes on a non-normal stop reason while tool calls are pending. That single log line would have saved us days.
3.3 There is no advisor seam
SpringAILLMProvider builds its ChatClient internally via ChatClient.builder(chatModel).defaultAdvisors(...). There is no builder override, no customizer hook, and getPromptSpec is private, so a CallAdvisor — Spring AI’s natural mechanism for observing or modifying requests and responses — cannot be added.
That left subclassing the ChatModel, which has its own trap. ChatModel#getOptions() is a default method returning empty ChatOptions, while BedrockProxyChatModel overrides it to return BedrockChatOptions. A decorator written as implements ChatModel silently inherits the default, so Spring AI’s ToolCallingAdvisor sees options that do not implement ToolCallingChatOptions, advertises no tools at all, and the model answers in prose. Tool calling disappears with no error anywhere. We lost a day before switching from delegation to extends BedrockProxyChatModel.
Recommendation: accept a ChatClient.Builder or Consumer<ChatClient.Builder> in the constructor, or add addAdvisor(Advisor). Either would have made §3.2 unnecessary and removed our need to touch the chat model.
3.4 Nothing bounds the tool-calling loop
AIOrchestrator delegates to Spring AI, whose ToolCallingAdvisor runs while (isToolCall) with no iteration limit. Neither layer offers a cap. A model that keeps calling a tool keeps being served: we observed a turn resubmit an identical query 58 times, and another re-fetch a 20K-token schema until the context window overflowed. Both burned real money and produced no answer. We added a per-turn budget inside our validate_sql tool, but a tool cannot bound tools it does not own, so loops on update_grid_data or get_database_schema stayed unbounded until we replaced the controller.
Recommendation: withMaxToolIterations(int) on AIOrchestrator, default perhaps 20, ending the turn with a distinguishable outcome when exceeded. Cost control and predictable latency both argue for it, and an application cannot implement it from outside.
3.5 Tools cannot receive application context
A tool callback’s only input is the arguments the model supplies. When the model invokes get_database_schema, there is no channel for the application to pass the user’s question, the selected filters, or anything else about the request. That matters for us because 594 tables do not fit the context window: we route to a relevant subset, and routing needs the question. Our workaround is a Supplier<TurnContext> captured in onRequest(), which works but makes the tool’s behaviour depend on hidden state rather than its arguments.
Recommendation: pass an application-supplied context object to tool callbacks, or document the onRequest()-snapshot pattern as intended. The second costs nothing and makes it a decision rather than a discovery.
4. Spring AI issues worth relaying upstream
| Issue | Effect |
|---|---|
Tool arguments serialised with Document.toString(), parsed strictly (§2.1) |
Any multi-line tool argument breaks the next request |
BedrockProxyChatModel.Builder has private constructor and fields; build() returns the base type |
A subclass cannot be produced by the builder, so client wiring must be duplicated by hand |
ChatModel#getOptions() is a default returning empty options (§3.3) |
Any implements ChatModel decorator silently disables tool calling |
Spring AI 2.0.0 does not expose Bedrock serviceTier via BedrockChatOptions |
No access to cheaper tiers without dropping to additionalModelRequestFields |
A thrown tool exception reaches the model as "Error executing tool: ..." |
Indistinguishable from a system fault, so the model does not try to correct it |
The first costs whole turns and should go first.
5. Tools that cost tokens and return nothing
5.1 get_grid_instructions is redundant by its own admission
Its description reads: “Calling this tool returns these same instructions — normally unnecessary since you are already reading them here.” The description is part of the manifest, so the model has already read the instructions before it can decide to call the tool. The only possible effect is a wasted round trip — and models take it; we logged gpt-oss-20b spending 215 output tokens on exactly this call.
Recommendation: delete it and attach the workflow text to update_grid_data’s description, where the model reads it at the moment of deciding what to commit. We did this and lost nothing.
5.2 get_grid_state returns either nothing useful or a duplicate
On a first question the grid is empty, so mandated step 1 costs a round trip to learn nothing — hence the add-on’s own “Do NOT stop after get_grid_state()”, which exists because models do stop. On a follow-up the tool returns the previous query, which is already in the provider’s chat memory (MessageWindowChatMemory, maxMessages(30)). So the model gets a second copy of something it has, and is invited to edit it. That is an active hazard: patching a previous query is how a column keeps its name while losing its correct table qualifier, producing SQL that reads plausibly and is wrong. There is also no way for an application to know whether a new question refines the previous one or replaces it, so the tool cannot be called at the right time even in principle.
Recommendation: do not mandate it in the workflow. Consider dropping it, or returning the displayed column labels rather than the SQL — that answers “what am I looking at” without handing back a string to mutate. We removed it; our best model’s score held while its inconclusive count went 2/12 → 0/12.
6. Extensibility
GridAIController is a good default assembly, closed at exactly the points where a non-trivial application needs to differ. The same is true of SpringAILLMProvider.
| Barrier (grid package) | Consequence |
|---|---|
INSTRUCTIONS_TEXT is a private constant |
Cannot describe application-contributed tools to the model |
Callbacks instantiated inside getTools() |
Cannot intercept updateData without reimplementing the class |
GridEntry is package-private |
Cannot reuse the pending/current-query lifecycle; must duplicate it |
GridFormatting is package-private |
Cannot adjust or extend cell formatting |
GridAITools.getGridState / updateGridData are package-private statics |
Cannot reuse one tool while replacing the other |
ValidationException is a private nested class |
Cannot return a safe validation message (§2.3) |
GridAITools is documented “intended only for internal use and can be removed” |
The only tool factory is explicitly not API |
Barrier (SpringAILLMProvider) |
Consequence |
|---|---|
ChatClient built internally; no override or customizer |
No advisor seam (§3.3) |
getPromptSpec is private |
Same |
Result reduced to call().content() |
All response metadata discarded (§3.2) |
stream subscribes on the calling thread |
The turn runs on the UI thread (§3.1) |
| Tool callbacks receive only model-supplied arguments | No channel for application context (§3.5) |
To change the instruction text and the commit gate — two small things — we reimplemented the controller: ~350 lines duplicating logic Vaadin already had, making GridRenderer and the AIController contract upgrade surface we must re-verify each release. The provider needed a subclass for two unrelated reasons and the chat model a third, none of which are behaviours we wanted to own.
Suggested shape
public class GridAIController implements AIController {
public void setInstructions(String instructions);
public void addInstructions(String additional);
public void setQueryPreflight(QueryPreflight preflight); // §2.5
public void setResultMetadataProbe(ResultMetadataProbe probe); // §2.6, dialect-pluggable
public void setToolErrorRelay(ToolErrorRelay relay); // §2.3
/** Called once with the built-in tools; return the set to expose. */
protected List<LLMProvider.ToolSpec> customizeTools(List<LLMProvider.ToolSpec> defaults);
}
customizeTools is the highest-value single addition: it would have let us drop two tools, keep two and add one, with the state lifecycle and renderer still Vaadin’s. Nothing else here would then have required a fork. Making GridEntry public — or exposing the pending/current query as protected on the controller — removes the last reason to reimplement.
7. Guidance findings for the built-in prompt text
The update_grid_data example uses a table that cannot exist: SELECT name AS "Employee Name", salary AS "Salary" FROM employees. For any application with a closed-world constraint (“every table must appear in the schema you were given”) this contradicts the rule, and it leaves columns unqualified, which is unsafe where the same column name appears on dozens of tables. Suggest omitting it or making it schematic (FROM <table from get_database_schema>).
“ALWAYS give every column a human-readable AS alias” can collide with application rules. Ours requires every column written as `table`.`column` so the owning table is stated where the column is used. Compatible in practice, but the two instructions arrive from different places and a reduced-reasoning model can read the alias mandate as permission to rename columns. A sentence clarifying that aliases affect display only would help.
Constraints are stated in only one place each. SELECT * and LIMIT/OFFSET prohibitions live in update_grid_data’s description; the workflow lives in get_grid_instructions. An application that replaces one string loses guidance it did not know it was replacing. Consolidating SQL rules and workflow into one overridable block would make replacement safer.
Aliases carry semantics. The dot-separated grouping convention ("Group.Column") is a nice feature, but it makes aliases structurally significant. We hit a related problem in our own jOOQ-based validation: a visitor walking the parsed query sees AS targets as Field nodes whose getName() is the alias label, so "Data Year" looked like an unqualified column reference. Not a Vaadin bug, but worth documenting for anyone doing static analysis on the generated SQL.
8. Evaluation results
12 golden questions, one iteration each, scored on whether the committed SQL binds to real tables and columns and answers the question. “Inconclusive” means the turn ended without committing.
| Model | Config | Passed | Inconc. | Notes |
|---|---|---|---|---|
qwen3-32b |
our controller, defaults | 10/12 | 0 | best result |
qwen3-next-80b |
our controller, max-tokens 8192 | 7/12 | 5 | |
gpt-oss-20b |
our controller, max-tokens 16384 | 3/12 | 2 | unexplained; see below |
qwen3-32b |
GridAIController + all our fixes |
10/12 | 2 | prior best |
qwen3-next-80b |
GridAIController + all our fixes |
8/12 | 3 | high variance |
gpt-oss-20b |
GridAIController, effort=medium |
7–9/12 | 3 | best gpt-oss config |
gpt-oss-20b |
GridAIController, no fixes |
5/12 | 6 | 3 JSON failures (§2.1), 1 loop (§2.3) |
gpt-oss-120b |
GridAIController, max-tokens 16384 |
5/12 | 2 | hallucinated a table name |
nemotron-super-3-120b |
max-tokens 16384 | 1/12 | 11 | 11 context-window overflows |
How to read this honestly. Single iterations, and we have seen ±2/12 variance on identical code.
Confident: the §2.1 and §2.2 defects are real, reproducible, and cost whole turns — the gpt-oss-20b no-fixes row is the clearest evidence, with 3 of 6 inconclusive turns being JSON failures. Removing the two useless tools cost nothing measurable and saves round trips. Relaying the validation reason to the model eliminated duplicate-resubmission loops entirely (58 identical resubmissions in one earlier run; 0 after).
Not confident: the gpt-oss-20b drop from 7–9/12 to 3/12 is unexplained. Ten of its twelve queries passed validation — real tables, real columns, correctly qualified — but only three answered the question. That is a semantic failure, not a binding failure, and may be variance or may be a genuine interaction with the stricter workflow; we would not act on it without more iterations. The qwen3-next-80b rise in inconclusive turns (3 → 5) is plausibly our own instruction “do not call update_grid_data before validate_sql returns VALID” being read as give up if validation fails, rather than anything about the add-on. We flag it because it illustrates a general risk in prescriptive workflow text: an instruction strict enough to prevent a bad commit can also prevent a good retry.
9. Priorities
If only four things change:
- §3.1 — get the non-streaming turn off the UI thread. Two lines, and the only item here a user cannot work around without understanding Reactor. Until fixed, every non-streaming application freezes its own UI for the length of a turn and probably blames its model for the latency.
- §2.2 — give every built-in tool a non-empty parameter schema. One line each, no API change, removes a whole class of provider-specific failure.
- §6 — add
customizeToolsand make the instructions overridable. Removes the reason to fork. - §2.3 — make the validation exception type public. Lets applications return actionable rejections, which is what stops models looping.
Then §3.2 (surface finish reasons) and §3.4 (bound the tool loop): the first because a silently truncated turn is currently indistinguishable from a successful one, the second because an unbounded loop spends real money. §2.5 and §2.6 are performance work and can wait. §2.1 belongs with the Spring AI team, though a documentation warning meanwhile would save the next team the fortnight it cost us.
10. What we kept
The parts we did not have to touch are the parts that would have been hardest to write. GridRenderer’s column generation, type-aware renderers, numeric alignment, dot-separated grouping and lazy loading all work well and we use them unchanged. DatabaseProviderAITools and the DatabaseProvider abstraction are the right shape.
AIController’s two lifecycle hooks are well specified, and the guarantee that onResponse fires exactly once per turn is what let us build reliable UI state handling. One qualification: it fires once, but on the commonest non-failure path it carries nothing. A turn where a tool refused the query and the model then stopped completes successfully with nothing staged — no error on the ResponseListener, no state-change event, error == null. A view keying its progress indicator on either signal waits forever; one keying on onResponse alone cannot tell “answered” from “gave up”. We inferred the difference from our own tool state. Adding a reason or outcome to onResponse would close that, and it is the same information §3.2 asks for.
The gap is not in the rendering or the abstractions. It is that the default tool assembly is a closed box, and for a warehouse of this size the defaults inside it are not the right ones.