Always-Listening Voice Commands for Vaadin web applications

Make your Vaadin web application more user-friendly and accessible with always-listening voice commands! With this quick 15-minute tutorial, you can integrate voice commands like "Alexa," "Hey Siri," and "Computer" into your application using the Picovoice Porcupine Wake Word Engine. Create a working demo and enhance the user experience of your web application with the following step-by-step guide.

Wake Word Detection is also known as Keyword Spotting, Hotword Detection, Always-Listening Voice Commands, Trigger Word Detection, and Voice Activation.

Short on time? Just clone the repo, create an accesskey, and run it.

Set up the Project

The first step is to create a new Vaadin Java application project:

1. Download a new project from start.vaadin.com with an empty view.

2. Create a new Java API for always-on keyword detection and import Picovoice libraries:

package com.example.application;

import com.vaadin.flow.component.UI;
import com.vaadin.flow.component.dependency.JsModule;
import com.vaadin.flow.component.dependency.NpmPackage;

@NpmPackage(value = "@picovoice/porcupine-web", version = "2.1.16")
@NpmPackage(value = "@picovoice/web-voice-processor", version = "4.0.5")
@JsModule("./porcupine-integration.js")
public class Porcupine { 

    boolean started = false;

    public Porcupine(String picovoiceAccesskey) {
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.key=$0;"
                    , picovoiceAccesskey);
    }

    public void start() {
        this.started = true;
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.start()");
    }

    public void stop() {
        this.started = false;
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.stop()");
    }

    public boolean isStarted() {
        return this.started;
    }
}

This is the server API available for the rest of the Vaadin Java application.

3. Create porcupine-integration.js in the project’s frontend folder.

import { WebVoiceProcessor } from "@picovoice/web-voice-processor";
import { PorcupineWorker, BuiltInKeyword } from "@picovoice/porcupine-web";
import modelParams from "./porcupine_params.js";

// a global 'vaadinPorcupine' integration instance is enough
window.vaadinPorcupine = window.vaadinPorcupine || {
    key: null,
    async start() {
        console.log('Starting wake word detection');
        window.vaadinPorcupine._worker = window.vaadinPorcupine._worker ||         
        await PorcupineWorker.create(
            window.vaadinPorcupine.key,
            [BuiltInKeyword.Computer],
            window.vaadinPorcupine.keywordDetectionCallback,
            {base64: modelParams }
        );
        await WebVoiceProcessor.subscribe(window.vaadinPorcupine._worker);
    },
    async stop() {
        console.log('Stopping wake word detection');
        await WebVoiceProcessor.unsubscribe(window.vaadinPorcupine._worker);
    },
    keywordDetectionCallback(detection) {
        console.log(`Detected keyword: ${detection.label}`);
        const e = new CustomEvent("voice-wakeword", 
                    { "detail": detection.label });
        document.body.dispatchEvent(e);
    }
}

This is the client-side part of the API integrating the in-browser wake word detection library.

4. Download the Porcupine model (i.e., Deep Neural Network). From the project frontend folder, run the following to turn the binary .pv model into a base64 string model.

echo "const model_params='$( cat porcupine_params.pv | base64 )';\nexport default model_params;\n\n" > porcupine_params.js

5. Get your Picovoice AccessKey

Go to Picovoice Console's dashboard. Copy your AccessKey.

6. Add a test button to the EmptyView.java to turn the wake word detection on and off:

final Porcupine porcupine = new Porcupine(System.getenv("PICOVOICE_ACCESSKEY"));

add(new Button("Start/Stop wake word detection", e -> { 

    if (!porcupine.isStarted()) {porcupine.start(); }

    else { porcupine.stop();}

}));

7. Run the application using your own AccessKey:

PICOVOICE_ACCESSKEY=your_accesskey_here ./mvnw

Source Code and more examples

The source code for a fully-working Vaadin demo with Porcupine is available in my GitHub repository. I’ve added a custom event and handler example there, and later I’ll show how to train and add your own custom wake word using Picovoice Porcupine.

Need help or have questions? Join our Discord server, and our team will be happy to help you!