getPage().executeJs("...") without waiting for result possible?

Flow
1

Hey, I am trying to execute JavaScript without awaiting confirmation that it was run, since I do not need any results back and I’d like to avoid filling up memory.

In my case I am sending “video” / a bunch of images very fast via javascript and even though I am synchronizing the fps on the server and client, the client might still execute/process the javascript code much slower than expected which leads in the memory quickly filling up because all of those pending images that are not sent yet.

    public void sendImage(Mat image) {
        try {
            byte[] jpg = encodeMatToJPG(image);
            String base64 = Base64.getEncoder().encodeToString(jpg);

            ui.access(() -> {
                ui.getPage().executeJs(
                        """
                                let img=document.createElement('img');\
                                img.style.position = 'fixed';\
                                img.style.height='100vh';\
                                img.style.width='auto';\
                                img.style.objectFit='cover';\
                                img.src='data:image/jpg;base64,' + $0;\
                                window.addNewImage(img);
                                """, base64);
                ui.push();
            });
        } catch (Exception e) {
            AL.warn("Possibly errors during sending image to ui.", e);
        }
    }
2

Doesn’t seem to be possible.
My workaround was simply adding a limit of pending javascript results that are allowed and skip/discard new frames if that limit is hit.

3

I suspect that what you’re seeing is not that it waits for confirmation but rather that it buffers new invocations while the previous batch is being sent and only sending a new batch once the previous has been delivered. It only keeps invocations in memory after they have been sent if you have interacted with the PendingJavaScriptResult object returned from executeJs. The logic that sets up the confirmation wait is in UidlWriter.

This means that you might actually even benefit from waiting for confirmation since that might help you avoid doing redundant invocations that will just end up buffered. By adjusting your timing based on received confirmations, you could send the next one at exactly the right time instead of causing buffering.

At the same time, it also seems like Flow might not be the best tool for what you try to achieve. Sending a stream of images to the browser might better handled using WebRTC together with some client-side logic to handle rendering. In that way, you’re using a stack that is optimized for low latency and also a transport mechanism that sends binary data as binary data without the 33% overhead that you get from base64 encoding the data and sending it as text.

4

Yeah thanks that is much more sane.
Funnily enough it seems to work though with 30fps and png quality at 60% or jpg quality at 100%, however increasing the png quality causes the issues I mentioned, maybe due to network latency then, no idea.

5

Probably a combination of network capacity (bandwidth and/or latency) and buffer sizes in the network stack. The previous websocket frame must be transmitted before a new one can be accepted. I don’t know if Atmoshpere only waits for the TCP stack to accept the bytes or if it also waits for some kind of acknowledgement before proceeding to the next frame but the result is anyways that there will be a delay until it asks for more data from the application.

This also means that you might have to test against different network scenarios and maybe even clients with different processing speed if you want to find a combination that works in all cases. Or alternatively, you implement some kind of adaptive approach that adjusts the transmit speed based on how quickly the data is received.

1 Like