Divide and conquer- Lighttpd's mod_wstunnel combines separate messages by using UNIX socket communication to the backend server

In my experience with lighttpd's mod_wstunnel, I have encountered a peculiar issue. When I send two messages in quick succession from the backend using a UNIX socket to communicate between lighttpd and the backend, I noticed that lighttpd logs show that it receives these messages as a single combined message.

However, if there is even a minimal delay of 1ms between the two write calls to the UNIX socket, lighttpd receives the messages separately and forwards them as such to the final client.

To demonstrate this issue, I created a simple proof of concept setup:

The configuration file for lighttpd (which can be launched using lighttpd -D -f lighttpd-poc.conf):

server.document-root = var.CWD 
server.bind = "0.0.0.0" 
server.port = 8042 

... (additional configurations) ...

Furthermore, the HTML page being served where the problem manifests easily:

<!DOCTYPE html> 
<script> 
    ... (WebSocket setup script) ...
</script> 

And finally, the simplified backend code responsible for handling websocket requests:

#include <iostream> 
#include <thread> 
#include <chrono> 

using namespace std::chrono_literals; 

int main() 
{ 
    ... (backend setup) ...

    while (true) { 
        ... (logic for processing messages) ...
            // std::this_thread::sleep_for(1ms); // Uncommenting this line resolves the issue. 
            ... (sending second message) ...
        } 
    } 

    return 0; 
} 

It's worth noting that uncommenting the std::this_thread::sleep_for(1ms) line in the backend code eliminates the message combining issue. This problem leads to failed JSON parsing by the browser when navigating to the specified URL due to the merged messages being received.

While workarounds exist like utilizing libraries or including the sleep function in the code, identifying the root cause of this buffering phenomenon is crucial. Understanding why and where this buffering occurs in the kernel will provide insights into resolving the issue effectively. Additionally, exploring solutions beyond temporary fixes is essential for long-term stability and optimal performance.

If relevant, the version of lighttpd being used is 1.4.55.

Answer №1

One user suggested in a previous comment that the issue of concatenating JSON messages in lighttpd/mod_wstunnel is addressed by RFC 6455, which mentions that intermediaries may combine or separate frames.

lighttpd mod_wstunnel functions as a websocket tunnel that generates websocket frames from backend data without examining the content. This allows the backend to transmit data in various formats, such as JSON, with no consideration for message boundaries within the JSON structure.

An alternative approach involves utilizing lighttpd mod_proxy with modifications to lighttpd.conf using

proxy.header += ("upgrade" => "enable")
. By implementing the websocket protocol at the proxy backend, you can take control over the websocket framing process.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What are the steps to restrict the scope of a <style> declaration in HTML?

Currently, I am in the process of creating a user interface for a webmail. Whenever I receive an email, I extract its content and place it within a <div> element for display purposes. However, one challenge that I am facing is that each email contain ...

What is the best way in Javascript to retrieve and process multiple stream chunks from a single API request using the fetch() method

I'm dealing with a Node/Express backend where there is a lengthy (10 - 15sec) API call that is responsible for setting up a user's account. To initiate this action from the front-end, I simply use a fetch('my/api/url') GET request. My ...

Obtain the string value of `reader.result` from the `FileReader

Struggling to retrieve a string in a reader.onloadend but it's constantly returning my entire function. Here is the function I'm using: uploadFile() { let vm = this; var file = vm.$refs["testeLogo"].files[0]; var reade ...

Tips for triggering a postback with jquery autocomplete when an option is selected

TEST AREA I've put together a basic demo of jquery-ui autocomplete. QUERY How do I trigger a postback when a selection is made from the autocomplete list? HTML <select id="ddl"></select> <input id="field" type="text"></input& ...

Flask application experiencing JavaScript issues on local host, yet functioning properly when accessed via 127.0.0.1

My web application is built using Python with Flask for the server, and HTML with JavaScript for the user interface that utilizes callback functions and POST methods. While the application runs smoothly with redirections and REST API calls at: Upon click ...

Guide for implementing async/await in conjunction with the eval() function within JavaScript

I'm currently using the eval function to evaluate strings and adding await to it to ensure all values are obtained, but unfortunately the await is not functioning correctly. Here is a snippet of my code: if (matchCard.card.status != "notstarted& ...

Failing to retain hyperlinks with ajax

Currently, I am utilizing ajax to transmit data from a sophisticated custom field wysiwyg editor. Within this setup, the specific div with the class 'bio' is what I'm addressing. The issue arises when the data is retrieved - all the original ...

Trouble with displaying my three.js 3D model

I've tried multiple solutions I found, but none have fixed the issue of not being able to display 3D models. Many have suggested that it could be a problem with the camera, but so far I haven't been able to resolve it. Any assistance would be gre ...

What is the significance of enclosing Button within CardActions in Material-UI?

As I explored the Card documentation on MUI, I found that the Button inside the card is always enclosed within CardActions. However, I couldn't quite grasp the purpose of this tag as it wasn't clearly explained in the documentation. The only noti ...

The value of a Highcharts series does not match that of a Data Source

Encountering an issue with Highcharts where the value of this.y does not reflect the data source. The discrepancy is apparent in the image below. Uncertain if this is a bug or user error. Screenshot illustrating the problem You can view my Demo here: htt ...

How to verify that the user is using the most up-to-date version of the application in React Native

Currently, I am focused on the application and have implemented API endpoints that return the latest version along with information on whether the update is mandatory. Within the application flow, a GET request is sent to these API endpoints to check the ...

Mastering the use of JArrays in the copy activity of Azure Data Factory

Currently, I am facing an issue with my copy activity where I am using a MongoDB JSON file as the source data and attempting to sink it into an Azure SQL Database. The problem arises from the fact that my JSON file consists of an array of email addresses. ...

What is the best approach for managing a One-to-One JSON loop?

My application features a class called "User" with an attribute that points to another user. The issue I'm facing is that one user has another user as their attribute, creating a loop. This situation differs from the typical scenario where one class h ...

Transforming JSON data from JavaScript to Python JSON format

I am working with JavaScript JSON data: var data = '{a: 10, b: 20}' Now I need to convert this into Python JSON. Any suggestions on how to achieve this? Current Scenario: I have a script that extracts text from a website. The data on the webs ...

I am currently experiencing difficulties with loading files in Magento even though they are present on the server

I am experiencing difficulties with a Magento 1.5.1 installation that was not a fresh setup, but rather one that was transferred to another server (files and database copied over). The issue I am facing is related to the failure of my Javascript files to ...

Tips for running a function at regular intervals in NodeJS

I've experimented with the setInterval() method before. While it seemed ideal, the problem I encountered was that it didn't start the first call immediately; instead, it waited for X seconds before producing the desired value. Is there an alterna ...

Is it true that Javascript does not allow for saving or outputting actions?

After coming across this question, I discovered a way to extract a specific element from a Google translate page using Javascript. However, I also learned that it is nearly impossible to directly save something to the clipboard in Javascript without user i ...

Unable to convert the array token into an instance of the model class

I'm currently attempting to send a JSON array list to a Spring REST endpoint and implementing my logic as shown below: Model Class : public class ABCDEF{ private String A; private String B; private ArrayList<Long> C; private ArrayList<Long ...

Is it possible to alter the name of a slot before displaying the element in the shadowDOM, depending on the slot utilized in the DOM?

In my project, I am working on implementing different features for both desktop and mobile devices. Some of these features can be replaced by slots. My goal is to have a slot that can be either replaced by a desktop slot called poster-foreground, or a mobi ...

Is there a way for app.use to identify and match requests that begin with the same path?

Given that app.use() responds to any path that starts with /, why does the request localhost:3000/foo match the second method instead of the first? app.use("/",express.static('public'), function(req,res,next) { console.log(& ...