Transmitting information through websockets in binary form involves converting numbers into text for encoding

Question

Transmitting information through websockets in binary form involves converting numbers into text for encoding

I have a project in which I need to transmit binary data over a websocket connection to an LED matrix. My plan is to send the binary data as a byte array, for example, creating a diagonal line from right to left on the matrix would be represented like this:

However, when attempting to send this data, regardless of the WebSocket client tool I use, the numbers are being encoded as strings.

For instance, when attempting to send the binary number 1, instead of transmitting 0x01, the socket client is sending 49 in decimal/0x31 in hex, which represents the character code for the string '1' in ASCII or Unicode.

This encoding issue seems to occur at the client side during websocket transmission, rather than being related to the Arduino code controlling the matrix or the server itself, as confirmed by monitoring with Wireshark.

The same problem persists whether using Firecamp or another client such as websocat:

Given these challenges, my question is: what is the correct method for sending binary numbers via websockets? Could it be that I am misunderstanding how the binary feature operates, and should I be sending numbers as strings and converting them back at the receiving end?

javascript websocket iot

Answer 1

Answer №1

After conducting research and introspection, I successfully identified the root of my problem. @myst also mentioned it in the preceding comment.

In the scenario where I inputted a number into the websocat command's interactive prompt without quotation marks, assuming it would be recognized as an integer, it actually treated the input as a string (which, in hindsight, makes sense for command-line tools handling arguments).

Similarly, when I attempted to add numbers by creating a test file, I mistakenly expected them to be recognized as numeric values but they were interpreted as strings.

During troubleshooting, I realized that sending binary data - like an image file - results in actual binary content being transmitted, rather than text representation.

For instance, running cat myimage.png won't display binary numbers; instead, it appears as garbled characters and spaces.

https://i.sstatic.net/ZZk8M.jpg

This "garble" isn't random; it signifies outputting sequences of binary code, with some elements lacking character encoding equivalents (resulting in blanks) while others align with character encodings but appear disorderly, resembling gibberish.

To grasp the concept better, I referred to a helpful guide on manually creating a binary file posted on Super User:

The technique involves utilizing the printf command.

$ printf "\x08\x00\x00\x10" > file1
$ hexdump file1
00000000  08 00 00 10                                   |....|
00000004

With this newfound knowledge, I crafted a binary file containing a simple hexadecimal progression from 1 to 8, which I then piped into the websocat utility:

printf "\x01\x02\x03\x04\x05\x06\x07\x08" > binfile \
> cat binfile | websocat -b ws://localhost:3002

This action yielded the intended result:

https://i.sstatic.net/wAkHP.jpg

Furthermore, upon revisiting the process without involving a file write operation, executing the command directly worked flawlessly:

printf "\x01\x02\x03\x04\x05\x06\x07\x08" | websocat -b ws://localhost:3002

Although the method didn't function as anticipated in Firecamp initially, discussions are ongoing with their engineers, including testing on a canary version for potential resolution.

I have now resolved the issue and am prepared to revisit and revise some of the existing code :P

Additional Insights

Collaborating with the Firecamp developers revealed that their message types for Array Buffer and Array Buffer View inadvertently conveyed ASCII codes instead of raw numerical values. Subsequent adjustments have rectified this matter (confirmed via a canary release), scheduled for inclusion in v2.0.

Answer 2

After conducting research and introspection, I successfully identified the root of my problem. @myst also mentioned it in the preceding comment.

In the scenario where I inputted a number into the websocat command's interactive prompt without quotation marks, assuming it would be recognized as an integer, it actually treated the input as a string (which, in hindsight, makes sense for command-line tools handling arguments).

Similarly, when I attempted to add numbers by creating a test file, I mistakenly expected them to be recognized as numeric values but they were interpreted as strings.

During troubleshooting, I realized that sending binary data - like an image file - results in actual binary content being transmitted, rather than text representation.

For instance, running cat myimage.png won't display binary numbers; instead, it appears as garbled characters and spaces.

https://i.sstatic.net/ZZk8M.jpg

This "garble" isn't random; it signifies outputting sequences of binary code, with some elements lacking character encoding equivalents (resulting in blanks) while others align with character encodings but appear disorderly, resembling gibberish.

To grasp the concept better, I referred to a helpful guide on manually creating a binary file posted on Super User:

The technique involves utilizing the printf command.

$ printf "\x08\x00\x00\x10" > file1
$ hexdump file1
00000000  08 00 00 10                                   |....|
00000004

With this newfound knowledge, I crafted a binary file containing a simple hexadecimal progression from 1 to 8, which I then piped into the websocat utility:

printf "\x01\x02\x03\x04\x05\x06\x07\x08" > binfile \
> cat binfile | websocat -b ws://localhost:3002

This action yielded the intended result:

https://i.sstatic.net/wAkHP.jpg

Furthermore, upon revisiting the process without involving a file write operation, executing the command directly worked flawlessly:

printf "\x01\x02\x03\x04\x05\x06\x07\x08" | websocat -b ws://localhost:3002

Although the method didn't function as anticipated in Firecamp initially, discussions are ongoing with their engineers, including testing on a canary version for potential resolution.

I have now resolved the issue and am prepared to revisit and revise some of the existing code :P

Additional Insights

Collaborating with the Firecamp developers revealed that their message types for Array Buffer and Array Buffer View inadvertently conveyed ASCII codes instead of raw numerical values. Subsequent adjustments have rectified this matter (confirmed via a canary release), scheduled for inclusion in v2.0.

Transmitting information through websockets in binary form involves converting numbers into text for encoding

Answer №1

Additional Insights

Similar questions

The authentication middleware is being executed for all routes within my app.js file, despite my intention to only apply it to a single route

Creating an Engaging Data Visualization: Blending Candlestick Insights with Line Graphs

I am looking to retrieve a sophisticated/nested JSON data using jQuery

Inserting an item into a list

The Ajax Control Upload consistently defaults to the default value and disregards any text input selected from the drop-down list

Can JQuery be used to detect text input in real-time in a textarea field?

When using TypeScript with custom components as children in React, the `type` returned by React.Children is a string representing the function

What is a global variable used for in JavaScript?

If the cursor hovers over an image, the onmouseover function does not work properly

Incorporating ZeroClipboard or ZClip (button for copying to clipboard) using JQuery's Ajax functionality

Angular Typescript error: Trying to assign a value to 'someProperty' property of an undefined object

Using Javascript to dynamically add and remove elements on click

Calculating date discrepancies with JavaScript

Obtaining the initial row information from jqGrid

Sorting arrays in javascript

The Lerna command for adding packages fails with an error message stating that the packages are not related

Is there a way to determine if the browser window is currently in use?

Adding options to a select element using JavaScript dynamically

Ways to show an input box even when there are no results found

A guide to storing a JavaScript variable in a MySQL database using Express.js