Refresh outfit designs or retrieve information from fragment shader post-processing

Question

Refresh outfit designs or retrieve information from fragment shader post-processing

While working on image processing in the fragment shader (specifically thresholding), I find myself wanting to access the processed result from JavaScript so that I can save the modified image using standard JavaScript methods.

I send the original texture to be segmented to the fragment shader as a uniform, where the thresholding happens. The result renders correctly on screen. But now my goal is to extract this thresholded image for saving purposes.

Considering the fact that the fragment shader runs over pixels rendered on the screen, I'm not even sure if it makes sense to try and access the processed image.

Despite the constraints, I am curious if there is an effective technique to execute complex image processing tasks within the fragment shaders and then somehow retrieve the resulting data for further use.

Another scenario I have in mind involves flood filling based on the mouse cursor position, among other things.

JavaScript's speed is insufficient for my needs, and while web workers may offer some improvement, I was hoping to capitalize on GPU acceleration for my image processing requirements.

(Just for context: I am currently using THREEJS along with custom shaders - though this might not be relevant information)

Warm regards, Nicolas

javascript image-processing three.js glsl webgl

Answer 1

Answer №1

If I have grasped your issue correctly, there are a couple of ways to tackle it. Imagine we have a canvas at our disposal. We can display the processed image on this canvas and proceed to save it either as a JPEG or PNG blob:

canvas.toBlob(
    (blob) => {
        /* Handle the blob, such as sending it to a server */
    },
    'image/jpeg',
    0.95
);

Alternatively, you can save it as a data URL:

const imageAsDataUrl = canvas.toDataURL('image/png');

In case you need access to the pixel values of the image, you can retrieve them using a WebGL call (assuming that the gl context is available):

const pixels = new Uint8Array(4 * canvas.width * canvas.height);
gl.readPixels(0, 0, canvas.width, canvas.height, gl.RGBA, gl.UNSIGNED_BYTE, pixels);

Answer 2

If I have grasped your issue correctly, there are a couple of ways to tackle it. Imagine we have a canvas at our disposal. We can display the processed image on this canvas and proceed to save it either as a JPEG or PNG blob:

canvas.toBlob(
    (blob) => {
        /* Handle the blob, such as sending it to a server */
    },
    'image/jpeg',
    0.95
);

Alternatively, you can save it as a data URL:

const imageAsDataUrl = canvas.toDataURL('image/png');

In case you need access to the pixel values of the image, you can retrieve them using a WebGL call (assuming that the gl context is available):

const pixels = new Uint8Array(4 * canvas.width * canvas.height);
gl.readPixels(0, 0, canvas.width, canvas.height, gl.RGBA, gl.UNSIGNED_BYTE, pixels);

Refresh outfit designs or retrieve information from fragment shader post-processing

Answer №1

Similar questions

Oops! An issue occurred on the server: ReferenceError - 'window' is not defined. This error occurred during

Creating an Android application with a masking feature

Changing the size of the logo as you scroll through the page

Having trouble with Vue and Vuex not returning the correct value in HTML?

What's the best way to implement image size and type validation, specifically for .jpg and .png files, using Multer?

Whenever I attempt to include state in a React class that is declared as a constant, I consistently encounter the error message: "TypeError:

The button fails to appear when navigating

What is the proper way to reference a property's value within another property of the same JavaScript Object?

Easily transforming data between JSON and CSV formats in JavaScript using user-defined headers

Unable to locate the Chart object within the chartjs-plugin-labels.js file

Disable the button until all input fields contain text in ASP

Is using multiple meshes with the same material in Three.js beneficial for performance?

Error message: The module '@project-serum/anchor' does not export the object 'AnchorProvider' as intended

Activate Jquery to display the submenu when clicked and conceal any other open submenus at the same time

Angular - Issue with setting default value in a reusable FormGroup select component

Guide on using Ajax to transmit data to Struts2 action class

Tell webpack to exclude a specific import

What is the best way to retrieve JSON data using JavaScript within a loop?

Resolving a Tricky Challenge with jQuery's

What method would you recommend for modifying HTML text that has already been loaded using JSP?