Is there a way to execute the UNet segmentation model in Javascript using TensorFlow JS?

Question

Is there a way to execute the UNet segmentation model in Javascript using TensorFlow JS?

I recently trained a UNet model in Python and saved the model as a Model.h5 file. Following the instructions in the tensorflow JS documentation on How to import a keras model, I used the tensorflowjs_converter tool to convert the model into a Model.json file along with several shard files.

My main query now is how to utilize this model in a JavaScript file for image segmentation. I have successfully loaded the model and displayed its summary in the console using the following code:

async function app() {
    
   model = await tf.loadLayersModel(MODEL_JSON_PATH);
   model.summary();
}

The model summary displayed in the browser console includes information about different layers and the total parameters of the model.

Now, assuming I have an image on my webpage within an img tag. How can I input that image into my model and receive the output? I have come across tf.browser.fromPixels but I am unsure about the next steps. Any assistance or references to relevant examples would be greatly appreciated.

javascript image-segmentation tensorflow.js

Answer 1

Answer №1

HTH:

const img = document.getElementById(elementID);
const tensor = tf.browser.fromPixels(img); // convert image to tensor
const model = await tf.loadLayersModel(MODEL_JSON_PATH); // load the model
const res = model.predict(tensor); // run the model
tf.dispose(tensor); // clean up the input tensor
const data = await res.arraySync(); // retrieve results as a standard array
tf.dispose(res); // clean up the results tensor
console.log(data);

UPDATED

Error encountered: expected input_1 to have shape [null,1024,1024,3] but received array with shape [512,512,3]

const resized = tf.resizeBilinear(tensor, [1024, 1024]);
const expanded = tf.expandDims(resized, 0);
...

Answer 2

HTH:

const img = document.getElementById(elementID);
const tensor = tf.browser.fromPixels(img); // convert image to tensor
const model = await tf.loadLayersModel(MODEL_JSON_PATH); // load the model
const res = model.predict(tensor); // run the model
tf.dispose(tensor); // clean up the input tensor
const data = await res.arraySync(); // retrieve results as a standard array
tf.dispose(res); // clean up the results tensor
console.log(data);

UPDATED

Error encountered: expected input_1 to have shape [null,1024,1024,3] but received array with shape [512,512,3]

const resized = tf.resizeBilinear(tensor, [1024, 1024]);
const expanded = tf.expandDims(resized, 0);
...

Is there a way to execute the UNet segmentation model in Javascript using TensorFlow JS?

Answer №1

Similar questions

Ajax Live Search Error Detected

What is the reason for Javascript XMLHttpRequest returning the octet-stream MIME type response as a string instead of binary

Using JavaScript to convert the text within a div into negative HTML code

Refinement of chosen selection

Unable to set two image layers on HTML canvas to display in different colors as anticipated

I'm experimenting with crafting a new color scheme using MUI, which will dynamically alter the background color of my card based on the API

Avoid the expansion of line decorations in Monaco editor

What are the best methods for creating genuinely random and highly secure session IDs?

Show more/less of the text snippet and main body of information

Updating a table cell triggers a change in every cell

Verify that the zip code provided in the input matches a record in the JSON data and extract the

What is the best way to create an answer label box that validates your response?

I am unable to give back an item

What is the best way to activate a function to control animation transitions?

Create a canvas and include input boxes in an HTML document

Alter the class when $dirty occurs in Angular

Refreshing the dropdown selection to a specific option using AngularJS and either JavaScript or jQuery

Are toggle functionalities triggered when an element is clicked?

Enhance your Verold model object with engaging animations

Guide on accessing oversized items (such as response objects) that are too immense for the command-line interface