https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwtY_K_Qnk3nsd76BjZG2yqDB2CO07r2yR0aZFhW3zJuqB0zrkf2XQJn4oqZLohp-YRQ9EvOdBmN_8K3CIuLF4FQyNHuT58P1L8uoADKq1AoKc9VLRO_a_yhWpEO17qDfTf75ppoG2RXns/s1600/0_NyEo0SUJOJKE26c6.png
Posted by Daniel Smilkov, Sandeep Gupta, and Vishy Tirumalashetty
TensorFlow.js is an open-source library that lets you define, train, and run machine learning models in Javascript. The library has empowered a new set of developers from the extensive JavaScript community to build and deploy machine learning models and enabled new classes of on-device computation. TensorFlow.js runs in all major browsers, server-side in Node.js, and as of recently, in WeChat and React Native providing hybrid-apps access to ML without having to leave the JS ecosystem. Today, we are excited to bring a new way for JavaScript developers to easily train a custom model on your own data and deploy it in your JavaScript application using the GCP AutoML service.
One of the main goals behind TensorFlow.js is to empower JS developers to use the benefits of ML without having to create their own models. There are two existing ways to bring the power of ML to your JavaScript applications: use one of our pre-packaged models, or fine-tune a model on your own data.
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
<script src="https://cdn.jsdelivr.net/npm/
@tensorflow-models/posenet">
</script>
<img id="tennis" src="tennis.jpg"/>
<script>
async function run() {
const net = await posenet.load();
const image = document.getElementById('tennis');
const pose = await net.estimateSinglePose(image);
console.log(pose);
}
run();
</script>
If you have your own data, you can start with a pre-trained model and fine-tune it. The pre-trained models and the training itself can live in Python or JavaScript. If the training is done in Python, you can seamlessly convert the SavedModel or Keras model to TensorFlow.js using our converter. If the model is already in TensorFlow.js, you don’t need to do any conversion. This codelab shows you how to take MobileNet, an image classification model optimized for edge devices, and fine-tune it on your own image dataset, all in JavaScript.
TensorFlow.js + AutoML
We are excited to announce the integration with Cloud AutoML Vision, a Cloud service that enables developers to train a custom model on their labelled data. If your use-case is related to image classification or object detection, you can use the UI to upload your data, train an edge model and directly export it to a TensorFlow.js model, without any coding. Note that custom models can be directly trained using the open-source TensorFlow.js library (or Python TensorFlow), but AutoML provides an easy-to-use service that automates this process for you.
To make it easy to deploy and run the models efficiently in TensorFlow.js, we published a @tensorflow/tfjs-automl library on NPM. Once you export the model, you can follow our step-by-step guides for image classification and object detection. The guides use the NPM library and show you how to load the pre-trained model in the browser and make a prediction on an image.
If you already have a custom model, AutoML can still be incredibly useful. AutoML Vision Edge models are optimized to a small memory footprint and offer low latency while delivering high accuracy. We worked with several customers and partners and got some amazing early results and feedback. CVP, a business and technology consulting company, has been evaluating some vision scenarios to help with workplace safety on edge-based models used in remote locations. Here is what Cal Zemelman, Director of Data science at CVP said on their results:
“We are working on a smartphone app for a Federal agency that performs image classification to help prevent accidents and improve public safety. The app is mainly written in JavaScript as a Progressive Web App to allow cross-platform usage, so we were training a Keras model and converting it afterwards to TensorFlow.js. With the new “Export to TensorFlow.js” feature, we were able to test out using AutoML to greatly streamline our model creation workflow. Previously, our best model was a retrained ResNet50 which got to about 91% accuracy. The middle of the road balanced performance model from AutoML Image Classification hit ~99% accuracy in 5 node hours. It is also 1/5 the size of the previous model and we saw a 60% improvement in the speed of inference.”
We are very excited about the early results our customers and partners are seeing and look forward to hearing more with this release.
You can train an AutoML Vision Edge model today. Once you train the model, you can export the TensorFlow.js model to your client-side application. With AutoML Vision Edge, you only pay for training for any usage beyond 15 node hours you get for free. There are no charges for exporting or client side predictions.
Next steps
