Machine Learning

Machine learning (ML) provides your machines with the ability to adjust their behavior based on models that recognize patterns or make predictions.

Common use cases include:

• Object detection, which enables machines to detect people, animals, plants, or other objects with bounding boxes, and to perform actions when they are detected.
• Object classification, which enables machines to separate people, animals, plants, or other objects into predefined categories based on their characteristics, and to perform different actions based on the classes of objects.
• Speech recognition, natural language processing, and speech synthesis, which enable machines to verbally communicate with us.

Viam provides two services that together enable visual machine learning capabilities: the ML model service and the Computer Vision service. For other use cases, consider creating custom functionality with a module.

Machine learning models

Machine Learning (ML) models are mathematical models that can recognize patterns. Currently Viam supports TensorFlow Lite, TensorFlow, ONNX, and PyTorch.

• You can upload externally trained models on the MODELS tab in the DATA section of the Viam app.
• You can train models on data from your machines using different training scripts.
• You can use ML models from the Viam Registry.
• You can use a model trained outside the Viam platform that’s already available on your machine.

For more information, see ML Models and Training Scripts.

ML Models

Machine learning models are mathematical models that can recognize patterns. You can use them on your machines to interpret the machine's surroundings.

Training scripts

The Viam registry hosts custom Python ML training scripts, which you can use to train machine learning models.

ML Model service

The ML model service deploys and runs a machine learning model, such as a TensorFlow or ONNX model, on your machine and makes its output accessible to other services. For example, the Computer Vision mlmodel service, which can detect or classify objects, is built to work with the inferences from an ML model service. As a detector, the service uses these inferences to interpret image data from images on your computer or a camera, drawing bounding boxes around objects. As a classifier, the service returns class labels and confidence score based off the inferences the underlying ML model makes from image data.

Next steps

For comprehensive guides on using data capture and synchronization together with the ML model service, see:

Capture and sync image data

Capture images from a camera on your machine and selectively sync images to the cloud with filtering.

Train computer vision models

Use your image data to create and label a dataset and train a computer vision ML model.

You can also follow one of these tutorials to see ML models in action:

Helmet Monitor

Get an email alert if people are not wearing hard hats.

Colored Object Follower

Instructions for detecting and following a colored object with a rover, like a SCUTTLE robot.

AI Companion Robot

Add object detection, speech recognition, natural language processing, and speech synthesis capabilities to a machine.