Amira-Avizo-artificial-intelligence-banner_870x361

Artificial Intelligence (AI) methods (such as machine learning and deep learning) for imaging and analysis applications have proved to be powerful approaches for improving resolution, reducing noise and automating segmentation.

AI-based processing tools are now available in Thermo Scientific Amira-Avizo Software and PerGeos Software. Advanced Texture Supervised Classification tools and Color Auto Classification, powered by Machine Learning, were released in Amira-Avizo Software version 2019.2 and PerGeos Software version 2019.2 while Deep Learning modules (both training and inference) have been introduced in Amira-Avizo Software version 2019.3 and PerGeos Software version 2019.3.

The use of AI-based tools in Amira-Avizo Software and PerGeos Software is a major leap forward and enriches processing capabilities by allowing the ability to mix both traditional and AI-based algorithms.


Deep Learning in Amira-Avizo Software and PerGeos Software

Deep-learned neural networks have proved to be an invaluable tool for many research and industrial purposes in recent years. Using deep learning for processing images allows researchers to go beyond traditional image processing for greatly improved results.

Amira-Avizo Software and PerGeos Software provide ideal environments for deep learning. A pre/post-processing and segmentation toolbox allows enhanced data for both the training phase (creating some ground truth-segmented datasets) and the post-prediction phase (improving predicted data) while leveraging the actual model building, training and prediction steps from an experienced deep learning framework such as TensorFlow, Keras or PyTorch. The workflow for learning on a manually segmented sub-volume and performing the prediction on a complete dataset is as simple as using those two Python script modules.

Amira-deep-learning-model-trained-membrane_870x522_Web
Deep learning model trained on a subvolume for detecting membrane, prediction applied on the full volume. Data courtesy of Cardona A, et al. 2010. An Integrated Micro- and Macroarchitectural Analysis of the Drosophila Brain by Computer-Assisted Serial Section Electron Microscopy. PLoS Biol 8(10): e1000502. doi:10.1371/journal.pbio.1000502.

Amira-Avizo Software and PerGeos Software's Python integration allows you to create multiple self-contained Python environments with their own Python set of packages.

An environment dedicated to deep learning can be easily set up using a new dedicated user interface.

Amira-Avizo-python-interface_870x212

While you can configure a deep learning environment based on your preferred Python package (Keras, TensorFlow, PyTorch) and use those libraries in Python script modules (pyscro), Amira-Avizo Software and PerGeos Software’s latest versions provide a default implementation based on Keras, a high-level neural networks API, written in Python and running on top of TensorFlow.

Learn more about Python

The deep learning training modules feature a highly configurable tool for training models using state-of-the-art architectures such as Unet. The training can occur from scratch (random weights) or from pre-trained weights.

Deep-learning-training-module_870x333 The training is monitored in real time using TensorBoard to track metrics such as loss and accuracy or visualize the model’s architecture.
Deep-learning-curve_870x613 Watch accuracy and loss on training and validation sets

The deep learning prediction module then allows for reuse of the trained models on data it has never seen before.

Deep-learning-prediction-module_870x236

An AI-powered image data analysis solution for fast and accurate feature and statistics extraction is also available in a set of advanced applications for EM systems in the Amira-Avizo2D Software.

Learn more about Amira-Avizo2D Software


Applications and Use cases

Amira-Avizo-artificial-intelligence-segmentation-TEM_580x580 Data courtesy of Arganda-Carreras et al. 2015 Frontiers in Neuroanatomy Cardona et al. 2012 PLoS Biology

Image segmentation of serial section TEM (ISBI 2012 Segmentation Challenge)

Image segmentation of mitochondria blobs

Mitochondria are difficult to segment using traditional approaches because they have connections with the outer endoplasmic reticulum and an internal membrane-like structure.

The model trained with Amira-Avizo Software’s deep learning tool allows the automatic extraction of mitochondria from a FIB/SEM stack. The training was done using only a few slices, which were segmented manually with Amira-Avizo Software’s segmentation editor. It was then possible to automatically segment the rest of the stack, saving hours of manual work.

Amira-Avizo-artificial-intelligence-segmentation-mitochondria_580x580
Manual segmentation using Amira-Avizo Software's segmentation editor
Amira-Avizo-artificial-intelligence-segmentation-mitochondria_870x498 3D visualization of the mitochondrias from the automatic segmentation of the full stack with deep learning. Data courtesy of Advanced Imaging Res. Center, Kurume Univ. Sch. Med.
Amira-Avizo-artificial-intelligence-image-denoising-splitted-SEM_580x580

Image denoising of SEM

For 3D serial sectioning and 2D tiling applications, time to data versus image quality has to be carefully balanced. Usually, the data is down-sampled a lot to process it. Following acquisition, conventional algorithms, such as gaussian-smoothing and non-local-means filtering, leave artifacts. Alternatively, deep learning algorithms can be tuned in such a way that they do not induce artifacts. Processing can be done relatively quickly when a deep learning model is available. Below, we highlight a model that can quickly restore SEM images.

Download the above example

Image super-resolution

High resolution images are often needed to clearly capture the desired structure details, while lower resolution acquisition may be imposed by exposure time and dose applied to the sample.

Super-resolution deep learning algorithms can restore realistic details from lower resolution images, dramatically facilitating image segmentation.

Amira-Avizo-artificial-intelligence-image-super-resolution_870x305

Color Auto Classification based on machine learning

Based on machine learning, the Color Auto Classification tool automatically segments a color image into labels. A supervised random forest method is used.

Thin-section_Pre-Post-processed Automatic segmentation of an optical image of a thin section using Color Auto Classification. Data courtesy of Stratum Reservoir.

Texture Supervised Classification based on machine learning

Texture classification is a machine learning technique that relies on learning texture patterns from markers defined by the user and then classifying each pixel of the image according to its similarity to the learned patterns.