Novel deep learning-based software detects and tracks individual cells with high precision

Cell growth and division are two of the most fundamental and essential features of life, and closely monitoring cell changes over time can give scientists key insights into dynamics of these biological processes. Time-lapse microscopy allows scientists to detect and track cells, but produces huge amounts of data that is nearly impossible to sort through manually.

Now, however, powerful data processing capabilities of modern deep learning models offer techniques to sort through so much imaging data. Assistant Professor of Biomolecular Engineering Ali Shariati and doctoral student Abolfazl Zarageri together with several student researchers in the Shariati lab have developed and released a new deep learning model called “DeepSea,” one of the only tools with the ability to segment cells, track them and detect their division to follow lineages of cells. DeepSea, which is detailed in a new paper in Cell Reports Methods, is one of the highest-accuracy tools of its kind. 

DeepSea’s model training dataset, user-friendly software, and open source code is available for use on the DeepSea website, and Shariati and his team of researchers have already used it to make new discoveries about stem cell growth and division. 

“The model is more efficient, has fewer parameters, and both segmentation and tracking are integrated into a user-friendly software,” Shariati said. “The software allows you to train the model for any cell type of interest, paving the way for future discoveries.”

Time-lapse microscopy, which captures a series of images from a microscope over time, allows researchers to monitor single cells over the course of an experiment to track phenomena such as differentiation — when stem cells become a specific type of cell — or change in shape and size over time. This can allow scientists to make new biological discoveries by measuring dynamics of cell biological phenomena at single cell level.

Once the scientists have gathered images, they need to carry out two main tasks: segmentation, or identifying the borders of individual cells from each other and the background; and tracking, or following a cell from one frame to the next. From that point, the researchers can further investigate characteristics such as size, shape, texture, how they move and change their shape, and more. 

Manually sorting through microscopy images is tedious, time consuming, and ultimately a task better suited for a computer — which is where DeepSea comes in. This efficient deep learning model can perform segmentation in less than a second, and track cells with 98 percent accuracy. 

Enabling the software to detect cell division was a particularly unique and challenging aspect of this project, as there are few if any other situations in which artificial intelligence and computer vision must track one object transforming into two. 

“This is a very unusual problem for object tracking,” Shariati said. “If you want to track a car or something, the car will be moving around and you can use machine learning and computer vision to follow them as they move. But for cells all of a sudden one object becomes two, and that's a fundamentally new problem that we needed to solve, and we were able to do so.” 

DeepSea is a generalizable model, meaning it can be used to track a variety of cell types. It uses a modified version of a popular model, 2D-UNET, with significantly less parameters to achieve both fast speeds and high accuracy. 

“We compared our model with some of the best cell segmentation models, and ours is now showing the best results in terms of precision, and speed, especially for these cell types,” said Zarageri, an electrical and computer engineering Ph.D. student in Shariati’s lab who led the creation of the software.

The researchers trained DeepSea using a dataset of images of cells manually segmented from their backgrounds, a time-intensive process as the images are often low-contrast and the cell bodies hard to make out. To aid in this process, the team developed another software tool to help crop, label, and edit the microscopy images of cells, which is also available at DeepSeas.org.

The training dataset included images of lung, muscle, and stem cells, meaning DeepSea achieves high precision across different cell types. More cell types can be added into future versions of the model. 

The researchers used DeepSea to study size regulation of embryonic stem cells, which are the foundation of multicellular life and can differentiate into every other cell type. They came away with the new discovery that embryonic stem cells, which are known to divide unusually fast, regulate their size so that smaller cells spend a longer time growing before producing the next generation of cells. 

“We found that if an embryonic stem cell is born small, they kind of know that they are small, so they spend more time growing before they go on and divide again,” Shariati said. “We do not know why and how exactly this happens, but at least that phenomenon is there.” 

In the future, the researchers plan to apply their existing software to gather data to study spatial relationships between cells, and how the cellular features are organized in 3-D patterns to form structures. 

The researchers also aim to resolve bottlenecks they have noticed in using their deep learning models, such as the lack of labeled images of cells that are used to train the models. They plan to use a class of machine learning frameworks called Generative Adversarial Networks (GANs) to create new synthetic data, images of cells that are already annotated to cut down on the time it takes to create labels. The researchers would then have large libraries of datasets of any cell type of interest with minimal human involvement.