From pixels to large-scale, interpretable properties: A deeper look into microscopy images using deep representational learning

Quantitative analysis of microscopy imaging data often starts by extracting local features, such as the position or movement of cells or macromolecular structures. Integrating this local information into a coherent description of larger-scale organization and dynamic reorganization remains a challenge. However, such large-scale quantitative descriptions are critical for an in-toto characterization of complex biological phenomena. Here, I highlight recent advances in computer vision showing that learned representations in deeper layers of generative AI are often continuously and semantically structured, allowing continuous variation of specific high-level visual features that characterize, for instance, a facial expression or a visual object. I argue that leveraging these possibilities in models trained to generate microscopy imaging data can reveal interpretable ‘semantic’ features capturing relevant large-scale modes of biological variation. Moreover, I will show that metrics based on representational similarity in deep learning models can be used for high-level structural comparisons when applied to imaging data. These examples illustrate the broad applicability of these novel image-analysis approaches that, based on the representational spaces of deep-learning models, can reveal more global, high-level features characterizing structural variation in a biological system as a whole.

The SCALE lunch seminar will take place in a hybrid form. All interested are invited to attend in person at the lecture hall of the MPI-BP with coffee and a light lunch in the atrium afterwards. At the same time, the lecture will also be streamed online via zoom. The zoom link will be sent by e-mail to the SCALE mailing list. To subscribe to the mailing list, please send an e-mail to scale-office@uni-frankfurt.de.