
Reconstructing the mind's eye: fmri-to-image...
NeurIPS (spotlight), 2023
Introduced a novel method to reconstruct seen images from fMRI brain signals by aligning brain activity with image embeddings and a diffusion model.
Sophont builds open, universal medical AI that understands pathology, neuroimaging, clinical text and more—empowering clinicians and researchers worldwide.
Read our medical AI manifesto →Collaborate with us at the frontier of open medical AI. Whether you're a healthcare provider, research institution, or investor, let's connect.
NeurIPS (spotlight), 2023
Introduced a novel method to reconstruct seen images from fMRI brain signals by aligning brain activity with image embeddings and a diffusion model.
ICML, 2024
Demonstrates that a shared fMRI-to-image model can be fine-tuned on just one hour of data from a new person, enabling high-quality brain decoding with minimal data.
Nature Biomedical Engineering, 2024
Presents RoentGen, a model that can generate realistic, high-resolution X-ray images from clinical text prompts, aiding in data augmentation and education.
AAAI, 2024
This model generates preliminary reports from chest X-rays, significantly improving radiologists' diagnostic efficiency without compromising accuracy.
Blog post, 2025
Provides a comprehensive overview of how LLMs are evaluated for medical applications, highlighting the critical need for more robust, multimodal evaluation frameworks.
Blog post, 2021
Details the community-led creation of DALL·E mini, an open-source text-to-image model, explaining its architecture, training, and impact.
JOSE, 2021
A lightweight, open-source JavaScript library for visualizing 3D brain maps directly in a web browser to share and explore fMRI meta-analysis results.
Optica, 2023
A microscopy technique using deep learning to create virtual H&E stains of unlabeled tissue, providing real-time histology without chemical staining.
arXiv, 2024
This study explores using fNIRS, a portable alternative to fMRI, for decoding visual imagery, showing promise in classifying imagined visual categories.
ICML, 2024
Introduces the Hourglass Diffusion Transformer (HDiT), a novel architecture for generating high-resolution images directly in pixel space.
Nature Communications, 2024
Identifies key challenges in NeuroAI training and provides recommendations for fostering an interdisciplinary environment for the next generation of scientists.
CVPR, 2025
Introduces NSD-Imagery, the first large-scale fMRI dataset dedicated to mental imagery, designed to advance brain decoding models for imagined visuals.
Under review
Presents a robust multimodal architecture that adapts fMRI-to-image models from visual perception to mental imagery, improving decoding of imagined scenes.