Multimodal foundation models for healthcare

Sophont builds open, universal medical AI that understands pathology, neuro­imaging, clinical text and more—empowering clinicians and researchers worldwide.

Announcing our 9M seed round →

MedARC, our open science Discord server →

Team

Partner With Us

Collaborate with us at the frontier of open medical AI. Whether you're a healthcare provider, research institution, or investor, let's connect.

Research Publications

fMRI brain visualization

Reconstructing the mind's eye: fmri-to-image...

P. Scotti, ... T.M. Abraham

NeurIPS (spotlight), 2023

Introduced a novel method to reconstruct seen images from fMRI brain signals by aligning brain activity with image embeddings and a diffusion model.

Abstract AI visualization

MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data

P. Scotti, ... T.M. Abraham

ICML, 2024

Demonstrates that a shared fMRI-to-image model can be fine-tuned on just one hour of data from a new person, enabling high-quality brain decoding with minimal data.

Chest X-ray on a monitor

A vision–language foundation model for the generation of realistic chest X-ray images

C. Bluethgen, ... T.M. Abraham, et al.

Nature Biomedical Engineering, 2024

Presents RoentGen, a model that can generate realistic, high-resolution X-ray images from clinical text prompts, aiding in data augmentation and education.

Doctor interpreting medical scans

A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation

Z. Chen, ... T.M. Abraham, et al.

AAAI, 2024

This model generates preliminary reports from chest X-rays, significantly improving radiologists' diagnostic efficiency without compromising accuracy.

Code on a dark screen

LLMs in medicine: evaluations, advances, and the future

T.M. Abraham

Blog post, 2025

Provides a comprehensive overview of how LLMs are evaluated for medical applications, highlighting the critical need for more robust, multimodal evaluation frameworks.

Abstract colorful generative art

DALL-E Mini

B. Dayma, ... T.M. Abraham, et al.

Blog post, 2021

Details the community-led creation of DALL·E mini, an open-source text-to-image model, explaining its architecture, training, and impact.

3D visualization of a human brain

EduCortex: browser-based 3D brain visualization of fMRI meta-analysis maps

P. Scotti, et al.

JOSE, 2021

A lightweight, open-source JavaScript library for visualizing 3D brain maps directly in a web browser to share and explore fMRI meta-analysis results.

Digital pathology slide

Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging...

T.M. Abraham, et al.

Optica, 2023

A microscopy technique using deep learning to create virtual H&E stains of unlabeled tissue, providing real-time histology without chemical staining.

Retina scan equipment

Progress Towards Decoding Visual Imagery via fNIRS

M. Adamic, ... P. Scotti, et al.

arXiv, 2024

This study explores using fNIRS, a portable alternative to fMRI, for decoding visual imagery, showing promise in classifying imagined visual categories.

Abstract data visualization with flowing lines

Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers

K. Crowson, ... T.M. Abraham, et al.

ICML, 2024

Introduces the Hourglass Diffusion Transformer (HDiT), a novel architecture for generating high-resolution images directly in pixel space.

Abstract representation of connected nodes

Trainees' perspectives and recommendations for catalyzing the next generation of NeuroAI researchers

A. Luppi, ... P. Scotti, & H. Gellersen

Nature Communications, 2024

Identifies key challenges in NeuroAI training and provides recommendations for fostering an interdisciplinary environment for the next generation of scientists.

Microscope view of cells

NSD-Imagery: A benchmark dataset for extending fMRI vision decoding methods to mental imagery

R. Kneeland, P. Scotti, et al.

CVPR, 2025

Introduces NSD-Imagery, the first large-scale fMRI dataset dedicated to mental imagery, designed to advance brain decoding models for imagined visuals.

Abstract neural network graphic

MIRAGE: Robust multi-modal architectures translate fMRI-to-image models from vision to mental imagery

R. Kneeland, ... P. Scotti, et al.

Under review

Presents a robust multimodal architecture that adapts fMRI-to-image models from visual perception to mental imagery, improving decoding of imagined scenes.

News & Press

Brain scan visualization

NY Times Discusses Our MindEye Research

The New York Times

Interactive Article, 2025

Our MindEye2 paper is discussed in a NY Times article exploring AI's remarkable capabilities across different body parts, specifically in the Brain section.

Medical AI manifesto

Medical AI Manifesto: Introducing Sophont

Tanishq Mathew Abraham

Blog Post, 2025

Tanishq shares his Medical AI Manifesto, introducing Sophont and our mission to build open multimodal foundation models for healthcare.

Mental imagery reconstruction

[Mind]ography: the world’s first printed memories

Fujifilm Australia

Media Feature, 2025

Media collaboration with Fujifilm Australia showcasing our breakthrough technology for reconstructing mental imagery from brain signals.

Podcast microphone

The Mind-Reading Revolution with Tanishq

Cognitive Revolution Podcast

Podcast Episode, 2025

Cognitive Revolution podcast where Tanishq discusses the future of brain-computer interfaces and medical AI.

Shared latent space visualization

Reading Minds from Shared Latent Space with Paul

Cognitive Revolution Podcast

Podcast Episode, 2025

Cognitive Revolution podcast where Paul dives into how we can decode thoughts from shared latent spaces, and the technical aspects of brain decoding.

MIT talk

MIT Talk: Open Foundation Models for the Future of Medicine

Tanishq Mathew Abraham

MIT Presentation, 2024

Tanishq presents at MIT about open medical foundation models and their transformative potential for the future of healthcare.

US Senate hearing

US Senate Hearing References Our Medical AI Work

U.S. Senate

Congressional Hearing, 2023

US Senate hearing briefly discusses our work as an example of medical AI technology applications in healthcare.

Jeremy Howard podcast

Deep Dive with Jeremy Howard on Life Journey

Jeremy Howard Podcast

Podcast Episode

Jeremy Howard and Tanishq go in-depth into Tanishq's upbringing, life journey, and path to becoming a leading AI researcher.

Conan appearance

Early TV Appearance on Conan

Conan O'Brien

TV Appearance

Young Tanishq appears on Conan O'Brien's show as a child prodigy and shares some science jokes.

Blog Posts

Paul Scotti

How to structure open science collaborations online

Paul Scotti

2025

Paul Scotti

Insights from the Algonauts 2025 Challenge

Paul Scotti

2025

Tanishq Abraham

LLMs in medicine: evaluations, advances, and the future

Tanishq Mathew Abraham

2025

Tanishq Abraham

Debunking DeepSeek Delusions

Tanishq Mathew Abraham

2025

MedARC

Evaluating the Medical Knowledge of Open LLMs

Tanishq Mathew Abraham

2024

Tanishq Abraham

Reinforcement Learning for Diffusion Models from Scratch

Tanishq Mathew Abraham

2023

MedARC

Announcing the launch of the Medical AI Research Center (MedARC)

Tanishq Mathew Abraham

2023