Multimodal foundation models are the future of healthcare.

Announcement Partner With Us

How it works

Sophont will release easily customizable open-source medical foundation models. Our multimodal foundation models will unify diverse medical data into powerful latent representations, enabling various novel downstream healthcare applications.

Drag-and-drop inputs into our foundation model to see how Sophont enables new customer solutions.

📋
Patient History
🩸
Lab Tests
🔬
Pathology Slides
🧠
Brain Scans
Customizable Base Model
💊

Pharma - Novel Biomarkers

📋 🩸 🔬 🧠

Can we identify Alzheimer's trial participants more efficiently?

Waiting for more data...

👨‍⚕️

Doctor - EHR Copilot

📋 🩸 🔬 🧠

Can you summarize patient’s clinical status and suggest next steps for monitoring and management?

Waiting for more data...

👤

Patient - EHR Assistant

📋 🩸 🔬 🧠

Can you explain what's happening in simple terms?

Waiting for more data...

Why Open Models Matter

Trust and transparency are critical for healthcare. At Sophont we default to transparency across all aspects of our models, publicly releasing everything and working in-the-open except for circumstances where doing so may risk compromising sensitive health data. We offer compute and talent to foster collaborations with academics, industry, and ML hobbyists, working to deliver high-impact open-source models and publications.

👁️

Full Transparency

We release model weights and detailed training information so researchers, clinicians, and regulators can fully trust our models and run them locally for maximum security.

🏋️

Robust Performance

Models can be locally deployed with consistent, static outputs. Continuous open evaluation of our models in varied environments ensures safety, consistency, and reliability.

🔧

Customizable

Fully open-sourced models allows researchers to easily fine-tune or customize our models to suit their particular use-cases and patient populations.

Downstream Applications

Truly multimodal medical AI opens up entirely new possibilities for the healthcare ecosystem.

🔬

Identify novel biomarkers

Multimodal foundation models have the potential to integrate clinical records, blood tests, and imaging data (e.g., brain scans, histopathology slides) to spot hidden patterns that traditional analyses might miss. Merging input modalities together into the same latent space can lead to the discovery of new predictive signals to suggest entirely new biomarkers for disease diagnosis or progression.

👥

Patient selection for clinical trials

We hope to partner with pharma and life sciences R&D teams to deploy AI models to more accurately forecast disease onset, enabling smarter patient cohort selection for clinical trials. Even slight gains in predictive diagnostic accuracy for risk conditions like Alzheimer's translates into immense cost savings for pharma, reducing trial lengths, minimizing patient dropouts, and accelerating breakthroughs in treatment.

👨‍⚕️

Copilot for doctors

Whereas coding copilots provide an environment for suggestions and summaries alongside programming projects, a doctor-facing EHR copilot could similarly provide benefits in the clinical workflow. We plan to soon offer clinical NLP foundation models that would prove useful to downstream use-cases such as allowing healthcare workers to flexibly summarize patient histories, test results, and imaging findings to efficiently get a holistic view of patient health.

🖥️

Patient-facing EHR interface

Sophont's foundation models could be fine-tuned for use-cases including translation of complex medical information—lab data, imaging reports, pathology scans—into clear, plain-language summaries for patients. A electronic health portal supplemented by Sophont's foundation models could provide easy-to-understand explanations of test results and recommended follow-up steps, empowering patients to better manage their own healthcare journey.

Our Team

Experts in generative AI, neuroscience, and medical imaging, with numerous top-tier publications and global collaborations.

Tanishq Abraham

Tanishq Abraham, Ph.D.

CEO

Leading generative AI researcher specializing in medical applications. Former Research Director at Stability AI and ex-CEO of MedARC. Kaggle grandmaster and former child prodigy who obtained his Bachelor's in Biomedical Engineering at 14 years old and his Ph.D. at 19 years old.

Paul Scotti

Paul Scotti, Ph.D.

CTO

NeuroAI expert who led the team behind MedARC's MindEye work that achieve state-of-the-art reconstructions of seen images from brain activity. Former postdoc at Princeton Neuroscience Institute and former Head of Neuroimaging at Stability AI.

Research Publications

We continually publish open, peer-reviewed research in top AI, neuroscience, and biomedical venues.

2023 NeurIPS (spotlight)

Reconstructing the mind's eye: fmri-to-image with contrastive learning and diffusion priors

P. Scotti, ... T.M. Abraham

Read Paper →
2024 ICML

MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data

P. Scotti, ... T.M. Abraham

Read Paper →
2024 Nature Biomedical Engineering

A vision–language foundation model for the generation of realistic chest X-ray images

C. Bluethgen, ... T.M. Abraham, et al.

Read Paper →
2024 AAAI

A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation

Z. Chen, ... T.M. Abraham, et al.

Read Paper →
2025 Blog post

LLMs in medicine: evaluations, advances, and the future

T.M. Abraham

Read Paper →
2021 Blog post

DALL-E Mini

B. Dayma, ... T.M. Abraham, et al.

Read Paper →
2021 JOSE

EduCortex: browser-based 3D brain visualization of fMRI meta-analysis maps

P. Scotti, et al.

Read Paper →
2023 Optica

Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual hematoxylin and eosin staining

T.M. Abraham, et al.

Read Paper →
2024 arXiv

Progress Towards Decoding Visual Imagery via fNIRS

M. Adamic, ... P. Scotti, et al.

Read Paper →
2024 ICML

Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers

K. Crowson, ... T.M. Abraham, et al.

Read Paper →
2024 Nature Communications

Trainees' perspectives and recommendations for catalyzing the next generation of NeuroAI researchers

A. Luppi, ... P. Scotti, & H. Gellersen

Read Paper →
In press CVPR

NSD-Imagery: A benchmark dataset for extending fMRI vision decoding methods to mental imagery

R. Kneeland, P. Scotti, et al.

Read Paper →
Under review

MIRAGE: Robust multi-modal architectures translate fMRI-to-image models from vision to mental imagery

R. Kneeland, ... P. Scotti, et al.

Partner With Sophont

Collaborate with us at the frontier of open medical AI. Whether you're a healthcare provider, research institution, or investor, join us in shaping the future of healthcare.

Email Us

[email protected]