About
Pengyuan Li is a Research Staff Member at the MIT-IBM Watson AI Lab, where his research focuses on multimodal foundation models. He is a lead developer of the IBM Granite Vision Models, a family of vision-language models designed for visual document understanding. He also serves as Co-PI on a joint MIT-IBM research project with Prof. Bill Freeman at MIT. Previously, he led data acquisition efforts for the development of large-scale language, code, and multimodal foundation models at IBM.
His broader research interests span multimodal learning, document intelligence, AI for science, and biomedical informatics. He collaborates with Prof. Paul Sternberg at Caltech and researchers at the Alliance of Genome Resources on AI-assisted biological knowledge discovery and biocuration. He is also an Adjunct Faculty member at the Data Science Institute, University of Delaware.
News
- Sep 2026: Gave a talk at OpenCV channel about Document Intelligence using Granite Vision and Docling
- June 2026: Our work ChartNet been featured by MIT News
- June 2026: Successfully organized the DataMFM workshop at CVPR 2026
- Feb 2026: Glad to present CurateBench: Evaluating LLMs on Evidence-Grounded, Ontology-Aligned Biocuration Tasks at ISB 2026 Virtual Biocuration Conference
- Jan 2026: Published Chart2CSV model and integrated with Docling to enable document intelligence
- Dec 2025: We will organize the DataMFM:Emerging Directions in Data for Multimodal Foundation Models workshop at CVPR 2026
- Aug 2025: Gave a talk on Granite Vision Models at OpenCV Live channel
- Jul 2025: Our Granite Vision model yield more than 100K downloads on HuggingFace
- Apr 2025: Pengyuan co-organized ‘AI and Biodata’ workshop at the 18th Annual International Biocuration Conference
- Apr 2025: Pengyuan presented ‘GeneScribe: Leveraging large language models for gene summary generation’ at the 18th Annual International Biocuration Conference
- Mar 2025: Published Large Vision models on behalf of IBM-Research on HuggingFace
- Feb 2023: Pengyuan joined the UD’s Data Science Institue as an adjunct faculty
- Jul 2021: Pengyuan joined the IBM-Research as a Research Staff Member