Bryan A. Plummer

Assistant Professor
Image and Video Computing Group
Department of Computer Science
Boston University
Office: MCS201
Email: bplum@bu.edu

CV, Google Scholar, Github

My research interests fall within the umbrella of artificial intelligence with a focus visual recognition scene understanding, interpretable machine learning, and understanding the relationship between vision and language.

As of July 2020 I am an Assistant Professor in the Department of Computer Science at Boston University. I obtained my PhD in the computer vision group at the University of Illinois at Urbana-Champaign advised by Svetlana Lazebnik. After that I worked as a Postdoctoral Associate with Kate Saenko and Stan Sclaroff and later as a Research Assistant Professor before my current position. During my PhD I was fortunate to be named a 3M Foundation Fellow, NSF GRFP honorable mention, and was able to spend my summers working with Hadi Kiapour (eBay), Shuai Zheng (eBay), Robinson Piramuthu (eBay), Matthew Brown (Google), Himanshu Arora (A9), and Stephen Kelley (MIT Lincoln Labs).

Recent News

  • Two papers accepted by ECCV 2020: our multilingual vision-language model SMALR was accepted for a spotlight and our explanation method for image similarity models SANE was accepted as a poster
  • My student Andrea took third place at the VizWiz Grand Challenge Workshop at CVPR 2020. Here is a video describing her image captioning approach.
  • In July 2020 I will start a position as an Assistant Professor at Boston University
  • MULE was accepted as an oral by AAAI 2020
  • I will serve as an area chair for ECCV 2020

PhD Students

Other Current Students

  • Yize Xie (masters)
  • Julius Frost (undergrad, with Kate Saenko)

Former Students

  • Farheen Rahman (undergrad, 2020)
  • Tammy Qiu (undergrad, 2019, with Kate Saenko), now at Microsoft
  • Paige Kordas (undergrad, 2018, with David Forsyth), now at Facebook

Publications Grouped by Topic

Go here for publications by year

Phrase Grounding




   

Bryan A. Plummer, Kevin J. Shih, Yichen Li, Ke Xu, Svetlana Lazebnik, Stan Sclaroff, Kate Saenko. Revisiting Image-Language Networks for Open-ended Phrase Detection. arXiv:1811.07212, 2018. [paper][code][bibtex]



   

Bryan A. Plummer, Paige Kordas, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu, Svetlana Lazebnik. Conditional Image-Text Embedding Networks. ECCV, 2018. [paper][code][bibtex]



   

Bryan A. Plummer, Arun Mallya, Christopher M. Cervantes, Julia Hockenmaier, Svetlana Lazebnik. Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues. ICCV, 2017. [paper][code][bibtex]



   

Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik. Flickr30K Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. IJCV, 123(1):74-93, 2017. [paper][project][bibtex]

Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik. Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. ICCV, 2015. [paper][project][bibtex]




Explainable AI




   

Bryan A. Plummer, Mariya I. Vasileva, Vitali Petsiuk, Kate Saenko, David Forsyth. Why do These Match? Explaining the Behavior of Image Similarity Models. ECCV, 2020. [paper][bibtex]




Language Representation Learning




   

Andrea Burns, Donghyun Kim, Derry Wijaya, Kate Saenko, Bryan A. Plummer. Learning to Scale Multilingual Representations for Vision-Language Tasks. ECCV, 2020. [paper][bibtex]



   

Donghyun Kim, Kuniaki Saito, Kate Saenko, Stan Sclaroff, Bryan A. Plummer. MULE: Multimodal Universal Language Embedding. AAAI, 2020. [paper][project][code][bibtex]



   

Andrea Burns, Reuben Tan, Kate Saenko, Stan Sclaroff, Bryan A. Plummer. Language Features Matter: Effective Language Representations for Vision-Language Tasks. ICCV, 2019. [paper][project][bibtex]




Vision and Language




   

Reuben Tan, Huijuan Xu, Kate Saenko, Bryan A. Plummer. LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval. arXiv:1909.13784, 2019. [paper][bibtex]



   

Huijuan Xu, Kun He, Bryan A. Plummer, Leonid Sigal, Stan Sclaroff, Kate Saenko. Multilevel Language and Vision Integration for Text-to-Clip Retrieval. AAAI, 2019. [paper][code][bibtex]



   

Tatiana Tommasi, Arun Mallya, Bryan A. Plummer, Svetlana Lazebnik, Alexander C. Berg, Tamara L. Berg. Combining Multiple Cues for Visual Madlibs Question Answering. IJCV, 127(1):38-60, 2019. [paper][bibtex]

Tatiana Tommasi, Arun Mallya, Bryan A. Plummer, Svetlana Lazebnik, Alexander C. Berg, Tamara L. Berg. Solving Visual Madlibs with Multiple Cues. BMVC, 2016. [paper][bibtex]



   

Bryan A. Plummer, Matthew Brown, Svetlana Lazebnik. Enhancing Video Summarization via Vision-Language Embedding. CVPR, 2017. [paper][bibtex]




Efficient Neural Networks




   

Bryan A. Plummer, Nikoli Dryden, Julius Frost, Torsten Hoefler, Kate Saenko. Shapeshifter Networks: Cross-layer Parameter Sharing for Scalable and Effective Deep Learning. arXiv:2006.10598, 2020. [paper][bibtex]



   

Donghyun Kim, Tian Lan, Chuhang Zou, Ning Xu, Bryan A. Plummer, Stan Sclaroff, Jayan Eledath, Gerard Medioni. Multi-Task Learning from Videos via Efficient Inter-Frame Attention. arXiv:2002.07362, 2020. [paper][bibtex]




Image Similarity and Search




   

Reuben Tan, Mariya I. Vasileva, Kate Saenko, Bryan A. Plummer. Learning Similarity Conditions Without Explicit Supervision. ICCV, 2019. [paper][code][bibtex]



   

Bryan A. Plummer, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu. Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback. WACV, 2019. [paper][bibtex]



   

Mariya I. Vasileva, Bryan A. Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, David Forsyth. Learning Type-Aware Embeddings for Fashion Compatibility. ECCV, 2018. [paper][code][bibtex]




Domain Adaptation/Transfer Learning




   

Donghyun Kim, Kuniaki Saito, Tae-Hyun Oh, Bryan A. Plummer, Stan Sclaroff, Kate Saenko. Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels. arXiv:2003.08264, 2020. [paper][bibtex]



   

P. Daphne Tsatsoulis, Bryan A. Plummer, David Forsyth. Visual Analogies: A Framework for Defining Aspect Categorization. ECCV TASK-CV Workshop, 2016. Workshop Best Paper Award. [paper][bibtex]