Bryan A. Plummer

Assistant Professor
Image and Video Computing Group
Department of Computer Science
Boston University
Office: CDS844
Email: bplum@bu.edu

CV, Google Scholar, Github
My research interests fall within the umbrella of artificial intelligence with a focus on visual recognition, scene recognition, interpretable and fair machine learning, and understanding the relationship between vision and language.

As of July 2020 I am an Assistant Professor in the Department of Computer Science at Boston University and a core faculty member of the Artificial Intelligence Research (AIR) initiative at the Rafik B. Hariri Institute for Computing and Computational Science & Engineering. I obtained my PhD in the computer vision group at the University of Illinois at Urbana-Champaign advised by Svetlana Lazebnik. After that I worked as a Postdoctoral Associate with Kate Saenko and Stan Sclaroff and later as a Research Assistant Professor before my current position. During my PhD I was fortunate to be named a 3M Foundation Fellow, NSF GRFP honorable mention, and was able to spend my summers working with Hadi Kiapour (eBay), Shuai Zheng (eBay), Robinson Piramuthu (eBay), Matthew Brown (Google), Himanshu Arora (A9), and Stephen Kelley (MIT Lincoln Labs).

Recent News

  • I will be serving as an Area Chair for NeurIPS'23
  • My student's paper introducing a new sampling strategy for bias mitigation and another student's paper that learns to relate audio and language to video regions without ground truth text annotations was accepted at CVPR 2023
  • I will be serving as an Area Chair for CVPR'23 and an Action Editor for ACL ARR
  • Three papers accepted by ECCV 2022, congratulations to my students!
  • Thanks to the MIT Lincoln Laboratory for supporting our research
  • Thanks to the National Science Foundation (Innovation: Bioinformatics) for supporting our research
  • My paper that introduces a new task where the goal is to generate layer weights for a neural network under a given parameter budget was accepted at ICLR 2022

PhD Students

Other Current Students

  • Murtadha Al-Barani (masters, with Kate Saenko)
  • Keanu Nichols (masters)
  • Manyuan Lu (masters)

Former Students

  • Divya Appapogu (masters)
  • Ashlesha Chaudhari (masters)
  • Soren Nelson (masters)
  • Zora Che (undergrad, 2022), continuing as a PhD student at University of Maryland, College Park
  • John (Yuanming) Chai (undergrad)
  • Julius Frost (BA/MS, 2021, with Kate Saenko), now at MORSE Corp
  • Sanjna Agrawal (undergrad, 2021)
  • Yize Xie (masters, 2020)
  • Farheen Rahman (undergrad, 2020)
  • Tammy Qiu (undergrad, 2019, with Kate Saenko), now a PhD student at Columbia University
  • Paige Kordas (undergrad, 2018, with David Forsyth), now at Facebook

Teaching

Publications Grouped by Topic

Go here for publications by year

Visual Grounding




   

Maan Qraitem, Bryan A. Plummer. Learning to Reason from General Concepts to Fine-grained Tokens for Discriminative Phrase Detection. arXiv:2112.03237, 2021. [paper][bibtex]



   

Reuben Tan, Bryan A. Plummer, Kate Saenko, Hailin Jin, Bryan Russell. Look at What I’m Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos. NeurIPS, 2021. [paper][bibtex]



   

Bryan A. Plummer, Kevin J. Shih, Yichen Li, Ke Xu, Svetlana Lazebnik, Stan Sclaroff, Kate Saenko. Revisiting Image-Language Networks for Open-ended Phrase Detection. TPAMI, 2020. [paper][code][bibtex]



   

Bryan A. Plummer, Paige Kordas, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu, Svetlana Lazebnik. Conditional Image-Text Embedding Networks. ECCV, 2018. [paper][code][bibtex]



   

Bryan A. Plummer, Arun Mallya, Christopher M. Cervantes, Julia Hockenmaier, Svetlana Lazebnik. Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues. ICCV, 2017. [paper][code][bibtex]



   

Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik. Flickr30K Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. IJCV, 123(1):74-93, 2017. [paper][project][bibtex]

Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik. Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. ICCV, 2015. [paper][project][bibtex]




News Understanding




   

Reuben Tan, Bryan A. Plummer, Kate Saenko, J. P. Lewis, Avneesh Sud, Thomas Leung. NewsStories: Illustrating articles with visual summaries. ECCV, 2022. [paper][project][bibtex]



   

Zhongping Zhang, Yiwen Gu, Bryan A. Plummer. Show and Write: Entity-aware News Generation with Image Information. arXiv:2112.05917, 2021. [paper][bibtex]



   

Reuben Tan, Bryan A. Plummer, Kate Saenko. Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News. EMNLP, 2020. [paper][project][bibtex]




Vision and Language




   

Reuben Tan, Arijit Ray, Andrea Burns, Bryan A. Plummer, Justin Salamon, Oriol Nieto, Bryan Russell, Kate Saenko. Language-Guided Audio-Visual Source Separation via Trimodal Consistency. CVPR, 2023. [paper][project][bibtex]



   

Andrea Burns, Deniz Arsan, Sanjna Agrawal, Ranjitha Kumar, Kate Saenko, Bryan A. Plummer. A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility. ECCV, 2022. [paper][code&data][bibtex]



   

Reuben Tan, Huijuan Xu, Kate Saenko, Bryan A. Plummer. LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval. WACV, 2021. [paper][bibtex]



   

Huijuan Xu, Kun He, Bryan A. Plummer, Leonid Sigal, Stan Sclaroff, Kate Saenko. Multilevel Language and Vision Integration for Text-to-Clip Retrieval. AAAI, 2019. [paper][code][bibtex]



   

Tatiana Tommasi, Arun Mallya, Bryan A. Plummer, Svetlana Lazebnik, Alexander C. Berg, Tamara L. Berg. Combining Multiple Cues for Visual Madlibs Question Answering. IJCV, 127(1):38-60, 2019. [paper][bibtex]

Tatiana Tommasi, Arun Mallya, Bryan A. Plummer, Svetlana Lazebnik, Alexander C. Berg, Tamara L. Berg. Solving Visual Madlibs with Multiple Cues. BMVC, 2016. [paper][bibtex]



   

Bryan A. Plummer, Matthew Brown, Svetlana Lazebnik. Enhancing Video Summarization via Vision-Language Embedding. CVPR, 2017. [paper][bibtex]




Controllable Manipulation




   

Nannan Li, Bryan A. Plummer. Supervised Attribute Information Removal and Reconstruction for Image Manipulation. ECCV, 2022. [paper][code][bibtex]




Efficient Neural Networks




   

Bryan A. Plummer*, Nikoli Dryden*, Julius Frost, Torsten Hoefler, Kate Saenko. Neural Parameter Allocation Search. ICLR, 2022. [paper][code][video][bibtex]



   

Donghyun Kim, Tian Lan, Chuhang Zou, Ning Xu, Bryan A. Plummer, Stan Sclaroff, Jayan Eledath, Gerard Medioni. Multi-Task Learning from Videos via Efficient Inter-Frame Attention. ICCV Workshops, 2021. [paper][bibtex]




Fair and Explainable AI




   

Maan Qraitem, Kate Saenko, Bryan A. Plummer. Bias Mimicking: A Simple Sampling Approach for Bias Mitigation. CVPR, 2023. [paper][code][bibtex]



   

Olivia Watkins, Sandy Huang, Julius Frost, Kush Bhatia, Eric Weiner, Pieter Abbeel, Trevor Darrell, Bryan A. Plummer, Kate Saenko, Anca Dragan. Explaining robot policies. Applied AI Letters, 2021. [paper][bibtex]



   

Bryan A. Plummer*, Mariya I. Vasileva*, Vitali Petsiuk, Kate Saenko, David Forsyth. Why do These Match? Explaining the Behavior of Image Similarity Models. ECCV, 2020. [paper][code][supplementary][bibtex]




Image Similarity and Search




   

Siqi Wang, Manyuan Lu, Nikita Moshkov, Juan C. Caicedo, Bryan A. Plummer. Anchoring to Exemplars for Training Mixture-of-Expert Cell Embeddings. arXiv:2112.03208, 2021. [paper][bibtex]



   

Samarth Mishra*, Zhongping Zhang*, Yuan Shen, Ranjitha Kumar, Venkatesh Saligrama, Bryan A. Plummer. Effectively Leveraging Attributes for Visual Similarity. ICCV, 2021. [paper][code][bibtex]



   

Donghyun Kim, Kuniaki Saito, Samarth Mishra, Stan Sclaroff, Kate Saenko, Bryan A. Plummer. Self-Supervised Visual Attribute Learning for Fashion Compatibility. ICCV Workshops, 2021. [paper][bibtex]



   

Reuben Tan, Mariya I. Vasileva, Kate Saenko, Bryan A. Plummer. Learning Similarity Conditions Without Explicit Supervision. ICCV, 2019. [paper][code][bibtex]



   

Bryan A. Plummer, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu. Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback. WACV, 2019. [paper][bibtex]



   

Mariya I. Vasileva, Bryan A. Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, David Forsyth. Learning Type-Aware Embeddings for Fashion Compatibility. ECCV, 2018. [paper][code][bibtex]




Domain Adaptation/Transfer Learning




   

Donghyun Kim, Kuniaki Saito, Tae-Hyun Oh, Bryan A. Plummer, Stan Sclaroff, Kate Saenko. CDS: Cross-Domain Self-supervised Pre-training. ICCV, 2021. [paper][bibtex]



   

P. Daphne Tsatsoulis, Bryan A. Plummer, David Forsyth. Visual Analogies: A Framework for Defining Aspect Categorization. ECCV TASK-CV Workshop, 2016. Workshop Best Paper Award. [paper][bibtex]




Language Representation Learning




   

Andrea Burns, Donghyun Kim, Derry Wijaya, Kate Saenko, Bryan A. Plummer. Learning to Scale Multilingual Representations for Vision-Language Tasks. ECCV, 2020. [paper][project][supplementary][bibtex]



   

Donghyun Kim, Kuniaki Saito, Kate Saenko, Stan Sclaroff, Bryan A. Plummer. MULE: Multimodal Universal Language Embedding. AAAI, 2020. [paper][project][code][bibtex]



   

Andrea Burns, Reuben Tan, Kate Saenko, Stan Sclaroff, Bryan A. Plummer. Language Features Matter: Effective Language Representations for Vision-Language Tasks. ICCV, 2019. [paper][project][bibtex]