Bryan A. Plummer

Assistant Professor
Image and Video Computing Group
Department of Computer Science
Boston University
Office: MCS206
Email: bplum@bu.edu

CV, Google Scholar, Github

My research interests fall within the umbrella of artificial intelligence with a focus visual recognition scene understanding, interpretable machine learning, and understanding the relationship between vision and language.

As of July 2020 I am an Assistant Professor in the Department of Computer Science at Boston University and a core faculty member of the Artificial Intelligence Research (AIR) initiative at the Rafik B. Hariri Institute for Computing and Computational Science & Engineering. I obtained my PhD in the computer vision group at the University of Illinois at Urbana-Champaign advised by Svetlana Lazebnik. After that I worked as a Postdoctoral Associate with Kate Saenko and Stan Sclaroff and later as a Research Assistant Professor before my current position. During my PhD I was fortunate to be named a 3M Foundation Fellow, NSF GRFP honorable mention, and was able to spend my summers working with Hadi Kiapour (eBay), Shuai Zheng (eBay), Robinson Piramuthu (eBay), Matthew Brown (Google), Himanshu Arora (A9), and Stephen Kelley (MIT Lincoln Labs).

PhD Students

Andrea Burns (with Kate Saneko)

Winner of the 2021 Google PhD Fellowship in Machine Perception

Reuben Tan (with Kate Saenko)
Zhongping Zhang
Diala Lteif (with Sarah Bargal)
Piotr Teterwak (with Kate Saenko)
Siqi Wang
Maan Qraitem (with Kate Saenko)
Nannan Li
Arijit Ray (with Kate Saenko)

Other Current Students

Murtadha Al-Barani (masters, with Kate Saenko)
Soren Nelson (masters)
Zora Che (undergrad)
John (Yuanming) Chai (undergrad)

Former Students

Julius Frost (BA/MS, 2021, with Kate Saenko), now at MORSE Corp
Sanjna Agrawal (undergrad, 2021)
Yize Xie (masters, 2020)
Farheen Rahman (undergrad, 2020)
Tammy Qiu (undergrad, 2019, with Kate Saenko), now at MIT Media Lab
Paige Kordas (undergrad, 2018, with David Forsyth), now at Facebook

Publications Grouped by Topic

Go here for publications by year

Phrase Grounding/Detection


		Bryan A. Plummer, Kevin J. Shih, Yichen Li, Ke Xu, Svetlana Lazebnik, Stan Sclaroff, Kate Saenko. Revisiting Image-Language Networks for Open-ended Phrase Detection. TPAMI, 2020. [paper][code][bibtex] @article{plummerPhraseDetection, author={Bryan A. Plummer and Kevin J. Shih and Yichen Li and Ke Xu and Svetlana Lazebnik and Stan Sclaroff and Kate Saenko}, title={Revisiting Image-Language Networks for Open-ended Phrase Detection}, journal={Transactions on Pattern Analysis and Machine Intelligence}, year={2020} }
		Bryan A. Plummer, Paige Kordas, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu, Svetlana Lazebnik. Conditional Image-Text Embedding Networks. ECCV, 2018. [paper][code][bibtex] @InProceedings{plummerCITE2018, Author = {Bryan A. Plummer and Paige Kordas and M. Hadi Kiapour and Shuai Zheng and Robinson Piramuthu and Svetlana Lazebnik}, Title = {Conditional Image-Text Embedding Networks}, booktitle = {The European Conference on Computer Vision (ECCV)}, Year = {2018} }
		Bryan A. Plummer, Arun Mallya, Christopher M. Cervantes, Julia Hockenmaier, Svetlana Lazebnik. Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues. ICCV, 2017. [paper][code][bibtex] @inproceedings{plummerPLCLC2017, Author = {Bryan A. Plummer and Arun Mallya and Christopher M. Cervantes and Julia Hockenmaier and Svetlana Lazebnik}, Title = {Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues}, booktitle = {The IEEE International Conference on Computer Vision (ICCV)}, Year = {2017} }
		Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik. Flickr30K Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. IJCV, 123(1):74-93, 2017. [paper][project][bibtex] @article{flickrentitiesijcv, author={Bryan A. Plummer and Liwei Wang and Christopher M. Cervantes and Juan C. Caicedo and Julia Hockenmaier and Svetlana Lazebnik}, title={Flickr30K Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models}, journal={International Journal of Computer Vision}, volume={123}, number={1}, pages={74-93}, year={2017} } Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik. Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. ICCV, 2015. [paper][project][bibtex] @inproceedings{flickrentities, author={Bryan A. Plummer and Liwei Wang and Christopher M. Cervantes and Juan C. Caicedo and Julia Hockenmaier and Svetlana Lazebnik}, title={Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models}, booktitle={The IEEE International Conference on Computer Vision (ICCV)}, year={2015} }

Defending Against Machine Manipulated Media

Reuben Tan, Bryan A. Plummer, Kate Saenko. Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News. EMNLP, 2020. [paper][project][bibtex]

Explainable AI

Olivia Watkins, Sandy Huang, Julius Frost, Kush Bhatia, Eric Weiner, Pieter Abbeel, Trevor Darrell, Bryan A. Plummer, Kate Saenko, Anca Dragan. Explaining robot policies. Applied AI Letters, 2021. [paper][bibtex]

Bryan A. Plummer*, Mariya I. Vasileva*, Vitali Petsiuk, Kate Saenko, David Forsyth. Why do These Match? Explaining the Behavior of Image Similarity Models. ECCV, 2020. [paper][code][supplementary][bibtex]

Language Representation Learning


		Andrea Burns, Donghyun Kim, Derry Wijaya, Kate Saenko, Bryan A. Plummer. Learning to Scale Multilingual Representations for Vision-Language Tasks. ECCV, 2020. [paper][project][supplementary][bibtex] @InProceedings{burnsSMALR2020, author={Andrea Burns and Donghyun Kim and Derry Wijaya and Kate Saenko and Bryan A. Plummer}, title={Learning to Scale Multilingual Representations for Vision-Language Tasks}, booktitle={The European Conference on Computer Vision (ECCV)}, year={2020} }
		Donghyun Kim, Kuniaki Saito, Kate Saenko, Stan Sclaroff, Bryan A. Plummer. MULE: Multimodal Universal Language Embedding. AAAI, 2020. [paper][project][code][bibtex] @inproceedings{kimMULEAAAI2020, author={Donghyun Kim and Kuniaki Saito and Kate Saenko and Stan Sclaroff and Bryan A. Plummer}, title={{MULE: Multimodal Universal Language Embedding}}, booktitle={AAAI Conference on Artificial Intelligence}, year={2020} }
		Andrea Burns, Reuben Tan, Kate Saenko, Stan Sclaroff, Bryan A. Plummer. Language Features Matter: Effective Language Representations for Vision-Language Tasks. ICCV, 2019. [paper][project][bibtex] @InProceedings{burnsLanguage2019, author={Andrea Burns and Reuben Tan and Kate Saenko and Stan Sclaroff and Bryan A. Plummer}, title={Language Features Matter: Effective Language Representations for Vision-Language Tasks}, booktitle={The IEEE International Conference on Computer Vision (ICCV)}, year={2019} }

Vision and Language


		Andrea Burns, Deniz Arsan, Sanjna Agrawal, Ranjitha Kumar, Kate Saenko, Bryan A. Plummer. Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments. arXiv:2104.08560, 2021. [paper][code&data][bibtex] @article{burnsMOTIF2021, author={Andrea Burns and Deniz Arsan and Sanjna Agrawal and Ranjitha Kumar and Kate Saenko and Bryan A. Plummer}, title={Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments}, journal={arXiv:2104.08560}, year={2021} }
		Reuben Tan, Bryan A. Plummer, Kate Saenko, Hailin Jin, Bryan Russell. Look at What I’m Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos. NeurIPS, 2021. [paper][bibtex] @inproceedings{tanCOMMA2021, author = {Reuben Tan and Bryan A. Plummer and Kate Saenko and Hailin Jin and Bryan Russell}, title = {Look at What I’m Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos}, booktitle = {Advances in Neural Information Processing Systems (NeurIPS)}, year = {2021} }
		Reuben Tan, Huijuan Xu, Kate Saenko, Bryan A. Plummer. LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval. WACV, 2021. [paper][bibtex] @InProceedings{tanLoGAN2021, author={Reuben Tan and Huijuan Xu and Kate Saenko and Bryan A. Plummer}, title={LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval}, booktitle={IEEE Winter Conference on Applications of Computer Vision (WACV)}, year={2021} }
		Huijuan Xu, Kun He, Bryan A. Plummer, Leonid Sigal, Stan Sclaroff, Kate Saenko. Multilevel Language and Vision Integration for Text-to-Clip Retrieval. AAAI, 2019. [paper][code][bibtex] @InProceedings{xuAAAI2019, author={Huijuan Xu and Kun He and Bryan A. Plummer and Leonid Sigal and Stan Sclaroff and Kate Saenko}, title={Multilevel Language and Vision Integration for Text-to-Clip Retrieval}, booktitle={AAAI Conference on Artificial Intelligence}, year={2019} }
		Tatiana Tommasi, Arun Mallya, Bryan A. Plummer, Svetlana Lazebnik, Alexander C. Berg, Tamara L. Berg. Combining Multiple Cues for Visual Madlibs Question Answering. IJCV, 127(1):38-60, 2019. [paper][bibtex] @article{tommasiIJCV2019, author={Tatiana Tommasi and Arun Mallya and Bryan A. Plummer and Svetlana Lazebnik and Alexander C. Berg and Tamara L. Berg}, title={Combining Multiple Cues for Visual Madlibs Question Answering}, journal={International Journal of Computer Vision}, volume={127}, number={1}, pages={38-60}, year={2019} } Tatiana Tommasi, Arun Mallya, Bryan A. Plummer, Svetlana Lazebnik, Alexander C. Berg, Tamara L. Berg. Solving Visual Madlibs with Multiple Cues. BMVC, 2016. [paper][bibtex] @inproceedings{tommasiBMVC2016, author = {Tatiana Tommasi and Arun Mallya and Bryan A. Plummer and Svetlana Lazebnik and Alexander C. Berg and Tamara L. Berg}, title = {Solving Visual Madlibs with Multiple Cues}, booktitle={British Machine Vision Conference (BMVC)}, year = {2016} }
		Bryan A. Plummer, Matthew Brown, Svetlana Lazebnik. Enhancing Video Summarization via Vision-Language Embedding. CVPR, 2017. [paper][bibtex] @InProceedings{plummerSummarization2017, author={Bryan A. Plummer and Matthew Brown and Svetlana Lazebnik}, title={Enhancing Video Summarization via Vision-Language Embedding}, booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, year={2017} }

Efficient Neural Networks

Bryan A. Plummer, Nikoli Dryden, Julius Frost, Torsten Hoefler, Kate Saenko. Neural Parameter Allocation Search. arXiv:2006.10598, 2020. [paper][bibtex]

Donghyun Kim, Tian Lan, Chuhang Zou, Ning Xu, Bryan A. Plummer, Stan Sclaroff, Jayan Eledath, Gerard Medioni. Multi-Task Learning from Videos via Efficient Inter-Frame Attention. arXiv:2002.07362, 2020. [paper][bibtex]

Image Similarity and Search


		Samarth Mishra, Zhongping Zhang, Yuan Shen, Ranjitha Kumar, Venkatesh Saligrama, Bryan A. Plummer. Effectively Leveraging Attributes for Visual Similarity. ICCV, 2021. [paper][code][bibtex] @InProceedings{mishraPAN2021, author={Samarth Mishra and Zhongping Zhang and Yuan Shen and Ranjitha Kumar and Venkatesh Saligrama and Bryan A. Plummer}, title={Effectively Leveraging Attributes for Visual Similarity}, booktitle={The IEEE International Conference on Computer Vision (ICCV)}, year={2021} }
		Reuben Tan, Mariya I. Vasileva, Kate Saenko, Bryan A. Plummer. Learning Similarity Conditions Without Explicit Supervision. ICCV, 2019. [paper][code][bibtex] @InProceedings{tanSimilarity2019, author={Reuben Tan and Mariya I. Vasileva and Kate Saenko and Bryan A. Plummer}, title={Learning Similarity Conditions Without Explicit Supervision}, booktitle={The IEEE International Conference on Computer Vision (ICCV)}, year={2019} }
		Bryan A. Plummer, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu. Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback. WACV, 2019. [paper][bibtex] @InProceedings{plummerHint2019, author={Bryan A. Plummer and M. Hadi Kiapour and Shuai Zheng and Robinson Piramuthu}, title={{Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback}}, booktitle={IEEE Winter Conference on Applications of Computer Vision (WACV)}, year={2019} }
		Mariya I. Vasileva, Bryan A. Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, David Forsyth. Learning Type-Aware Embeddings for Fashion Compatibility. ECCV, 2018. [paper][code][bibtex] @inproceedings{VasilevaECCV18FasionCompatibility, Author = {Mariya I. Vasileva and Bryan A. Plummer and Krishna Dusad and Shreya Rajpal and Ranjitha Kumar and David Forsyth}, Title = {Learning Type-Aware Embeddings for Fashion Compatibility}, booktitle = {The European Conference on Computer Vision (ECCV)}, Year = {2018} }

Domain Adaptation/Transfer Learning

Donghyun Kim, Kuniaki Saito, Tae-Hyun Oh, Bryan A. Plummer, Stan Sclaroff, Kate Saenko. CDS: Cross-Domain Self-supervised Pre-training. ICCV, 2021. [paper][bibtex]

P. Daphne Tsatsoulis, Bryan A. Plummer, David Forsyth. Visual Analogies: A Framework for Defining Aspect Categorization. ECCV TASK-CV Workshop, 2016. Workshop Best Paper Award. [paper][bibtex]

Bryan A. Plummer

Recent News

PhD Students

Other Current Students

Former Students

Publications Grouped by Topic