My research interests fall within the umbrella of artificial intelligence with a focus visual recognition scene understanding, interpretable machine learning, and understanding the relationship between vision and language.
|
As of July 2020 I am an Assistant Professor in the Department of Computer Science at Boston University and a core faculty member of the Artificial Intelligence Research (AIR) initiative at the Rafik B. Hariri Institute for Computing and Computational Science & Engineering. I obtained my PhD in the computer vision group at the University of Illinois at Urbana-Champaign advised by Svetlana Lazebnik. After that I worked as a Postdoctoral Associate with Kate Saenko and Stan Sclaroff and later as a Research Assistant Professor before my current position. During my PhD I was fortunate to be named a 3M Foundation Fellow, NSF GRFP honorable mention, and was able to spend my summers working with Hadi Kiapour (eBay), Shuai Zheng (eBay), Robinson Piramuthu (eBay), Matthew Brown (Google), Himanshu Arora (A9), and Stephen Kelley (MIT Lincoln Labs).
|
Recent News
- My student's paper introducing a new sampling strategy for bias mitigation and another student's paper that learns to relate audio and language to video regions without ground truth text annotations was accepted at CVPR 2023
- I will be serving as an Area Chair for CVPR'23 and an Action Editor for ACL ARR
- Three papers accepted by ECCV 2022, congratulations to my students!
- Thanks to the MIT Lincoln Laboratory for supporting our research
- Thanks to the National Science Foundation (Innovation: Bioinformatics) for supporting our research
- My paper that introduces a new task where the goal is to generate layer weights for a neural network under a given parameter budget was accepted at ICLR 2022
|
PhD Students
Other Current Students
- Murtadha Al-Barani (masters, with Kate Saenko)
- Keanu Nichols (masters)
- Manyuan Lu (masters)
Former Students
- Divya Appapogu (masters)
- Ashlesha Chaudhari (masters)
- Soren Nelson (masters)
- Zora Che (undergrad, 2022), continuing as a PhD student at University of Maryland, College Park
- John (Yuanming) Chai (undergrad)
- Julius Frost (BA/MS, 2021, with Kate Saenko), now at MORSE Corp
- Sanjna Agrawal (undergrad, 2021)
- Yize Xie (masters, 2020)
- Farheen Rahman (undergrad, 2020)
- Tammy Qiu (undergrad, 2019, with Kate Saenko), now a PhD student at Columbia University
- Paige Kordas (undergrad, 2018, with David Forsyth), now at Facebook
|
Teaching
|
Publications Grouped by Topic
Go here for publications by year
|
| | |
| | Maan Qraitem, Bryan A. Plummer. Learning to Reason from General Concepts to Fine-grained Tokens for Discriminative Phrase Detection. arXiv:2112.03237, 2021. [paper][bibtex] @article{qraitemPFPNet2021,
author={Maan Qraitem and Bryan A. Plummer},
title={Learning to Reason from General Concepts to Fine-grained Tokens for Discriminative Phrase Detection},
journal={arXiv:2112.03237},
year={2021}
}
|
| | Reuben Tan, Bryan A. Plummer, Kate Saenko, Hailin Jin, Bryan Russell. Look at What I’m Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos. NeurIPS, 2021. [paper][bibtex] @inproceedings{tanCOMMA2021,
author = {Reuben Tan and Bryan A. Plummer and Kate Saenko and Hailin Jin and Bryan Russell},
title = {Look at What I’m Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2021}
}
|
| | Bryan A. Plummer, Kevin J. Shih, Yichen Li, Ke Xu, Svetlana Lazebnik, Stan Sclaroff, Kate Saenko. Revisiting Image-Language Networks for Open-ended Phrase Detection. TPAMI, 2020. [paper][code][bibtex] @article{plummerPhraseDetection,
author={Bryan A. Plummer and Kevin J. Shih and Yichen Li and Ke Xu and Svetlana Lazebnik and Stan Sclaroff and Kate Saenko},
title={Revisiting Image-Language Networks for Open-ended Phrase Detection},
journal={Transactions on Pattern Analysis and Machine Intelligence},
year={2020}
}
|
| | Bryan A. Plummer, Paige Kordas, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu, Svetlana Lazebnik. Conditional Image-Text Embedding Networks. ECCV, 2018. [paper][code][bibtex] @InProceedings{plummerCITE2018,
Author = {Bryan A. Plummer and Paige Kordas and M. Hadi Kiapour and Shuai Zheng and Robinson Piramuthu and Svetlana Lazebnik},
Title = {Conditional Image-Text Embedding Networks},
booktitle = {The European Conference on Computer Vision (ECCV)},
Year = {2018}
}
|
| | Bryan A. Plummer, Arun Mallya, Christopher M. Cervantes, Julia Hockenmaier, Svetlana Lazebnik. Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues. ICCV, 2017. [paper][code][bibtex] @inproceedings{plummerPLCLC2017,
Author = {Bryan A. Plummer and Arun Mallya and Christopher M. Cervantes and Julia Hockenmaier and Svetlana Lazebnik},
Title = {Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
Year = {2017}
}
|
| | Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik. Flickr30K Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. IJCV, 123(1):74-93, 2017. [paper][project][bibtex] @article{flickrentitiesijcv,
author={Bryan A. Plummer and Liwei Wang and Christopher M. Cervantes and Juan C. Caicedo and Julia Hockenmaier and Svetlana Lazebnik},
title={Flickr30K Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models},
journal={International Journal of Computer Vision},
volume={123},
number={1},
pages={74-93},
year={2017}
}
Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik. Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models. ICCV, 2015. [paper][project][bibtex] @inproceedings{flickrentities,
author={Bryan A. Plummer and Liwei Wang and Christopher M. Cervantes and Juan C. Caicedo and Julia Hockenmaier and Svetlana Lazebnik},
title={Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models},
booktitle={The IEEE International Conference on Computer Vision (ICCV)},
year={2015}
}
|
| | |
| | Reuben Tan, Bryan A. Plummer, Kate Saenko, J. P. Lewis, Avneesh Sud, Thomas Leung. NewsStories: Illustrating articles with visual summaries. ECCV, 2022. [paper][project][bibtex] @InProceedings{TanNews2022,
author={Reuben Tan and Bryan A. Plummer and Kate Saenko and J. P. Lewis and Avneesh Sud and Thomas Leung},
title={NewsStories: Illustrating articles with visual summaries},
booktitle={The European Conference on Computer Vision (ECCV)},
year={2022}}
|
| | Zhongping Zhang, Yiwen Gu, Bryan A. Plummer. Show and Write: Entity-aware News Generation with Image Information. arXiv:2112.05917, 2021. [paper][bibtex] @article{zhangENGIN2021,
author={Zhongping Zhang and Yiwen Gu and Bryan A. Plummer},
title={Show and Write: Entity-aware News Generation with Image Information},
journal={arXiv:2112.05917},
year={2021}
}
|
| | Reuben Tan, Bryan A. Plummer, Kate Saenko. Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News. EMNLP, 2020. [paper][project][bibtex] @InProceedings{tanDIDAN2020,
author={Reuben Tan and Bryan A. Plummer and Kate Saenko},
title={Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News},
booktitle={Empirical Methods in Natural Language Processing (EMNLP)},
year={2020}
}
|
| | |
| | Reuben Tan, Arijit Ray, Andrea Burns, Bryan A. Plummer, Justin Salamon, Oriol Nieto, Bryan Russell, Kate Saenko. Language-Guided Audio-Visual Source Separation via Trimodal Consistency. CVPR, 2023. [][bibtex] @InProceedings{TanAVSet2023,
author={Reuben Tan and Arijit Ray and Andrea Burns and Bryan A. Plummer and Justin Salamon and Oriol Nieto and Bryan Russell and Kate Saenko},
title={Language-Guided Audio-Visual Source Separation via Trimodal Consistency},
booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2023}}
|
| | Andrea Burns, Deniz Arsan, Sanjna Agrawal, Ranjitha Kumar, Kate Saenko, Bryan A. Plummer. A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility. ECCV, 2022. [paper][code&data][bibtex] @InProceedings{BurnsMoTIF2022,
author={Andrea Burns and Deniz Arsan and Sanjna Agrawal and Ranjitha Kumar and Kate Saenko and Bryan A. Plummer},
title={A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility},
booktitle={The European Conference on Computer Vision (ECCV)},
year={2022}}
|
| | Reuben Tan, Huijuan Xu, Kate Saenko, Bryan A. Plummer. LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval. WACV, 2021. [paper][bibtex] @InProceedings{tanLoGAN2021,
author={Reuben Tan and Huijuan Xu and Kate Saenko and Bryan A. Plummer},
title={LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval},
booktitle={IEEE Winter Conference on Applications of Computer Vision (WACV)},
year={2021}
}
|
| | Huijuan Xu, Kun He, Bryan A. Plummer, Leonid Sigal, Stan Sclaroff, Kate Saenko. Multilevel Language and Vision Integration for Text-to-Clip Retrieval. AAAI, 2019. [paper][code][bibtex] @InProceedings{xuAAAI2019,
author={Huijuan Xu and Kun He and Bryan A. Plummer and Leonid Sigal and Stan Sclaroff and Kate Saenko},
title={Multilevel Language and Vision Integration for Text-to-Clip Retrieval},
booktitle={AAAI Conference on Artificial Intelligence},
year={2019}
}
|
| | Tatiana Tommasi, Arun Mallya, Bryan A. Plummer, Svetlana Lazebnik, Alexander C. Berg, Tamara L. Berg. Combining Multiple Cues for Visual Madlibs Question Answering. IJCV, 127(1):38-60, 2019. [paper][bibtex] @article{tommasiIJCV2019,
author={Tatiana Tommasi and Arun Mallya and Bryan A. Plummer and Svetlana Lazebnik and Alexander C. Berg and Tamara L. Berg},
title={Combining Multiple Cues for Visual Madlibs Question Answering},
journal={International Journal of Computer Vision},
volume={127},
number={1},
pages={38-60},
year={2019}
}
Tatiana Tommasi, Arun Mallya, Bryan A. Plummer, Svetlana Lazebnik, Alexander C. Berg, Tamara L. Berg. Solving Visual Madlibs with Multiple Cues. BMVC, 2016. [paper][bibtex] @inproceedings{tommasiBMVC2016,
author = {Tatiana Tommasi and Arun Mallya and Bryan A. Plummer and Svetlana Lazebnik and Alexander C. Berg and Tamara L. Berg},
title = {Solving Visual Madlibs with Multiple Cues},
booktitle={British Machine Vision Conference (BMVC)},
year = {2016}
}
|
| | Bryan A. Plummer, Matthew Brown, Svetlana Lazebnik. Enhancing Video Summarization via Vision-Language Embedding. CVPR, 2017. [paper][bibtex] @InProceedings{plummerSummarization2017,
author={Bryan A. Plummer and Matthew Brown and Svetlana Lazebnik},
title={Enhancing Video Summarization via Vision-Language Embedding},
booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2017}
}
|
| | |
| | Bryan A. Plummer*, Nikoli Dryden*, Julius Frost, Torsten Hoefler, Kate Saenko. Neural Parameter Allocation Search. ICLR, 2022. [paper][code][video][bibtex] @InProceedings{plummerNPAS2022,
author={Bryan A. Plummer and Nikoli Dryden and Julius Frost and Torsten Hoefler and Kate Saenko},
title={Neural Parameter Allocation Search},
booktitle={International Conference on Learning Representations (ICLR)},
year={2022}
}
|
| | Donghyun Kim, Tian Lan, Chuhang Zou, Ning Xu, Bryan A. Plummer, Stan Sclaroff, Jayan Eledath, Gerard Medioni. Multi-Task Learning from Videos via Efficient Inter-Frame Attention. ICCV Workshops, 2021. [paper][bibtex] @article{kimMTLearning2021,
author={Donghyun Kim and Tian Lan and Chuhang Zou and Ning Xu and Bryan A. Plummer and Stan Sclaroff and Jayan Eledath and Gerard Medioni},
title={Multi-Task Learning from Videos via Efficient Inter-Frame Attention},
journal={The IEEE International Conference on Computer Vision (ICCV) Workshops},
year={2021}
}
|
| | |
| | Maan Qraitem, Kate Saenko, Bryan A. Plummer. Bias Mimicking: A Simple Sampling Approach for Bias Mitigation. CVPR, 2023. [paper][code][bibtex] @InProceedings{QraitemBM2023,
author={Maan Qraitem and Kate Saenko and Bryan A. Plummer},
title={Bias Mimicking: A Simple Sampling Approach for Bias Mitigation},
booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2023}}
|
| | Olivia Watkins, Sandy Huang, Julius Frost, Kush Bhatia, Eric Weiner, Pieter Abbeel, Trevor Darrell, Bryan A. Plummer, Kate Saenko, Anca Dragan. Explaining robot policies. Applied AI Letters, 2021. [paper][bibtex] @article{watkinsRobot2021,
author={Olivia Watkins and Sandy Huang and Julius Frost and Kush Bhatia and Eric Weiner and Pieter Abbeel and Trevor Darrell and Bryan A. Plummer and Kate Saenko and Anca Dragan},
title={Explaining robot policies},
journal={Applied AI Letters},
year={2021}
}
|
| | Bryan A. Plummer*, Mariya I. Vasileva*, Vitali Petsiuk, Kate Saenko, David Forsyth. Why do These Match? Explaining the Behavior of Image Similarity Models. ECCV, 2020. [paper][code][supplementary][bibtex] @InProceedings{plummerSimilarityExplanations2020,
author={Bryan A. Plummer and Mariya I. Vasileva and Vitali Petsiuk and Kate Saenko and David Forsyth},
title={Why do These Match? Explaining the Behavior of Image Similarity Models},
booktitle = {The European Conference on Computer Vision (ECCV)},
year = {2020}
}
|
| | |
| | Siqi Wang, Manyuan Lu, Nikita Moshkov, Juan C. Caicedo, Bryan A. Plummer. Anchoring to Exemplars for Training Mixture-of-Expert Cell Embeddings. arXiv:2112.03208, 2021. [paper][bibtex] @article{wangTEAMS2021,
author={Siqi Wang and Manyuan Lu and Nikita Moshkov and Juan C. Caicedo and Bryan A. Plummer},
title={Anchoring to Exemplars for Training Mixture-of-Expert Cell Embeddings},
journal={arXiv:2112.03208},
year={2021}
}
|
| | Samarth Mishra*, Zhongping Zhang*, Yuan Shen, Ranjitha Kumar, Venkatesh Saligrama, Bryan A. Plummer. Effectively Leveraging Attributes for Visual Similarity. ICCV, 2021. [paper][code][bibtex] @InProceedings{mishraPAN2021,
author={Samarth Mishra and Zhongping Zhang and Yuan Shen and Ranjitha Kumar and Venkatesh Saligrama and Bryan A. Plummer},
title={Effectively Leveraging Attributes for Visual Similarity},
booktitle={The IEEE International Conference on Computer Vision (ICCV)},
year={2021}
}
|
| | Donghyun Kim, Kuniaki Saito, Samarth Mishra, Stan Sclaroff, Kate Saenko, Bryan A. Plummer. Self-Supervised Visual Attribute Learning for Fashion Compatibility. ICCV Workshops, 2021. [paper][bibtex] @article{kimMTLearning2021,
author={Donghyun Kim and Kuniaki Saito and Samarth Mishra and Stan Sclaroff and Kate Saenko and Bryan A. Plummer},
title={Self-Supervised Visual Attribute Learning for Fashion Compatibility},
journal={The IEEE International Conference on Computer Vision (ICCV) Workshops},
year={2021}
}
|
| | Reuben Tan, Mariya I. Vasileva, Kate Saenko, Bryan A. Plummer. Learning Similarity Conditions Without Explicit Supervision. ICCV, 2019. [paper][code][bibtex] @InProceedings{tanSimilarity2019,
author={Reuben Tan and Mariya I. Vasileva and Kate Saenko and Bryan A. Plummer},
title={Learning Similarity Conditions Without Explicit Supervision},
booktitle={The IEEE International Conference on Computer Vision (ICCV)},
year={2019}
}
|
| | Bryan A. Plummer, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu. Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback. WACV, 2019. [paper][bibtex] @InProceedings{plummerHint2019,
author={Bryan A. Plummer and M. Hadi Kiapour and Shuai Zheng and Robinson Piramuthu},
title={{Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback}},
booktitle={IEEE Winter Conference on Applications of Computer Vision (WACV)},
year={2019}
}
|
| | Mariya I. Vasileva, Bryan A. Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, David Forsyth. Learning Type-Aware Embeddings for Fashion Compatibility. ECCV, 2018. [paper][code][bibtex] @inproceedings{VasilevaECCV18FasionCompatibility,
Author = {Mariya I. Vasileva and Bryan A. Plummer and Krishna Dusad and Shreya Rajpal and Ranjitha Kumar and David Forsyth},
Title = {Learning Type-Aware Embeddings for Fashion Compatibility},
booktitle = {The European Conference on Computer Vision (ECCV)},
Year = {2018}
}
|
| | |
| | Donghyun Kim, Kuniaki Saito, Tae-Hyun Oh, Bryan A. Plummer, Stan Sclaroff, Kate Saenko. CDS: Cross-Domain Self-supervised Pre-training. ICCV, 2021. [paper][bibtex] @InProceedings{kimCDS2021,
author={Donghyun Kim and Kuniaki Saito and Tae-Hyun Oh and Bryan A. Plummer and Stan Sclaroff and Kate Saenko},
title={{CDS: Cross-Domain Self-supervised Pre-training}},
booktitle={The IEEE International Conference on Computer Vision (ICCV)},
year={2021}
}
|
| | P. Daphne Tsatsoulis, Bryan A. Plummer, David Forsyth. Visual Analogies: A Framework for Defining Aspect Categorization. ECCV TASK-CV Workshop, 2016. Workshop Best Paper Award. [paper][bibtex] @inproceedings{TsatsoulisECCV2016,
author = {P. Daphne Tsatsoulis and Bryan A. Plummer and David Forsyth},
title = {Visual Analogies: A Framework for Defining Aspect Categorization},
booktitle={The European Conference on Computer Vision Workshop (ECCVW) on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV)},
year = {2016}
}
|
| | |
| | Andrea Burns, Donghyun Kim, Derry Wijaya, Kate Saenko, Bryan A. Plummer. Learning to Scale Multilingual Representations for Vision-Language Tasks. ECCV, 2020. [paper][project][supplementary][bibtex] @InProceedings{burnsSMALR2020,
author={Andrea Burns and Donghyun Kim and Derry Wijaya and Kate Saenko and Bryan A. Plummer},
title={Learning to Scale Multilingual Representations for Vision-Language Tasks},
booktitle={The European Conference on Computer Vision (ECCV)},
year={2020}
}
|
| | Donghyun Kim, Kuniaki Saito, Kate Saenko, Stan Sclaroff, Bryan A. Plummer. MULE: Multimodal Universal Language Embedding. AAAI, 2020. [paper][project][code][bibtex] @inproceedings{kimMULEAAAI2020,
author={Donghyun Kim and Kuniaki Saito and Kate Saenko and Stan Sclaroff and Bryan A. Plummer},
title={{MULE: Multimodal Universal Language Embedding}},
booktitle={AAAI Conference on Artificial Intelligence},
year={2020}
}
|
| | Andrea Burns, Reuben Tan, Kate Saenko, Stan Sclaroff, Bryan A. Plummer. Language Features Matter: Effective Language Representations for Vision-Language Tasks. ICCV, 2019. [paper][project][bibtex] @InProceedings{burnsLanguage2019,
author={Andrea Burns and Reuben Tan and Kate Saenko and Stan Sclaroff and Bryan A. Plummer},
title={Language Features Matter: Effective Language Representations for Vision-Language Tasks},
booktitle={The IEEE International Conference on Computer Vision (ICCV)},
year={2019}
}
|
