Motivated by the scarcity of high-quality labeled biomedical text, as well as the success of data programming, we introduce KRISS-Search. By leveraging the Unified Medical Language Systems (UMLS) ontology, KRISS-Search addresses an interactive few-shot span recommendation task that we propose. We first introduce unsupervised KRISS-Search and show that our method outperforms existing methods in identifying spans that are semantically similar to a given span of interest, with >50% AUPRC improvement relative to PubMedBERT. We then introduce supervised KRISS-Search, which leverages human interaction to improve the notion of similarity used by unsupervised KRISS-Search. Through simulated human feedback, we demonstrate an enhanced F1 score of 0.68 in classifying spans as semantically similar or different in the low-label setting, outperforming PubMedBERT by 2 F1 points. Finally, supervised KRISS-Search demonstrates competitive or superior performance compared to PubMedBERT in few-shot biomedical named entity recognition (NER) across five benchmark datasets, with an average improvement of 5.6 F1 points. We envision KRISS-Search increasing the efficiency of programmatic data labeling and also providing broader utility as an interactive biomedical search engine.
Louis Blankemeier, Theodore Zhao, Robert Tinn, Sid Kiblawi, Yu Gu, Akshay Chaudhari, Hoifung Poon, Sheng Zhang, Mu Wei, and J. Preston. 2023.Interactive Span Recommendation for Biomedical Text. InProceedings of the 5th Clinical Natural Language Processing Workshop, pages 373–384, Toronto, Canada. Association for Computational Linguistics.
@inproceedings{blankemeier-etal-2023-interactive, title = "Interactive Span Recommendation for Biomedical Text", author = "Blankemeier, Louis and Zhao, Theodore and Tinn, Robert and Kiblawi, Sid and Gu, Yu and Chaudhari, Akshay and Poon, Hoifung and Zhang, Sheng and Wei, Mu and Preston, J.", editor = "Naumann, Tristan and Ben Abacha, Asma and Bethard, Steven and Roberts, Kirk and Rumshisky, Anna", booktitle = "Proceedings of the 5th Clinical Natural Language Processing Workshop", month = jul, year = "2023", address = "Toronto, Canada", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2023.clinicalnlp-1.40/", doi = "10.18653/v1/2023.clinicalnlp-1.40", pages = "373--384", abstract = "Motivated by the scarcity of high-quality labeled biomedical text, as well as the success of data programming, we introduce KRISS-Search. By leveraging the Unified Medical Language Systems (UMLS) ontology, KRISS-Search addresses an interactive few-shot span recommendation task that we propose. We first introduce unsupervised KRISS-Search and show that our method outperforms existing methods in identifying spans that are semantically similar to a given span of interest, with {\ensuremath{>}}50{\%} AUPRC improvement relative to PubMedBERT. We then introduce supervised KRISS-Search, which leverages human interaction to improve the notion of similarity used by unsupervised KRISS-Search. Through simulated human feedback, we demonstrate an enhanced F1 score of 0.68 in classifying spans as semantically similar or different in the low-label setting, outperforming PubMedBERT by 2 F1 points. Finally, supervised KRISS-Search demonstrates competitive or superior performance compared to PubMedBERT in few-shot biomedical named entity recognition (NER) across five benchmark datasets, with an average improvement of 5.6 F1 points. We envision KRISS-Search increasing the efficiency of programmatic data labeling and also providing broader utility as an interactive biomedical search engine."}
%0 Conference Proceedings%T Interactive Span Recommendation for Biomedical Text%A Blankemeier, Louis%A Zhao, Theodore%A Tinn, Robert%A Kiblawi, Sid%A Gu, Yu%A Chaudhari, Akshay%A Poon, Hoifung%A Zhang, Sheng%A Wei, Mu%A Preston, J.%Y Naumann, Tristan%Y Ben Abacha, Asma%Y Bethard, Steven%Y Roberts, Kirk%Y Rumshisky, Anna%S Proceedings of the 5th Clinical Natural Language Processing Workshop%D 2023%8 July%I Association for Computational Linguistics%C Toronto, Canada%F blankemeier-etal-2023-interactive%X Motivated by the scarcity of high-quality labeled biomedical text, as well as the success of data programming, we introduce KRISS-Search. By leveraging the Unified Medical Language Systems (UMLS) ontology, KRISS-Search addresses an interactive few-shot span recommendation task that we propose. We first introduce unsupervised KRISS-Search and show that our method outperforms existing methods in identifying spans that are semantically similar to a given span of interest, with \ensuremath>50% AUPRC improvement relative to PubMedBERT. We then introduce supervised KRISS-Search, which leverages human interaction to improve the notion of similarity used by unsupervised KRISS-Search. Through simulated human feedback, we demonstrate an enhanced F1 score of 0.68 in classifying spans as semantically similar or different in the low-label setting, outperforming PubMedBERT by 2 F1 points. Finally, supervised KRISS-Search demonstrates competitive or superior performance compared to PubMedBERT in few-shot biomedical named entity recognition (NER) across five benchmark datasets, with an average improvement of 5.6 F1 points. We envision KRISS-Search increasing the efficiency of programmatic data labeling and also providing broader utility as an interactive biomedical search engine.%R 10.18653/v1/2023.clinicalnlp-1.40%U https://aclanthology.org/2023.clinicalnlp-1.40/%U https://doi.org/10.18653/v1/2023.clinicalnlp-1.40%P 373-384
Louis Blankemeier, Theodore Zhao, Robert Tinn, Sid Kiblawi, Yu Gu, Akshay Chaudhari, Hoifung Poon, Sheng Zhang, Mu Wei, and J. Preston. 2023.Interactive Span Recommendation for Biomedical Text. InProceedings of the 5th Clinical Natural Language Processing Workshop, pages 373–384, Toronto, Canada. Association for Computational Linguistics.