Computer Science > Computer Vision and Pattern Recognition
arXiv:2407.10870 (cs)
[Submitted on 15 Jul 2024]
Title:GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM
View a PDF of the paper titled GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM, by Keshav Bimbraw and 3 other authors
View PDFHTML (experimental)Abstract:Large vision-language models (LVLMs), such as the Generative Pre-trained Transformer 4-omni (GPT-4o), are emerging multi-modal foundation models which have great potential as powerful artificial-intelligence (AI) assistance tools for a myriad of applications, including healthcare, industrial, and academic sectors. Although such foundation models perform well in a wide range of general tasks, their capability without fine-tuning is often limited in specialized tasks. However, full fine-tuning of large foundation models is challenging due to enormous computation/memory/dataset requirements. We show that GPT-4o can decode hand gestures from forearm ultrasound data even with no fine-tuning, and improves with few-shot, in-context learning.
| Comments: | 8 pages, 9 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2407.10870 [cs.CV] |
| (orarXiv:2407.10870v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2407.10870 arXiv-issued DOI via DataCite |
Submission history
From: Toshiaki Koike-Akino [view email][v1] Mon, 15 Jul 2024 16:18:06 UTC (3,148 KB)
Full-text links:
Access Paper:
- View PDF
- HTML (experimental)
- TeX Source
- Other Formats
View a PDF of the paper titled GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM, by Keshav Bimbraw and 3 other authors
Current browse context:
cs.CV
References & Citations
Bookmark
Bibliographic and Citation Tools
Bibliographic Explorer(What is the Explorer?)
Connected Papers(What is Connected Papers?)
Litmaps(What is Litmaps?)
scite Smart Citations(What are Smart Citations?)
Code, Data and Media Associated with this Article
alphaXiv(What is alphaXiv?)
CatalyzeX Code Finder for Papers(What is CatalyzeX?)
DagsHub(What is DagsHub?)
Gotit.pub(What is GotitPub?)
Hugging Face(What is Huggingface?)
Papers with Code(What is Papers with Code?)
ScienceCast(What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower(What are Influence Flowers?)
CORE Recommender(What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community?Learn more about arXivLabs.