Movatterモバイル変換

[0]ホーム
URL:

Skip to main content

Advertisement

Springer Nature Link
Log in

Enhancing Text-Image Person Retrieval Through Nuances Varied Sample

  • Conference paper
  • First Online:

Part of the book series:Lecture Notes in Computer Science ((LNCS,volume 14425))

  • 1711Accesses

Abstract

Text-image person retrieval is a task that involves searching for a specific individual based on a corresponding textual description. However, a key challenge in this task is achieving modal alignment while conducting fine-grained retrieval. Current methods utilize classification and metric losses to enhance discrimination and alignment. Nevertheless, the substantial dissimilarities between samples often impede the network’s capacity to learn discriminative fine-grained information. To tackle this issue and enable the network to focus on intricate details, we introduce the Nuanced Variation Module (NVM). This module generates artificially difficult negative samples, which serve as a guide for directing the network’s attention towards discerning nuances. The incorporation of NVM-constructed hard-negative samples enhances the alignment loss and facilitates the network’s attentiveness to details. Additionally, we leverage the image text matching task to explicitly augment the network’s fine-grained ability. By adopting our NVM method, the network can extract an ample amount of fine-grained features, thereby mitigating the interference caused by challenging negative samples. Extensive experiments demonstrate that our proposed method achieves competitive performance compared to state-of-the-art approaches on publicly available datasets.

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 9151
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 11439
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

References

  1. Chen, Y., Zhang, G., Lu, Y., Wang, Z., Zheng, Y.: TIPCB: a simple but effective part-based convolutional baseline for text-based person search. Neurocomputing494, 171–181 (2022)

    Article  Google Scholar 

  2. Ding, Z., Ding, C., Shao, Z., Tao, D.: Semantically self-aligned network for text-to-image part-aware person re-identification. arXiv preprintarXiv:2107.12666 (2021)

  3. Farooq, A., Awais, M., Kittler, J., Khalid, S.S.: AXM-net: implicit cross-modal feature alignment for person re-identification. Pattern Recogn.36(4), 4477–4485 (2022)

    Google Scholar 

  4. Frome, A., et al.: Devise: a deep visual-semantic embedding model. Adv. Neural. Inf. Process. Syst.26, 1–9 (2013)

    Google Scholar 

  5. Gao, C., et al.: Contextual non-local alignment over full-scale representation for text-based person search. arXiv preprintarXiv:2101.03036 (2021)

  6. Gu, J., Cai, J., Joty, S.R., Niu, L., Wang, G.: Look, imagine and match: Improving textual-visual cross-modal retrieval with generative models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7181–7189 (2018)

    Google Scholar 

  7. Han, X., He, S., Zhang, L., Xiang, T.: Text-based person search with limited data. arXiv preprintarXiv:2110.10807 (2021)

  8. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  9. Jiang, D., Ye, M.: Cross-modal implicit relation reasoning and aligning for text-to-image person retrieval. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2787–2797 (2023)

    Google Scholar 

  10. Klein, B., Lev, G., Sadeh, G., Wolf, L.: Associating neural word embeddings with deep image representations using fisher vectors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4437–4446 (2015)

    Google Scholar 

  11. Li, S., Cao, M., Zhang, M.: Learning semanticaligned feature representation for text-based person search. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2724–2728. IEEE (2022)

    Google Scholar 

  12. Li, S., Xiao, T., Li, H., Zhou, B., Yue, D., Wang, X.: Person search with natural language description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1970–1979 (2017)

    Google Scholar 

  13. Lv, X., Zhao, C., Chen, W.: A novel hard mining center-triplet loss for person re-identification. In: Lin, Z., et al. (eds.) PRCV 2019. LNCS, vol. 11859, pp. 199–210. Springer, Cham (2019).https://doi.org/10.1007/978-3-030-31726-3_17

    Chapter  Google Scholar 

  14. Niu, K., Huang, Y., Ouyang, W., Wang, L.: Improving description-based person re-identification by multi-granularity image-text alignments. IEEE Trans. Image Process.29, 5542–5556 (2020)

    Article  Google Scholar 

  15. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)

    Google Scholar 

  16. Sarafianos, N., Xu, X., Kakadiaris, I.A.: Adversarial representation learning for text-to-image matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5814–5824 (2019)

    Google Scholar 

  17. Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. arXiv preprintarXiv:1508.07909 (2015)

  18. Shao, Z., Zhang, X., Fang, M., Lin, Z., Wang, J., Ding, C.: Learning granularity-unified representations for text-to-image person re-identification. arXiv preprintarXiv:2207.07802 (2022)

  19. Shu, X., et al.: See finer, see more: implicit modality alignment for text-based person retrieval. arXiv preprintarXiv:2208.08608 (2022)

  20. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  21. Wang, L., Li, Y., Lazebnik, S.: Learning deep structure-preserving image-text embeddings. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5005–5013 (2016)

    Google Scholar 

  22. Wang, Z., Fang, Z., Wang, J., Yang, Y.: ViTAA: visual-textual attributes alignment in person search by natural language. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 402–420. Springer, Cham (2020).https://doi.org/10.1007/978-3-030-58610-2_24

    Chapter  Google Scholar 

  23. Wang, Z., Xue, J., Zhu, A., Li, Y., Zhang, M., Zhong, C.: Amen: Adversarial multi-space embedding network for text-based person re-identification. In: Pattern Recognition and Computer Vision: 4th Chinese Conference, PRCV 2021, Beijing, China, October 29-November 1, 2021, Proceedings, Part II 4. pp. 462–473. Springer (2021)

    Google Scholar 

  24. Wang, Z., Zhu, A., Xue, J., Wan, X., Liu, C., Wang, T., Li, Y.: Caibc: Capturing all-round information beyond color for text-based person retrieval. arXiv preprintarXiv:2209.05773 (2022)

  25. Wang, Z., Zhu, A., Xue, J., Wan, X., Liu, C., Wang, T., Li, Y.: Look before you leap: Improving text-based person retrieval by learning a consistent cross-modal common manifold. In: Proceedings of the 30th ACM International Conference on Multimedia. pp. 1984–1992 (2022)

    Google Scholar 

  26. Wu, Y., Yan, Z., Han, X., Li, G., Zou, C., Cui, S.: LapScore: language-guided person search via color reasoning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1624–1633 (2021)

    Google Scholar 

  27. Yan, S., Dong, N., Zhang, L., Tang, J.: Clip-driven fine-grained text-image person re-identification. arXiv preprintarXiv:2210.10276 (2022)

  28. Yan, S., Tang, H., Zhang, L., Tang, J.: Image-specific information suppression and implicit local alignment for text-based person search. arXiv preprintarXiv:2208.14365 (2022)

  29. Zhang, Ying, Lu, Huchuan: Deep cross-modal projection learning for image-text matching. In: Ferrari, Vittorio, Hebert, Martial, Sminchisescu, Cristian, Weiss, Yair (eds.) ECCV 2018. LNCS, vol. 11205, pp. 707–723. Springer, Cham (2018).https://doi.org/10.1007/978-3-030-01246-5_42

    Chapter  Google Scholar 

  30. Zheng, Z., Zheng, L., Garrett, M., Yang, Y., Xu, M., Shen, Y.D.: Dual-path convolutional image-text embeddings with instance loss. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM)16(2), 1–23 (2020)

    Article  Google Scholar 

  31. Zhu, A., et al.: DSSL: deep surroundings-person separation learning for text-based person retrieval. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 209–217 (2021)

    Google Scholar 

Download references

Acknowledgement

This work was supported by National Key R &D Program of China (No. 2022ZD0 118202), the National Science Fund for Distinguished Young Scholars (No. 620256 03), the National Natural Science Foundation of China (No. U21B2037, No. U22B2051, No. 62176222, No. 62176223, No. 62176226, No. 62072386, No. 620723 87, No. 62072389, No. 62002305 and No. 62272401), and the Natural Science Foundation of Fujian Province of China (No. 2021J01002, No. 2022J06001).

Author information

Authors and Affiliations

  1. Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, School of Informatics, Xiamen University, 361005, Xiamen, People’s Republic of China

    Jiaer Xia, Haozhe Yang, Yan Zhang & Pingyang Dai

Authors
  1. Jiaer Xia

    You can also search for this author inPubMed Google Scholar

  2. Haozhe Yang

    You can also search for this author inPubMed Google Scholar

  3. Yan Zhang

    You can also search for this author inPubMed Google Scholar

  4. Pingyang Dai

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toPingyang Dai.

Editor information

Editors and Affiliations

  1. Nanjing University of Information Science and Technology, Nanjing, China

    Qingshan Liu

  2. Xiamen University, Xiamen, China

    Hanzi Wang

  3. Beijing University of Posts and Telecommunications, Beijing, China

    Zhanyu Ma

  4. Sun Yat-sen University, Guangzhou, China

    Weishi Zheng

  5. Peking University, Beijing, China

    Hongbin Zha

  6. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  7. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  8. Xiamen University, Xiamen, China

    Rongrong Ji

Rights and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xia, J., Yang, H., Zhang, Y., Dai, P. (2024). Enhancing Text-Image Person Retrieval Through Nuances Varied Sample. In: Liu, Q.,et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14425. Springer, Singapore. https://doi.org/10.1007/978-981-99-8429-9_15

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 9151
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 11439
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

[8]ページ先頭
©2009-2025 Movatter.jp