Movatterモバイル変換

[0]ホーム
URL:

Skip to main content
Springer Nature Link
Log in

Sparse pinball Universum nonparallel support vector machine and its safe screening rule

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Nonparallel support vector machine (NPSVM) is an effective and popular classification technique, which introduces the\(\epsilon \)-insensitive loss function instead of the quadratic loss function in twin support vector machine (TSVM), making the model have the same sparsity and kernel strategy as support vector machine (SVM). However, NPSVM is sensitive to noise points and does not utilize the prior knowledge embedded in the unlabeled samples. Therefore, to improve its generalization ability and robustness, a sparse pinball Universum nonparallel support vector machine (SPUNPSVM) is first proposed in this paper. On the one hand, the sparse pinball loss is employed to enhance the robustness. On the other hand, it exploits the Universum data, which do not belong to any class, to embed prior knowledge into the model. Numerical experiments have verified its effectiveness. Furthermore, to further speed up SPUNPSVM, we propose a safe screening rule (SSR-SPUNPSVM) based on its sparsity, which achieves acceleration without sacrificing accuracy. Numerical experiments and statistical tests demonstrate the superiority of our SSR-SPUNPSVM.

This is a preview of subscription content,log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  2. Ashraf N, Nayel H, Aldawsari M, Shashirekha H, Elshishtawy T (2024) BFCI at AraFinNLP2024: Support Vector Machines for Arabic Financial Text Classification. In: 2nd Arabic natural language processing conference, Bangkok, Thailand, pp 423–444

  3. Capada A, Deculawan R, Garcia L, Oquias S, Resurreccion R, Cu J, Suarez M (2024) Analyzing Emotion Impact of Mukbang Viewing Through Facial Expression Recognition using Support Vector Machine. In: 26th International conference on multimodal interaction, companion 2024, San Jose, Costa Rica, pp 129–133

  4. Pattanayak R, Behera H, Panigrahi S (2023) A novel high order hesitant fuzzy time series forecasting by using mean aggregated membership value with support vector machine. Inf Sci 626:494–523

    MATH  Google Scholar 

  5. Liu L, Li P, Chu M, Liu S (2023) Robust nonparallel support vector machine with privileged information for pattern recognition. Int J Mach Learn Cybernet 14:1465–1482

    MATH  Google Scholar 

  6. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    MATH  Google Scholar 

  7. Jayadeva R, Khemchandani S (2007) Chandra, Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    MATH  Google Scholar 

  8. Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybernet 44(7):1067–1079

    MATH  Google Scholar 

  9. Liu L, Chu M, Gong R, Zhang L (2021) An improved nonparallel support vector machine. IEEE Trans Neural Netw Learn Syst 32(11):5129–5143

    MathSciNet MATH  Google Scholar 

  10. Wang C, Wang H, Zhou Z (2022) Reductive and effective discriminative information-based nonparallel support vector machine. Appl Intell 52:8259–8278

    MATH  Google Scholar 

  11. Liu L, Li P, Chu M, Zhai Z (2022) L2-loss nonparallel bounded support vector machine for robust classification and its DCD-type solver. Appl Soft Comput 126:109125

    Google Scholar 

  12. Liu L, Li P, Chu M, Gong R (2023) Nonparallel Support Vector Machine with L2-norm Loss and its DCD-type Solver. Neural Process Lett 55:4819–4841

    MATH  Google Scholar 

  13. Huang X, Shi L, Suykens JAK (2014) Support vector machine classifier with pinball loss. IEEE Trans Pattern Anal Mach Intell 36(5):984–997

    MATH  Google Scholar 

  14. Tanveer M, Sharma A, Suganthan PN (2019) General twin support vector machine with pinball loss function. Inf Sci 494:311–327

  15. Ganaie MA, Tanveer M, Jangir J (2023) EEG signal classification via pinball universum twin support vector machine. Annal Operations Res 328(1):451–492

    MATH  Google Scholar 

  16. Li H, Xu Y (2024) Support matrix machine with truncated pinball loss for classification. Appl Soft Comput 154:111311

    MATH  Google Scholar 

  17. Zhang L, Dong D, Luo L, Liu D (2024) A Novel Fuzzy Large Margin Distribution Machine With Unified Pinball Loss. IEEE Trans Fuzzy Syst 32:1782–1795

    MATH  Google Scholar 

  18. Tanveer M, Tiwari A, Choudhary R, Jalan S (2019) Sparse pinball twin support vector machines. Appl Soft Comput 78:164–175

    MATH  Google Scholar 

  19. Vapnik V (2006) Estimation of Dependences Based on Empirical Data. Springer, Second Editiontion

    MATH  Google Scholar 

  20. Weston J, Collobert R, Sinz FH, Bottou L, Vapnik V (2006) Inference with the universum. In: 23th international conference, Pittsburgh, Pennsylvania, USA, pp 1009–1016

  21. Qi Z, Tian Y, Shi Y (2012) Twin support vector machine with universum data. Neural Netw 36:112–119

    MATH  Google Scholar 

  22. Richhariya B, Tanveer M (2022) A fuzzy universum least squares twin support vector machine (FULSTSVM). Neural Comput Appl 34:11411–11422

    MATH  Google Scholar 

  23. Ganaie MA, Tanveer M, Initiative ADN (2022) KNN weighted reduced universum twin SVM for class imbalance learning. Knowl Based Syst 245:108578

    MATH  Google Scholar 

  24. Lou C, Xie X (2024) Intuitionistic fuzzy multi-view support vector machines with universum data. Appl Intell 54:1365–1385

    MATH  Google Scholar 

  25. Shi L, Feng B, Dai Y, Liu G, Shi Y (2023) Pedestrian indoor localization method based on integrated particle filter. IEEE Trans Instrument Measure 72:1–10

    MATH  Google Scholar 

  26. Liu Z, Yang J, Wang L, Chang Y (2023) A novel relation aware wrapper method for feature selection. Pattern Recognition 140:109566

    MATH  Google Scholar 

  27. Carrasco M, Ivorra B, López J, Ramos A (2025) Embedded feature selection for robust probability learning machines. Pattern Recognit 159:111157

    MATH  Google Scholar 

  28. Yu L, Ma X, Li S (2023) A fast conjugate functional gain sequential minimal optimization training algorithm for LS-SVM model. Neural Comput Appl 35(8):6095–6113

    MATH  Google Scholar 

  29. Liang R, Wu X, Zhang Z (2024) Linearized alternating direction method of multipliers for elastic-net support vector machines. Pattern Recognit 148:110134

    MATH  Google Scholar 

  30. An F, Zhao B, Cui B, Chen Y, Qu L, Yu Z, Zeng R (2024) DC cascaded energy storage system based on DC collector with gradient descent method. IEEE Trans Indust Electron 71(2):1594–1605

  31. Yu Z, Li P (2021) A trust region method with project step for bound constrained optimization without compact condition. Int J Comput Math 98(3):449–460

    MathSciNet MATH  Google Scholar 

  32. Pang X, Xu Y (2023) A reconstructed feasible solution-based safe feature elimination rule for expediting multi-task lasso. Inf Sci 642:119142

    MATH  Google Scholar 

  33. Ogawa K, Suzuki Y, Takeuchi I (2013) Safe screening of non-support vectors in pathwise SVM computation. In: 30th international conference on machine learning, Atlanta, GA, USA, pp 1382–1390

  34. Liu J, Zhao Z, Wang J, Ye J (2014) Safe screening with variational inequalities and its application to lasso. In: 31th international conference on machine learning, Beijing, China, pp 289–297

  35. Wang J, Wonka P, Ye J (2014) Scaling SVM and least absolute deviations via exact data reduction. In: 31th international conference on machine learning, Beijing, China, pp 523–531

  36. Li Y, Sun H (2023) Safe sample screening for robust twin support vector machine. Appl Intell 53:20059–20075

    MATH  Google Scholar 

  37. Pang X, Xu C, Xu Y (2024) MTKSVCR: A novel multi-task multi-class support vector machine with safe acceleration rule. Neural Netw 175:106317

    MATH  Google Scholar 

  38. Yang Z, Chen W, Zhang H, Xu Y, Shi L, Zhao J (2024) A safe screening rule with bi-level optimization of\(\nu \) support vector machine. Pattern Recognit 155:110644

    Google Scholar 

  39. Wang H, Zhu J, Feng F (2023) Elastic net twin support vector machine and its safe screening rules. Inf Sci 635:99–125

    MATH  Google Scholar 

  40. Wang X, Xu Y (2023) Label Pair of Instances-Based Safe Screening for Multilabel Rank Support Vector Machine. IEEE Trans Syst, Man, Cybernet 53:1907–1919

    MATH  Google Scholar 

  41. Wang H, Zhu J, Zhang S (2023) Safe screening rules for multi-view support vector machines. Neural Netw 166:326–343

    MATH  Google Scholar 

  42. Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework. J Multiple-Valued Logic Soft Comput 17(2–3):255–287

    MATH  Google Scholar 

  43. Yin J, Li C (2023) Feature selection SVM through Universum and its applications on text classification Feature selection SVM through Universum. In: 2nd international conference on signal processing, computer networks and communications, Xiamen, China, pp 408–414

  44. Li C, Huang L, Shao Y, Guo T, Mao Y (2024) Feature selection by Universum embedding. Pattern Recognit 153:110514

    Google Scholar 

  45. Liu Y, Pass R (2023) One-Way Functions and the Hardness of (Probabilistic) Time-Bounded Kolmogorov Complexity w.r.t. Samplable Distributions. In: 43rd annual international cryptology conference, Santa Barbara, CA, USA, pp 20–24

  46. Hromkovic J (2024) Kolmogorov complexity and nondeterminism versus determinism for polynomial time computations. Theoretical Comput Sci 1013:114747

    MathSciNet MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the reviewers for the helpful comments and suggestions, which have improved the presentation. This work was supported in part by the National Natural Science Foundation of China (No. 12301402, 12071475, 62102236), Youth Innovation Team of Higher Education Institutions in Shandong Province (2024KJG016), and Shandong Provincial Natural Science Foundation, China (No. ZR2022QA003, ZR2023QA042).

Author information

Authors and Affiliations

  1. Business School, Shandong Normal University, Jinan, 250014, China

    Hongmei Wang, Ping Li & Yuyan Zheng

  2. Faculty of Mathematics and Artificial Intelligence, Qilu University of Technology (Shandong Academy of Sciences), Jinan, 250353, China

    Kun Jiang

  3. College of Science, China Agricultural University, Beijing, 100083, China

    Yitian Xu

Authors
  1. Hongmei Wang

    You can also search for this author inPubMed Google Scholar

  2. Ping Li

    You can also search for this author inPubMed Google Scholar

  3. Yuyan Zheng

    You can also search for this author inPubMed Google Scholar

  4. Kun Jiang

    You can also search for this author inPubMed Google Scholar

  5. Yitian Xu

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toYitian Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A The proof of Theorem1

Obviously, the objective function\(\frac{1}{2}\lambda ^{T}ZZ^{T}\lambda -\lambda ^{T}\nu \) of problem (27) is a Gateaux differentiable function on\(\lambda \in [0,P]\). It is noted that the ranges of the optimal solutions\(\lambda _{0}^{*}\) and\(\lambda _{1}^{*}\) are not the same, so we transform them as

$$\begin{aligned}&P_{0}\odot \lambda _{1}^{*}\oslash P_{1}\in [0,P_{0}],\end{aligned}$$
(A.1)
$$\begin{aligned}&P_{1}\odot \lambda _{0}^{*}\oslash P_{0}\in [0,P_{1}]. \end{aligned}$$
(A.2)

According to (A.1), (A.2), and Lemma1, the following two inequalities can be obtained:

$$\begin{aligned}&\langle ZZ^{T}\lambda _{0}^{*} - \nu ,~P_{0}\odot \lambda _{1}^{*}\oslash P_{1}-\lambda _{0}^{*} \rangle \ge 0, \\&\langle ZZ^{T}\lambda _{1}^{*} - \nu ,~P_{1}\odot \lambda _{0}^{*}\oslash P_{0}-\lambda _{1}^{*} \rangle \ge 0. \end{aligned}$$

By adding the above two inequalities and rearranging them, we can obtain

$$\begin{aligned}&(\lambda _{1}^{*}-P_{1}\odot \lambda _{0}^{*}\oslash P_{0})^{T}(ZZ^{T}\lambda _{0}^{*}-ZZ^{T}\lambda _{1}^{*})\ge 0 \\ \Rightarrow&\lambda _{1}^{*T}ZZ^{T}\lambda _{1}^{*}+\lambda _{0}^{*T}ZZ^{T}(P_{1}\odot \lambda _{0}^{*}\oslash P_{0})\\&-\lambda _{1}^{*T}ZZ^{T}{[}\lambda _{0}^{*}\odot (P_{0}+P_{1})\oslash P_{0}{]}\le 0 \\ \Rightarrow&\lambda _{1}^{*T}ZZ^{T}\lambda _{1}^{*}+\frac{1}{4}[\lambda _{0}^{*}\odot (P_{1}+P_{0})\oslash P_{0}]^{T}ZZ^{T}[\lambda _{0}^{*}\odot (P_{1}+P_{0})\oslash P_{0}{]}\\&-\lambda _{1}^{*T}ZZ^{T}[\lambda _{0}^{*}(P_{0}+P_{1})\oslash P_{0}{]}\\ ~&~~~\le \frac{1}{4}[\lambda _{0}^{*}\odot (P_{1}+P_{0})\oslash P_{0}]^{T}ZZ^{T}[\lambda _{0}^{*}\odot (P_{1}+P_{0})\oslash P_{0}]\\&-\lambda _{0}^{*T}ZZ^{T}(\lambda _{0}^{*}\odot P_{1}\oslash P_{0})\\ ~\Rightarrow&\parallel Z^{T}\lambda _{1}^{*}-\frac{1}{2}Z^{T}{[}\lambda _{0}^{*}\odot (P_{1}+P_{0})\oslash P_{0}{]}\parallel \\ ~&~~~\le \frac{1}{2}\parallel Z^{T}{[}\lambda _{0}^{*}\odot (P_{1}-P_{0})\oslash P_{0}{]}\parallel . \end{aligned}$$

Then, the final conclusion can be obtained.

Appendix B The proof of Theorem2

According to Theorem1, we know that\(Z^{T}\lambda ^{*}_{1}\) is in the region\(\Omega \)=Ball(c,r), where\(c=\frac{1}{2}Z^{T}[\lambda _{0}^{*}\odot (P_{1}+P_{0})\oslash P_{0}]\) and\(r=\frac{1}{2}\parallel Z^{T}[\lambda _{0}^{*}\odot (P_{1}-P_{0})\oslash P_{0}]\parallel \). Then, the lower bound of\(Z_{i}Z^{T}\lambda _{1}^{*}\) can be obtained:

$$\begin{aligned}&\underset{Z^{T}\lambda ^{*}_{1}\in \Omega }{min}Z_{i}Z^{T}\lambda _{1}^{*} \\ =&\underset{Z^{T}\lambda ^{*}_{1}\in Ball(c,r)}{min}Z_{i}Z^{T}\lambda _{1}^{*} \\ =&\underset{\widetilde{r}:\parallel \widetilde{r}\parallel \le r}{min} Z_{i}(c+\widetilde{r}) \\ =&\frac{1}{2}Z_{i}Z^{T}[\lambda _{0}^{*}\odot (P_{0}+P_{1})\oslash P_{0}]-\frac{1}{2}\parallel Z_{i}\parallel \parallel Z^{T}[\lambda _{0}^{*}\odot (P_{1}-P_{0})\oslash P_{0}]\parallel . \end{aligned}$$

Similarly, the upper bound of\(H_{i}\lambda _{1}^{*}\) is

$$\begin{aligned} \underset{Z^{T}\lambda ^{*}_{1}\in \Omega }{max}Z_{i}Z^{T}\lambda _{1}^{*}&= \frac{1}{2}Z_{i}Z^{T}[\lambda _{0}^{*}\odot (P_{0}+P_{1})\oslash P_{0}]\nonumber \\&+\frac{1}{2}\parallel Z_{i}\parallel \parallel Z^{T}[\lambda _{0}^{*}\odot (P_{1}-P_{0})\oslash P_{0}]\parallel . \end{aligned}$$

Then, according to (40), the final rule can be obtained.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Access this article

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Advertisement

[8]ページ先頭
©2009-2025 Movatter.jp