Movatterモバイル変換

[0]ホーム
URL:

Skip to main content

Advertisement

Springer Nature Link
Log in

Approximate Fault Tolerance for Edge Stream Processing

  • Conference paper
  • First Online:

Abstract

Existing distributed stream processing systems generally guarantee fault tolerance by switching to standby machines and reprocessing lost data. In edge computing environments, however, we have to duplicate each edge for this conventional approach. This duplication cost increases sharply with expansion in the system scale. To solve this problem, we propose an approach to supportapproximate fault tolerance without edge duplication. We focus on environmental monitoring applications and utilize the correlation between sensors. In this paper, we assume that each edge estimates missing data from the observed data and aggregates them approximately. We provide a method to estimate the outputs of failed edges taking care of the uncertainty of the processing results at each edge. Our method allows the server to continue processing without waiting for the recovery of failed edges. We also show that the validity of our method by experiments using synthetic data.

This paper is based on results obtained from a project, JPNP16007, commissioned by the New Energy and Industrial Technology Development Organization (NEDO). Also, This work was partly supported by KAKENHI (16H01722 and 20K19804).

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 5719
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 7149
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

References

  1. 2JCIE-BL Environment Sensor\(|\) OMRON - Americas.https://components.omron.com/product-detail?partId=73064. Accessed 20 Apr 2021

  2. Apache Flink: Stateful Computations over Data Streams.https://flink.apache.org/ . Accessed 20 Apr 2021

  3. Apache Storm.https://storm.apache.org/. Accessed 20 Apr 2021

  4. CRAN - package mvnormtest.https://CRAN.R-project.org/package=mvnormtest. Accessed 20 Apr 2021

  5. Mvnorm function\(|\) R Documentation.https://www.rdocumentation.org/packages/mvtnorm/versions/1.0-11/topics/Mvnorm. Accessed 20 Apr 2021

  6. Spark Streaming\(|\) Apache Spark.https://spark.apache.org/streaming/. Accessed 20 Apr 2021

  7. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006).https://doi.org/10.1007/978-0-387-45528-0

  8. Daiki, T., Kento, S., Yoshiharu, I.: Approximate streaming aggregation with low-latency and high-reliability for edge computing. IEICE Trans. Inf. Syst.J104-D(5), 463–475 (2021). (in Japanese)

    Google Scholar 

  9. Deshpande, A., Guestrin, C., Madden, S.R., Hellerstein, J.M., Hong, W.: Model-based approximate querying in sensor networks. VLDB J.14(4), 417–443 (2005)

    Article  Google Scholar 

  10. Enders, C.K.: Applied Missing Data Analysis. Guilford Press, New York (2010)

    Google Scholar 

  11. Huang, Q., Lee, P.P.C.: Toward high-performance distributed stream processing via approximate fault tolerance. Proc. VLDB10(3), 73–84 (2016)

    Article  Google Scholar 

  12. Hwang, J.H., Balazinska, M., Rasin, A., Çetintemel, U., Stonebraker, M., Zdonik, S.: High-availability algorithms for distributed stream processing. In: Proceedings of ICDE, pp. 779–790, April 2005

    Google Scholar 

  13. Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: Distributed stream computing platform. In: 2010 IEEE International Conference on Data Mining Workshops, pp. 170–177, January 2010

    Google Scholar 

  14. Rasmussen, E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2006)

    Google Scholar 

  15. Shi, W., Cao, J., Zhang, Q., Li, Y., Xu, L.: Edge computing: vision and challenges. IEEE Internet Things J.3(5), 637–646 (2016)

    Article  Google Scholar 

  16. Zaharia, M., Das, T., Li, H., Hunter, T., Shenker, S., Stoica, I.: Discretized streams: a fault-tolerant model for scalable stream processing. Technical report, California University of Berkeley, Department of Electrical Engineering and Computer Science, (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Informatics, Nagoya University, Aichi, Japan

    Daiki Takao, Kento Sugiura & Yoshiharu Ishikawa

Authors
  1. Daiki Takao

    You can also search for this author inPubMed Google Scholar

  2. Kento Sugiura

    You can also search for this author inPubMed Google Scholar

  3. Yoshiharu Ishikawa

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toDaiki Takao.

Editor information

Editors and Affiliations

  1. Johannes Kepler University of Linz, Linz, Austria

    Gabriele Kotsis

  2. Vienna University of Technology, Vienna, Austria

    A Min Tjoa

  3. Johannes Kepler University of Linz, Linz, Austria

    Ismail Khalil

  4. Software Competence Center Hagenberg, Hagenberg, Austria

    Bernhard Moser

  5. Johannes Kepler University of Linz, Linz, Austria

    Atif Mashkoor

  6. Johannes Kepler University of Linz, Linz, Austria

    Johannes Sametinger

  7. University of Innsbruck, Innsbruck, Austria

    Anna Fensel

  8. Software Competence Center Hagenberg, Hagenberg, Austria

    Jorge Martinez-Gil

  9. Software Competence Center Hagenberg, Hagenberg, Austria

    Lukas Fischer

  10. Software Competence Center Hagenberg, Hagenberg, Austria

    Gerald Czech

  11. Software Competence Center Hagenberg, Hagenberg, Australia

    Florian Sobieczky

  12. Sino-Pak Center for Artificial Intelligence, Haripur, Pakistan

    Sohail Khan

Rights and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Takao, D., Sugiura, K., Ishikawa, Y. (2021). Approximate Fault Tolerance for Edge Stream Processing. In: Kotsis, G.,et al. Database and Expert Systems Applications - DEXA 2021 Workshops. DEXA 2021. Communications in Computer and Information Science, vol 1479. Springer, Cham. https://doi.org/10.1007/978-3-030-87101-7_17

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 5719
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 7149
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

[8]ページ先頭
©2009-2025 Movatter.jp