spark-kafka-integration

Star

Here are 12 public repositories matching this topic...

Language:All

Filter by language

All12 Scala7 Java3 Python2

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

spider-123-eng /Spark

Star54

Apache Spark is a fast, in-memory data processing engine with elegant and expressive development API's to allow data workers to efficiently execute streaming, machine learning or SQL workloads that require fast iterative access to datasets.This project will have sample programs for Spark in Scala language .

streaming consumer parquet kafka-producer spark-sql spark-kafka-integration spark-streaming-data spark-transformations spark-to-cassandra-connection spark-dataframes spark-joins spark-hive-context spark-jdbc-connection spark-with-mangodb spark-aggregations-using-dataframe spark-use-cases cassandra-installation spark-datadog spark-mangodb spark-catalog-api

UpdatedNov 16, 2022
Scala

zekeriyyaa /PySpark-Structured-Streaming-ROS-Kafka-ApacheSpark-Cassandra

Star19

A structured streaming was applied to the robot data from ROS-Gazebo simulation environment using Apache Spark. Data is collected in Kafka, analyzed by Apache Spark and stored in Cassandra.

python apache-spark cqlsh python3 ros pyspark spark-streaming kafka-consumer data-analysis apache-kafka kafka-producer apache-cassandra structured-streaming spark-sql spark-kafka-integration spark-cassandra-connector spark-cassandra ros-noetic spark-kafka-connector

UpdatedFeb 6, 2022
Python

akashsethi24 /Spark-Kafka-Example

Star6

Example for Data Reading from and Writing to from Kafka Topic using Apache Spark DataFrame and DataSet

kafka apache-spark spark-streaming kafka-streams spark-kafka-integration custom-kafka-serializer custom-kafka-deseriazer kafka-decoder kafka-encoder

UpdatedOct 13, 2017
Scala

xuanbo /spark-programming

Star4

spark代码

spark spark-streaming spark-sql spark-kafka-integration

UpdatedJun 21, 2022
Scala

mayankrawat /CSVJoin

Star2

Use this project to join data from multiple csv files. Currently in this project we support one to one and one to many join. Along with this you can find how to use kafka producer efficiently with spark.