Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit50dcbc6

Browse files
author
James Lee
committed
adopt
1 parent8ffb3e5 commit50dcbc6

File tree

8 files changed

+42
-79
lines changed

8 files changed

+42
-79
lines changed

‎rdd/WordCount.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
importsys
2+
frompysparkimportSparkContext
3+
4+
if__name__=="__main__":
5+
sc=SparkContext("local","word count")
6+
lines=sc.textFile("in/word_count.text")
7+
words=lines.flatMap(lambdaline:line.split(" "))
8+
wordCounts=words.countByValue()
9+
forword,countinwordCounts.items():
10+
print(word,count)

‎rdd/WordCount.scala

Lines changed: 0 additions & 22 deletions
This file was deleted.
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
2+
importsys
3+
frompysparkimportSparkContext
4+
5+
if__name__=="__main__":
6+
7+
'''
8+
Create a Spark program to read the first 100 prime numbers from in/prime_nums.text,
9+
print the sum of those numbers to console.
10+
Each row of the input file contains 10 prime numbers separated by spaces.
11+
'''

‎rdd/sumOfNumbers/SumOfNumbersProblem.scala

Lines changed: 0 additions & 13 deletions
This file was deleted.
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
importsys
2+
frompysparkimportSparkContext
3+
4+
if__name__=="__main__":
5+
sc=SparkContext("local","primeNumbers")
6+
lines=sc.textFile("in/prime_nums.text")
7+
numbers=lines.flatMap(lambdaline:line.split("\t"))
8+
validNumbers=numbers.filter(lambdanumber:number)
9+
intNumbers=validNumbers.map(lambdanumber:int(number))
10+
print("Sum is: ")
11+
print(intNumbers.reduce(lambdax,y:x+y))

‎rdd/sumOfNumbers/SumOfNumbersSolution.scala

Lines changed: 0 additions & 25 deletions
This file was deleted.

‎rdd/take/TakeExample.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
importsys
2+
frompysparkimportSparkContext
3+
4+
if__name__=="__main__":
5+
sc=SparkContext("local","take")
6+
inputWords= ["spark","hadoop","spark","hive","pig","cassandra","hadoop"]
7+
wordRdd=sc.parallelize(inputWords)
8+
words=wordRdd.take(3)
9+
forwordinwords:
10+
print(word)

‎rdd/take/TakeExample.scala

Lines changed: 0 additions & 19 deletions
This file was deleted.

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp