Commit43f7883

Pedro Bernardo

committed

Added pairRdd/aggregation/reducebykey/WordCount.py

1 parent7bcccc9 commit43f7883Copy full SHA for 43f7883

File tree

+14

-0

lines changed

+14

-0

lines changed

Lines changed: 14 additions & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,14 @@`
	`1`	`+frompysparkimportSparkContext`
	`2`	`+`
	`3`	`+if__name__=="__main__":`
	`4`	`+`
	`5`	`+sc=SparkContext("local","wordCounts")`
	`6`	`+sc.setLogLevel("ERROR")`
	`7`	`+`
	`8`	`+lines=sc.textFile("in/word_count.text")`
	`9`	`+wordRdd=lines.flatMap(lambdaline:line.split(" "))`
	`10`	`+wordPairRdd=wordRdd.map(lambdaword: (word,1))`
	`11`	`+`
	`12`	`+wordCounts=wordPairRdd.reduceByKey(lambdax,y:x+y)`
	`13`	`+forword,countinwordCounts.collect():`
	`14`	`+print("{} : {}".format(word,count))`

Comments

(0)