Commita6dc078

Pedro Bernardo

committed

Changed SortedWordCountSolution.py to use the sortBy operation

1 parentf9c885f commita6dc078Copy full SHA for a6dc078

File tree

-10

lines changed

-10

lines changed

Lines changed: 8 additions & 10 deletions

Original file line number	Diff line number	Diff line change
`@@ -1,20 +1,18 @@`
`1`		`-frompysparkimportSparkContext`
	`1`	`+frompysparkimportSparkContext,SparkConf`
`2`	`2`
`3`	`3`	`if__name__=="__main__":`
`4`		`-`
`5`		`-sc=SparkContext("local","wordCounts")`
`6`		`-sc.setLogLevel("ERROR")`
	`4`	`+conf=SparkConf().setAppName("wordCounts").setMaster("local[*]")`
	`5`	`+sc=SparkContext(conf=conf)`
	`6`	`+`
`7`	`7`	`lines=sc.textFile("in/word_count.text")`
`8`	`8`	`wordRdd=lines.flatMap(lambdaline:line.split(" "))`
`9`	`9`
`10`	`10`	`wordPairRdd=wordRdd.map(lambdaword: (word,1))`
`11`	`11`	`wordToCountPairs=wordPairRdd.reduceByKey(lambdax,y:x+y)`
`12`	`12`
`13`		`-countToWordParis=wordToCountPairs.map(lambdawordToCount: (wordToCount[1],wordToCount[0]))`
`14`		`-`
`15`		`-sortedCountToWordParis=countToWordParis.sortByKey(ascending=False)`
	`13`	`+sortedWordCountPairs=wordToCountPairs \`
	`14`	`+ .sortBy(lambdawordCount:wordCount[1],ascending=False)`
`16`	`15`
`17`		`-sortedWordToCountPairs=sortedCountToWordParis.map(lambdacountToWord: (countToWord[1],countToWord[0]))`
`18`		`-`
`19`		`-forword,countinsortedWordToCountPairs.collect():`
	`16`	`+forword,countinsortedWordCountPairs.collect():`
`20`	`17`	`print("{} : {}".format(word,count))`
	`18`	`+`

Comments

(0)