- Notifications
You must be signed in to change notification settings - Fork302
Scala to Python - sparkSql folder#2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
| @@ -1,5 +1,5 @@ | |||
| import re | |||
| class Utils(): | |||
| COMMA_DELIMITER = re.compile(''',(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''') | |||
jleetutorialSep 30, 2017 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
why do we need to make this small change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I made some changes to the original one in scala to run in python (because the languages parse regex in a different way), and among those changes I, mistakenly, made the new one not match commas within single quotations too. But this raised some problems when processing the stackoverflow survey. I just removed that.
* Add env/ and .vscode/ to .gitignore* Add IAM requirements.txt* Python Service account keys (jleetutorial#5)Add IAM service account keys snippets* Python Quickstart (jleetutorial#3)Add IAM quickstart* Python Service accounts (jleetutorial#4)Add IAM service account snippets* Add query grantable roles and tests (jleetutorial#2)* Add whitespace and correct string formatting* Add extra CR* Lint
Small change to regex on commons/Utils.py
Converted the scala files on the sparkSql folder to python
Ignored sparkSql/TypedDataset.scala
Changed sparkSql/RddDatasetConversion to sparkSql/RddDataframeConversion