- Notifications
You must be signed in to change notification settings - Fork380
Support java.util.StringJoiner#3396
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
spamegg1 commentedJul 18, 2023
- ported from Scala.jsFix #4872: Implement java.util.StringJoiner. scala-js/scala-js#4873
- fixesSupport java.util.StringJoiner #3355
- ported from Scala.jsscala-js/scala-js#4873
LeeTibbert commentedJul 18, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Thank you for submitting this PR. The "lint" failure can be corrected by running scalafmt at the command line. At the project root "scripts/scalafmt", Doc built sucessfully, so that is a good thing. |
| @@ -0,0 +1,52 @@ | |||
| // Ported from Scala.js commit: 57d71da dated: 2023-05-31 | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thank you, this origin & commit info is quite useful to future developers trying to keep things up to date.
| } | ||
| @TestdeftestNPE():Unit= { | ||
| assumeTrue("requires compliant null pointers",Platform.hasCompliantNullPointers) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Different developers and/or reviewers may well give different suggestions. Wojciech is the lead developer and gatekeeper of the merge, so that is the most valid opinion.
By my understanding, aided by@armanbilge, both JVM and Scala Native have compliant null pointer checks.
My practical suggestion is to comment out line#1736 add a /* */ comment before it stating that both Scala JVM & Scala Native have compliant pointers (giving the url from Arman:https://discord.com/channels/632150470000902164/635668881951686686/1130847953062473738) and stating
that the commented Scala.js line is left as a reference point to the Scala.js original.
LeeTibbertJul 18, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
A full fix would be to edit both
./unit-tests/jvm/src/test/scala/org/scalanative/testsuite/utils/Platform.scala./unit-tests/native/src/test/scala/org/scalanative/testsuite/utils/Platform.scalaand add a line such as (cribbed from Scala.js JVM Platform.scala)def hasCompliantNullPointers: Boolean = true Again, posting the reference URL as justification in the Scala Native case would be good.
Since thehasCompliant is unlikely to be used outside of the StringJoinerTest, is to place the declaration at the top of the class, with a comment about both JVM and Scala Native
having compliant null pointers, so defining to allow the
Scala.js code to remain unaltered.
Now that I think of it, I like the third approach as useful but minimal effort and unlikely to trip-up future maintainers.
Your thoughts?
Actually, the third alternative requires a slight edit ofPlatform. to be removed from theassumeTrue.
That makes the include of Platform unnecessary & unused, increasing the edit count.
Dancing in the land of complexity. Arrgh.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
My original approach was to just remove it entirely. But commenting it out (your first suggestion) sounds good.
Almost there, the failures are because The third alternative in one of my review comments suggests defining |
- fix linting- fix `hasCompliantNullPointer` issues, add comments
| @TestdeftestNPE():Unit= { | ||
| /** Both Scala Native and Scala JVM has compliant null pointers. The |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Nice handling of complexity.
A native speaker of English would probably use "have" instead of "has". Not worth a CI run, leave some room for improvement for the next generation.
The content is what is important, IMO.
LeeTibbert commentedJul 18, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Looks good to me (LGTM). Well done@spamegg1 I am going offline now, but will check again this evening to see if CI encountered any problems. |
| if (isEmpty) | ||
| isEmpty=false | ||
| else | ||
| value+= delimiter | ||
| value+= newElement// if newElement is null, adds "null" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Scala.js has used string concat here becouse string are native construct in JS runtime, and this action does not contain performance penalty. However, on JVM or Native we don't have native String primitives and their concatontation is expensive. We should usejava.lang.StringBuilder instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Something like this?
valbuilder= java.lang.StringBuilder(value)if (isEmpty) isEmpty=falseelse builder.append(delimiter)if (newElement==null) builder.append("null")else builder.append(newElement) value= builder.toString()this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
yeah, something like that. Should do the job. We probably don't need to store result in value
- use `StringBuilder` instead of `+`- fix typo
| // `+` is expensive on JVM/Native, use StringBuilder instead. | ||
| valbuilder=new java.lang.StringBuilder(value) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
When having a new string builder for eachadd invocation, its cost would be similar to String.+. Instead we should keep single instance ofStringBuilder that would be used for alladd operations. It should replacevalue: String which would not be required any more. In to string we materialize the curren, joined value usingStringBuilder.toString.
Sorry if I was not clear about it previously.
| defthis( | ||
| delimiter:CharSequence, | ||
| prefix:CharSequence, | ||
| suffix:CharSequence | ||
| )= | ||
| this(delimiter.toString(), prefix.toString(), suffix.toString()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We can use this auxilary constructor as main constructor here. The variant using Strings is a Scala.js specific optimization. We should keep all string-like elements as CharSequence in JVM/Native.
Why to do that? In all this operations we don't really need to materialize a string. By having a CharSequence we can directly use String, StringBuilder, StringBuffer, CharBuffer and others and pass them directly to final StringBuilder for efficent joining.
| /** The current value, excluding prefix and suffix.*/ | ||
| privatevarvalue:String="" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It's works nice in JS, but for native/jvm we can do better. The main operation for adding new string isadd(CharSequence). Upon String materialization intoString we need to add custom prefix and suffix. Instead of constructing intermiediete string we can store parts of it comming fromadd. We can use the fact the List prepend in Scala is very cheap. Here's how I think it could be implemented to more efficient in jvm/native. We have extreamly fast add, and slower materializaiton
privatedefdelimiter, prefix,suffix:CharSequenceprivatedefemptyValue:CharSequence|Nullprivatevarparts:List[CharSequence]=Nildefadd(v:CharSequence)= { parts::= v;this }deftoString()= {if parts.isEmpty&& emptyValue!=nullthen emptyValueelse {valsb=new java.lang.StringBuilder(this.length())// alloc exactly as much buffer as required to skip internal buffer resize sb.append(prefix)valhead:: tail= parts.reverse():@unchecked// we know parts is non-empty, parts are in reverse order sb.append(head) tail.foreach{ part=> sb.append(delimiter).append(part) } sb.append(suffix) sb.toString() }defmerge(other:StringJoiner):StringJoiner= {// take all the parts from other and joinif (!other.isEmpty){ parts= other.parts:::this.parts }this }deflength():Int= {valbaseLength=if(emptyValue!=null) emptyValue.lengthelse prefix.length+ suffix.length()valdelimLength= delimiter.length() parts.foldLeft(baseLength){case (acc, part)=> acc+ delimLength+ part.length } }}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It looks like, withmerge, we have to use the delimiter ofother for parts that come fromother:
scala>valmultiple=StringJoiner(";","[","]").setEmptyValue("--").add("a").add("b").add("c")valmultiple: java.util.StringJoiner= [a;b;c] scala>valjoiner=StringJoiner(",","{","}").add("one").merge(multiple).add("two")valjoiner: java.util.StringJoiner= {one, a;b;c, two}// here a;b;c is using delimiter of multiple, not of joiner
So theparts = other.parts ::: this.parts approach doesn't work (or maybe I'm not seeing something obvious).
What should I do?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Oh, ok, that's oversight from me. So docs state that the other.parts are concatanate using other delimieter, and it should not add prefix/suffix or empty value. So we can do either do 1 one of things here:
- flatMap other.parts to create a list of delimiters and parts and then prepend them to this.parts
- create new StringBuilder (which is also an instance of CharSequence) and materialize the other joiner without prefix/suffix or empty value. For that we can just copy-past code from toString with minor modifications, or even refector them out to common function. I think this solution might be better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think I kinda understand what you're saying.
I'll have to figure out the string length of (other.parts + the delimiters that go between them) for the newStringBuilder, I might also refactor that into another helper function and reuse it inlength().
Just another monkey wrench but you should only use javalibs not Scala but you can use |
spamegg1 commentedJul 21, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
I have an implementation that passes the tests. What can I use in place of |
LeeTibbert commentedJul 21, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
FYI, the reason for the "please do not use scalalib" is to break a dependency where scalalib relies upon For me, most of the time knowing the reasoning for a prohibition makes it easier to live with it. @ekrich Please keep me on the rails here... I also usually use a simple What pattern matching is concerning you. I did a quick look through the |
spamegg1 commentedJul 21, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Considering Wojciech's approach above (relying on prepend
My current implementation changed completely, I haven't pushed the changes yet. I'm using Should I just push it? |
- making suggested changes
@spamegg1 I really should have given more info like Lee has done. I have been doing this awhile so I wasn't really communicating enough information. We try to learn from Scala.js which has removed all the Scala stdlib from their javalib so we are headed that direction. Most of our collection code has examples because much of it has been ported from Scala.js so that is a good resource many times. Here is some code, it has 2 annotations from the Scala stdlib but not much else. This has examples of |
- add missing parentheses
| privatedeflengthOf( | ||
| parts: scala.collection.immutable.List[CharSequence], | ||
| delim:CharSequence | ||
| ):Int= { | ||
| valdelimLength= delim.length | ||
| partsmatch { | ||
| caseNil=>0 | ||
| case head:: tail=> | ||
| tail.foldLeft(head.length)((acc, part)=> | ||
| acc+ delimLength+ part.length | ||
| ) | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
| privatedeflengthOf( | |
| parts: scala.collection.immutable.List[CharSequence], | |
| delim:CharSequence | |
| ):Int= { | |
| valdelimLength= delim.length | |
| partsmatch { | |
| caseNil=>0 | |
| case head:: tail=> | |
| tail.foldLeft(head.length)((acc, part)=> | |
| acc+ delimLength+ part.length | |
| ) | |
| } | |
| } | |
| // at the top | |
| importjava.{util=>ju} | |
| importScalaOps._ | |
| privatedeflengthOf( | |
| parts: ju.List[CharSequence], | |
| delim:CharSequence | |
| ):Int= { | |
| valdelimLength= delim.length | |
| if (parts.length=0)0 | |
| else { | |
| valheadLen= parts(0).length | |
| parts.subList(1, parts.length).ScalaOps.foldLeft(headLen)((acc, part)=> | |
| acc+ delimLength+ part.length | |
| ) | |
| } | |
| } |
This should be close to what you need. Hope this helps.
- greatly simplify implementation using `ScalaOps`
@spamegg1 CI is all green, great. I want you/us to succeed early, especially on a first PR. Without holding up this PR, let me mention a few things for one or more possible PRs. I have heard the expression "Those who propose, dispose." more than
|
- making suggested changes
- fixing parentheses
LeeTibbert commentedJul 23, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Bug reports are a sign that someone is actually using the product of your hard work (I keep telling myself that I tried using the three argument constructor of
I believe this makes the constructor inaccessible (outside the package, including sub-packages.??) Scala.Js declares the primary constructor as private but then has a 1-arg & 3-arg public method (corresponding Ithink that the There must be a good reason why the code of this PR is the way it is. Sorry if I am missing it. PS: Once I hacked a private copy of |
spamegg1 commentedJul 24, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
It's my mistake. I wasn't sure but I thought switching to I'm not sure how to make an alternate constructor now. Should it also take So confused 😕 |
- changing constructors and fields to use `String`, and removing null checks
@spamegg1 I need to apologize. You may be confused because it looks like I gave you advise which I think the part that is tripping me/us up in trying to convert the original .Js code to SN is the |
I have spent several hours on this in the middle of the night and have run out of time. I created a private study version of StringJoiner.scala. That revealed a problem with having I suggest that this PR, assuming it passes CI, be merged and remaining issues be addressed in one or I will check in on Thursday to see where the discussion goes. |
@LeeTibbert In the latest commit I changed the primary (private) constructor to use |
LeeTibbert commentedJul 27, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
This PR gives a good outward facing base level1. The reasonably done is far better than the absent I have a next evolution PR ready to submit after this PR is merged. It uses a It also passes I have not done performance metrics but think it will be correct and have a lower resource cost. This will make a difference when someone wants to Collect a stream of 10_000 elements. I think submitting the file as a follow-on PR will cause less confusion than posting a gist. Footnotes
|
Congratulation on the merge of your first Scala Native PR. Programming/Software_Engineering in the large. Looking forward to your next PR. |
* Support java.util.StringJoiner- ported from Scala.jsscala-js/scala-js#4873(cherry picked from commitb1bba76)
* Support java.util.StringJoiner- ported from Scala.jsscala-js/scala-js#4873(cherry picked from commitb1bba76)