Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork136
extend simple peer (not peerjs, oops) to handle buffered/packet transmission; add raw dependency w/MIT license#25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:master
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
dmonad left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hey@disarticulate, thanks so much for this!
I like the approach that you have taken. But I'm a bit hesitant to release a new major release of y-webrtc with these breaking changes. However, I think it should be possible for us to make this change non-breaking by making it optional whether we adapt your polyfill (or another one which might fix other issues).
| encodePacket({ chunk, txOrd, index, length, totalSize, chunkSize}){ | ||
| constencoded=concatenate(Uint8Array,[ | ||
| newUint8Array(newInt64BE(txOrd).toArrayBuffer()),// 8 bytes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I guess the reason why you don't use BigUint64Array is that it is not yet supported in Safari.
I developed an efficient encoder exactly for this problem:https://github.com/dmonad/lib0/blob/main/encoding.js
import*asencodingfrom'lib0/encoding'import*asdecodingfrom'lib0/decoding'// example of encoding unsigned integers and strings efficientlyconstencoder=newencoding.createEncoder()encoding.writeVarUint(encoder,256)encoding.writeVarString(encoder,'Hello world!')constbuf=encoding.toUint8Array(encoder)constdecoder=newdecoding.createDecoder(buf)decoding.readVarUint(decoder)// => 256decoding.readVarString(decoder)// => 'Hello world!'decoding.hasContent(decoder)// => false - all data is read
The documentation for other encoding techniques is here:https://github.com/dmonad/lib0
But I realize that this is already working in the current state. But maybe we can avoid pulling in more dependencies that are superflous.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
For a header, we need to pad the values. Note the// 8 bytes comments I've mocked up a replacement class like:
class Int64 { constuctor (int64) { this.encoder = new encoding.createEncoder() encoding.writeVarUint(encoder, int64) } toArrayBuffer () { return encoding.toUint8Array(this.encoder) }}However, when I test the recommended dependency, we get:
...encoding.writeVarUint(encoder, 1)encoding.toUint8Array(encoder)> Uint8Array(1) [ 1 ]its not obvious to me how to do pad/unpad safely, so the whole decode/encode would need a rewrite
| importPeerfrom'simple-peer/simplepeer.min.js' | ||
| // import Peer from 'simple-peer/simplepeer.min.js' | ||
| importPeerfrom'./SimplePeerExtended' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This is interesting. So you implemented a layer around simple-peer that handles this issue.
Would it be possible that you publish a separate package that we can include as a polyfill for the default implementation? This is already done in y-websocket where we define theWebSocket as an argument.
newWebsocketProvider(URL,room,{WebSocket:MyCustomWebsocketPolyfill})
We could do something similar to y-webrtc without breaking the existing API.
newWebrtcProvider(room,yjs,{SimplePeer:SimplePeerExtended})
I'd prefer that approach because it allows us to test out your implementation before breaking everyone else's existing deployments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
i can publish a module and put something up on github, but it seems like you'd need move SimplePeer to a peerDependency to avoid duplicating code, otherwise your library is pulling in Peer while your user is also pulling in a modified peer.
Also, it looks likeWebrtcConn is what's using thePeer and that's being called fromSignalingConn...etc...
I can't figure out how you'd thread that option into the proper slot because it call comes from a globalimport Peer from 'simple-peer/simplepeer.min.js'
datakurre commentedApr 13, 2022
@disarticulate Is this patch the latest version you've been applying? I see errors when clients disconnect. Those seem to go away, if I include Beyond that, a supported way to apply customized SimplePeer would be great 🙏. |
disarticulate commentedApr 28, 2022
Yes. I haven't been developing with it recently, and haven't followed any yjs updates since this was proposed. checkhttps://github.com/disarticulate/y-webrtc for when it was developed. it looks like simple-peer would need be moved to a peer dependency. I think the behavior is stable, but how it should be integrated takes some considerations that I haven't returned to. |
disarticulate commentedJul 30, 2022 via email
The boring answer is the encoder selected is encoding 64 bit integers.The arbitrary answer is it made the header simple to decode and encode.Someone could rightsize it all, some of those values are purely redundant.I considered a system that did packet routing so wanted space. …On Thu, Jul 28, 2022, 17:14 Vitor de Mello Freitas ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/SimplePeerExtended.js <#25 (comment)>: > +} + +class SimplePeerExtended extends Peer { + constructor (opts) { + super(opts) + this._opts = opts + this._txOrdinal = 0 + this._rxPackets = [] + this._txPause = false + this.webRTCMessageQueue = [] + this.webRTCPaused = false + } + + encodePacket ({ chunk, txOrd, index, length, totalSize, chunkSize }) { + const encoded = concatenate(Uint8Array, [ + new Uint8Array(new Int64BE(txOrd).toArrayBuffer()), // 8 bytes@disarticulate <https://github.com/disarticulate> Why do the values need to be padded to 8 bytes in the header? — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFHWPLDSXMKA2FL3GAULI3VWMA2TANCNFSM45JCHYTA> . You are receiving this because you were mentioned.Message ID: ***@***.***> |
KiddoV commentedJan 14, 2025
Any update on this PR? |
Closes#20
sendand_onChannelMessagefunctions to handle chunked data packets using custom defined packetsencodePacketanddecodePacketThe custom packet setup could probably be simplfied as we're counting each full data (
txOrd), each packet sent (index), how many packets (length), the size of the packet's data (chunkSize) and the total size of the data (totalSize)Ran the tests and get some typescript errors that come from the
SimplePeerExtended extends Peerwhere we're using something from the parent class that isnt available. Also, the minified dependency would either need to be brought in officially, or ignored.