- Notifications
You must be signed in to change notification settings - Fork90
TCP/UDP: new function is_listening: t -> ~port:int -> callback option#508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This is useful for proxies/middleware/interception of requests, a runningexample is let's encrypt and the HTTP challenge.The methodology is as follows:- the unikernel requests (via https from let's encrypt) a challenge and solves it (using a private key, some cryptographic computations)- the let's encrypt server (wants to proof the ownership of the hostname in the certificate signing request) requests via HTTP (port 80) a specific resource (http://example.com/.well-known/acme-challenge/...)- the unikernel needs to properly reply to that challengeNow, one path (that we took until now) is to treat this .well-knwon/acme-challengevery special in any unikernel that we wrote.Another path is to create a let's encrypt http challenge library that takes astack, and whenever it needs it registers itself for port 80, proxyingeverything it is not interested in, to the old handler (thus, is_listening),and serving the .well-known/acme-challenge.Concurrent updates to the "listen" hashtable are dangerous of course, greatcare has to be taken (if some other parts of the application as well re-registerlisteners). But I'm confident since listen, unlisten, and is_listening are pure(not in Lwt monad), it's fine and can be dealt with. Another option would be toimplement a real protocol/locking around the shared global resource of listeningports (but I'd first see whether we run into such troubles).Another example is the let's encrypt ALPN challenge, where the process is as follows:- the unikernel requests (via https from let's encrypt) a challenge and solves it (using a private key, some cryptographic computations)- the let's encrypt server (wants to proof the ownership of the hostname in the signing request) connects via TLS on port 443 with a specific ALPN string- the unikernel needs to reply with a specially craftes self-signed certificateThis can, as above, be implemented by a temporary proxy while the challenge isin process -- without service interruptions for other parties (web browser, ...)
hannesm commentedApr 11, 2023
I added a second function, I'm not convinced this is the right thing to do (though it is very convenient for my use case). The implementation is rather basic (and works fine for my use case, but not for generality - where you may have a task already blocking on I'd like to finish and evaluate the prototype I have before merging this here.. |
| includeTcpip.Tcp.S | ||
| withtype ipaddr=Ipaddr.t | ||
| andtypeflow=Lwt_unix.file_descr |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
any insight whether this is needed somewhere?
| |Someb ->Cstruct.length b | ||
| letadd_lts= | ||
| ignore(Lwt_dllist.add_l (Some s) t.q) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
any idea whether any other things must be updated? I frankly don't understand much of theadd_r below, but it deals with variouscur_size andmax_size.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
also, dot.readers need to be notified?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
cur_size andmax_size seem to be r(elated to a window of available data in the buffer (cur_size is what is currently available andmax_size the bound for available data, but it should be possible to exceed that limit with the linked list data structure).
To keep things going, I think it's best to updatecur_size and callnotify_size_watcher to say that the data is online. Something like (the first comparison inadd_r seems to be there to avoid exceedingmax_size (again), I'm not sure the problem could be anything other than higher memory consumption, but it may be best to take care of that?):
let add_l t s = match Lwt_dllist.take_opt_l t.readers with | None -> t.cur_size <- Int32.(add t.cur_size (of_int (seglen s))); ignore(Lwt_dllist.add_l (Some s) t.q) notify_size_watcher t | Some w -> Lwt.return (Lwt.wakeup u s)| letunreadfdbuf= | ||
| let buf=Cstruct.append buf fd.bufin | ||
| fd.buf<- buf |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
what needs to be handled (for a complete, general API) is if a lwt task is already inLwt_cstruct.read -- where the read should be cancelled and thebuf provided here being returned to the caller.
This is useful for proxies/middleware/interception of requests, a running example is let's encrypt and the HTTP challenge.
The methodology is as follows:
Now, one path (that we took until now) is to treat this .well-knwon/acme-challenge very special in any unikernel that we wrote.
Another path is to create a let's encrypt http challenge library that takes a stack, and whenever it needs it registers itself for port 80, proxying everything it is not interested in, to the old handler (thus, is_listening), and serving the .well-known/acme-challenge.
Concurrent updates to the "listen" hashtable are dangerous of course, great care has to be taken (if some other parts of the application as well re-register listeners). But I'm confident since listen, unlisten, and is_listening are pure (not in Lwt monad), it's fine and can be dealt with. Another option would be to implement a real protocol/locking around the shared global resource of listening ports (but I'd first see whether we run into such troubles).
Another example is the let's encrypt ALPN challenge, where the process is as follows:
This can, as above, be implemented by a temporary proxy while the challenge is in process -- without service interruptions for other parties (web browser, ...)