Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

src: implement DataQueue and non-memory resident Blob#45258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
jasnell wants to merge3 commits intonodejs:mainfromjasnell:dataqueue

Conversation

jasnell
Copy link
Member

@jasnelljasnell commentedOct 31, 2022•
edited
Loading

This isthe start of workbeing collaboratively done by myself and@flakey5 to update the implementation ofBlob to support non-memory resident data sources such as files as well as enable proper streaming support.

It is a work in progress and remain draft status until it is ready to go. Opening it for transparency.

Some background:

Currently,Blob objects consist solely of memory-resident data, backed by a collection ofv8::BackingStore instances. While this behavior is correct standard/spec defined behavior, it is extremely inefficient. To support other use cases ofBlob, we want to be able to efficiently and effectively support non-memory resident data sources such as files. To do so will require a number of key changes on the internals of theBlob class.

Further, the current implementation ofBlob suffers from an overly simplistic streaming model where the entireBlob is first read into a singleArrayBuffer before passing the data on to the stream API. Effectively meaningBlobs are never actually streamed.

While going through this evaluation, we came to realization that many of the underlying requirements forBlob are (surprisingly) shared by the internal data management requirements for the QUIC implementation that is in progress, and can enable a range of other use cases also, so we have decided to take an approach that addresses those multiple use cases.

When complete, this PR is going to do two things:

This PR does three things:

  1. Introduces a new C++ data structure calledDataQueue which acts as a sequence of memory-resident and non-memory-resident data sources that can be consumed as a single logical stream of data. This is designed to meet the needs of bothBlob and QUIC initially (the bits relevant to QUIC will come later)
  2. Update the currentBlob implementation to useDataQueue internally in support of the existing all-memory-resident model.
  3. Provides an API for acquiring a File-backedBlob that will support efficient, non-memory resident use cases.

This PR is far from complete. I would strongly recommend that you hold off performing any review on it at all until after we flip the bit to mark it ready for review. We are opening this draft PR now purely for the sake of transparency so that folks can be aware of what we are working on.

I also want to avoid any and all bikeshedding on the design until we at least have the updatedBlob implementation ready to go, and many of the design decisions made will likely not be clear until that concrete case is implemented.

Specifically for Blob, this PRwill enable enables:

  1. Proper streaming support. No longer will theBlob data be collected into a singleArrayBuffer and then passed on to the stream, but data will flow into the stream as one would actually expect.
  2. File-backedBlob. The ability to acquire aBlob that is backed by an on-disc file. As well as the ability to use suchBlob instances efficiently within otherBlob instances, e.g.new Blob(['string', fileBackedBlob])

For QUIC, eventually theDataQueue implemented here will replace theStream::Queue mechanism implemented in that PR.DataQueue will not completely replace that mechanism but covers a sizable chunk of it, which should further simplify and reduce the size of that rather complex bit of work.

As for other use cases ofDataQueue... we'll address those later, but we do have a range of cases in mind (including the recent discussions around Node.js providing an efficient http static file server).

/cc@mcollina (who prompted this work due to Undici and Fetch use case requirements)

theanarkh, bricss, benjamingr, jimmywarting, and styfle reacted with thumbs up emojitonivj5, mcollina, bricss, RaisinTen, jimmywarting, and metcoder95 reacted with heart emojironag, KhafraDev, flakey5, bricss, RaisinTen, jimmywarting, gfortaine, styfle, PositiveGrid-it, and hikarunoryoma reacted with rocket emojibricss, devsnek, and metcoder95 reacted with eyes emoji
@nodejs-github-botnodejs-github-bot added buildIssues and PRs related to build files or the CI. c++Issues and PRs that require attention from people who are familiar with C++. needs-ciPRs that need a full CI run. labelsOct 31, 2022
@jasnell
Copy link
MemberAuthor

@mcollina ... this is ready for review!

flakey5 reacted with hooray emoji

@mcollina
Copy link
Member

cc@KhafraDev

KhafraDev reacted with thumbs up emoji

Copy link
Member

@mcollinamcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

lgtm

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@jasnelljasnellforce-pushed thedataqueue branch 2 times, most recently from9b037f1 to18ba98eCompareDecember 19, 2022 23:56
@nodejs-github-bot

This comment was marked as outdated.

@jasnelljasnell added semver-minorPRs that contain new features and should be released in the next minor version. experimentalIssues and PRs related to experimental features. labelsDec 20, 2022
@nodejs-github-bot
Copy link
Collaborator

addaleax
addaleax previously requested changesDec 21, 2022
@jasnell
Copy link
MemberAuthor

Moving this back to Draft status. Trying to work through@addaleax's feedback and wanted to make it so that theFdEntry Reader is usingFileHandle as aStreamBase but for some reason I'm having a difficult time getting it to work to perform parallel reads of the same file with two separate fds. Will move back out of draft status once I've got it working.

addaleax reacted with thumbs up emoji

@jasnelljasnell marked this pull request as draftDecember 22, 2022 21:02
@juanarbol
Copy link
Member

Would you mind to backporting this to v18.x?

targos pushed a commit that referenced this pull requestMar 13, 2023
See documentation in dataqueue/queue.h for detailsCo-authored-by: flakey5 <73616808+flakey5@users.noreply.github.com>PR-URL:#45258Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos pushed a commit that referenced this pull requestMar 13, 2023
Co-authored-by: flakey5 <73616808+flakey5@users.noreply.github.com>PR-URL:#45258Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos pushed a commit that referenced this pull requestMar 13, 2023
Co-authored-by: flakey5 <73616808+flakey5@users.noreply.github.com>PR-URL:#45258Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos added a commit that referenced this pull requestMar 14, 2023
Notable changes:buffer:  * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell)#46500doc:  * add marco-ippolito to collaborators (Marco Ippolito)#46816events:  * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna)#46523lib:  * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5)#46387src:  * (SEMVER-MINOR) add `fs.openAsBlob` to support File-backed Blobs (James M Snell)#45258tls:  * (SEMVER-MINOR) support automatic DHE (Tobias Nießen)#46978url:  * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell)#46308wasi:  * (SEMVER-MINOR) add support for version when creating WASI (Michael Dawson)#46469worker:  * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee)#46832PR-URL:#47086
targos added a commit that referenced this pull requestMar 14, 2023
Notable changes:buffer:  * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell)#46500doc:  * add marco-ippolito to collaborators (Marco Ippolito)#46816events:  * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna)#46523lib:  * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5)#46387src:  * (SEMVER-MINOR) add `fs.openAsBlob` to support File-backed Blobs (James M Snell)#45258tls:  * (SEMVER-MINOR) support automatic DHE (Tobias Nießen)#46978url:  * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell)#46308wasi:  * (SEMVER-MINOR) add support for version when creating WASI (Michael Dawson)#46469worker:  * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee)#46832PR-URL:#47087
@targostargos mentioned this pull requestMar 14, 2023
targos added a commit that referenced this pull requestMar 14, 2023
Notable changes:buffer:  * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell)#46500doc:  * add marco-ippolito to collaborators (Marco Ippolito)#46816events:  * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna)#46523lib:  * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5)#46387src:  * (SEMVER-MINOR) add `fs.openAsBlob` to support File-backed Blobs (James M Snell)#45258tls:  * (SEMVER-MINOR) support automatic DHE (Tobias Nießen)#46978url:  * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell)#46308wasi:  * (SEMVER-MINOR) add support for version when creating WASI (Michael Dawson)#46469worker:  * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee)#46832PR-URL:#47087
targos added a commit that referenced this pull requestMar 14, 2023
Notable changes:buffer:  * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell)#46500doc:  * add marco-ippolito to collaborators (Marco Ippolito)#46816events:  * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna)#46523lib:  * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5)#46387src:  * (SEMVER-MINOR) add `fs.openAsBlob` to support File-backed Blobs (James M Snell)#45258tls:  * (SEMVER-MINOR) support automatic DHE (Tobias Nießen)#46978url:  * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell)#46308wasi:  * (SEMVER-MINOR) add support for version when creating WASI (Michael Dawson)#46469worker:  * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee)#46832PR-URL:#47087
@jimmywarting
Copy link

I notice that blobs transfered over workers aren't readable at all.

  • it only applies to blobs coming fromfs.openAsBlob
  • in-memory constructed blobs works fine.

mini repro:

import{Worker}from'worker_threads'newWorker('./worker.js').once('message',blob=>blob.text())
// worker.jsimport{parentPort}from'node:worker_threads'import{openAsBlob}from'node:fs'constblob=awaitopenAsBlob(import.meta.url.slice(7))// file:// doesn't workparentPort.postMessage(blob)

I would assume that if you also closed the worker it would still work just fine to read the blob, even if it came from another thread...

Error stacktrace
FATAL ERROR: v8::HandleScope::CreateHandle() Cannot create a handle without a HandleScope 1: 0x104e1df7c node::Abort() [/opt/homebrew/Cellar/node/19.8.1/bin/node] 2: 0x104e1f33c node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/19.8.1/bin/node] 3: 0x104f7a064 v8::Utils::ReportApiFailure(char const*, char const*) [/opt/homebrew/Cellar/node/19.8.1/bin/node] 4: 0x1050cb130 v8::internal::HandleScope::Extend(v8::internal::Isolate*) [/opt/homebrew/Cellar/node/19.8.1/bin/node] 5: 0x104f7bf04 v8::EscapableHandleScope::EscapableHandleScope(v8::Isolate*) [/opt/homebrew/Cellar/node/19.8.1/bin/node] 6: 0x104f98978 v8::ObjectTemplate::NewInstance(v8::Local<v8::Context>) [/opt/homebrew/Cellar/node/19.8.1/bin/node] 7: 0x104e241dc node::fs::FileHandle::New(node::fs::BindingData*, int, v8::Local<v8::Object>, std::__1::optional<long long>, std::__1::optional<long long>) [/opt/homebrew/Cellar/node/19.8.1/bin/node] 8: 0x104d869ec node::(anonymous namespace)::FdEntry::get_reader() [/opt/homebrew/Cellar/node/19.8.1/bin/node] 9: 0x104d847b8 node::(anonymous namespace)::IdempotentDataQueueReader::Pull(std::__1::function<void (int, node::DataQueue::Vec const*, unsigned long, std::__1::function<void (unsigned long)>)>, int, node::DataQueue::Vec*, unsigned long, unsigned long) [/opt/homebrew/Cellar/node/19.8.1/bin/node]10: 0x104d86518 node::(anonymous namespace)::DataQueueEntry::ReaderImpl::Pull(std::__1::function<void (int, node::DataQueue::Vec const*, unsigned long, std::__1::function<void (unsigned long)>)>, int, node::DataQueue::Vec*, unsigned long, unsigned long) [/opt/homebrew/Cellar/node/19.8.1/bin/node]11: 0x104d84928 node::(anonymous namespace)::IdempotentDataQueueReader::Pull(std::__1::function<void (int, node::DataQueue::Vec const*, unsigned long, std::__1::function<void (unsigned long)>)>, int, node::DataQueue::Vec*, unsigned long, unsigned long) [/opt/homebrew/Cellar/node/19.8.1/bin/node]12: 0x104defcf8 node::Blob::Reader::Pull(v8::FunctionCallbackInfo<v8::Value> const&) [/opt/homebrew/Cellar/node/19.8.1/bin/node]13: 0x104fd8a58 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/opt/homebrew/Cellar/node/19.8.1/bin/node]14: 0x104fd8858 v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, unsigned long*, int) [/opt/homebrew/Cellar/node/19.8.1/bin/node]15: 0x104fd82b8 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]16: 0x104c733ec Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/opt/homebrew/Cellar/node/19.8.1/bin/node]17: 0x104bf0064 Builtins_InterpreterEntryTrampoline [/opt/homebrew/Cellar/node/19.8.1/bin/node]18: 0x104bf0064 Builtins_InterpreterEntryTrampoline [/opt/homebrew/Cellar/node/19.8.1/bin/node]19: 0x104bf0064 Builtins_InterpreterEntryTrampoline [/opt/homebrew/Cellar/node/19.8.1/bin/node]20: 0x104bf0064 Builtins_InterpreterEntryTrampoline [/opt/homebrew/Cellar/node/19.8.1/bin/node]21: 0x104c28ab0 Builtins_GeneratorPrototypeNext [/opt/homebrew/Cellar/node/19.8.1/bin/node]22: 0x104bee4f0 Builtins_JSEntryTrampoline [/opt/homebrew/Cellar/node/19.8.1/bin/node]23: 0x104bee184 Builtins_JSEntry [/opt/homebrew/Cellar/node/19.8.1/bin/node]24: 0x10509aa90 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/19.8.1/bin/node]25: 0x10509b1a8 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/19.8.1/bin/node]26: 0x1053a0c8c v8::internal::SourceTextModule::ExecuteModule(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SourceTextModule>) [/opt/homebrew/Cellar/node/19.8.1/bin/node]27: 0x1053a067c v8::internal::SourceTextModule::InnerModuleEvaluation(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SourceTextModule>, v8::internal::ZoneForwardList<v8::internal::Handle<v8::internal::SourceTextModule> >*, unsigned int*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]28: 0x1053a03e0 v8::internal::SourceTextModule::InnerModuleEvaluation(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SourceTextModule>, v8::internal::ZoneForwardList<v8::internal::Handle<v8::internal::SourceTextModule> >*, unsigned int*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]29: 0x10539ff6c v8::internal::SourceTextModule::Evaluate(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SourceTextModule>) [/opt/homebrew/Cellar/node/19.8.1/bin/node]30: 0x105355240 v8::internal::Module::Evaluate(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Module>) [/opt/homebrew/Cellar/node/19.8.1/bin/node]31: 0x104f849f8 v8::Module::Evaluate(v8::Local<v8::Context>) [/opt/homebrew/Cellar/node/19.8.1/bin/node]32: 0x104de085c node::loader::ModuleWrap::Evaluate(v8::FunctionCallbackInfo<v8::Value> const&) [/opt/homebrew/Cellar/node/19.8.1/bin/node]33: 0x104fd8a58 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/opt/homebrew/Cellar/node/19.8.1/bin/node]34: 0x104fd8858 v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, unsigned long*, int) [/opt/homebrew/Cellar/node/19.8.1/bin/node]35: 0x104fd82b8 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]36: 0x104c733ec Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/opt/homebrew/Cellar/node/19.8.1/bin/node]37: 0x104bf0064 Builtins_InterpreterEntryTrampoline [/opt/homebrew/Cellar/node/19.8.1/bin/node]38: 0x104c268b4 Builtins_AsyncFunctionAwaitResolveClosure [/opt/homebrew/Cellar/node/19.8.1/bin/node]39: 0x104cc6e38 Builtins_PromiseFulfillReactionJob [/opt/homebrew/Cellar/node/19.8.1/bin/node]40: 0x104c16834 Builtins_RunMicrotasks [/opt/homebrew/Cellar/node/19.8.1/bin/node]41: 0x104bee3c4 Builtins_JSRunMicrotasksEntry [/opt/homebrew/Cellar/node/19.8.1/bin/node]42: 0x10509aa60 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/19.8.1/bin/node]43: 0x10509b1a8 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/19.8.1/bin/node]44: 0x1050bdc7c v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]45: 0x1050bdaac v8::internal::MicrotaskQueue::PerformCheckpointInternal(v8::Isolate*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]46: 0x104d50ae8 node::InternalCallbackScope::Close() [/opt/homebrew/Cellar/node/19.8.1/bin/node]47: 0x104e25864 node::fs::FileHandle::CloseReq::Resolve() [/opt/homebrew/Cellar/node/19.8.1/bin/node]48: 0x104e26304 node::fs::FileHandle::ClosePromise()::$_0::__invoke(uv_fs_s*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]49: 0x104e182b4 node::MakeLibuvRequestCallback<uv_fs_s, void (*)(uv_fs_s*)>::Wrapper(uv_fs_s*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]50: 0x10784aff8 uv__work_done [/opt/homebrew/Cellar/libuv/1.44.2/lib/libuv.1.dylib]51: 0x10784e3d0 uv__async_io [/opt/homebrew/Cellar/libuv/1.44.2/lib/libuv.1.dylib]52: 0x10785e1e0 uv__io_poll [/opt/homebrew/Cellar/libuv/1.44.2/lib/libuv.1.dylib]53: 0x10784e7d0 uv_run [/opt/homebrew/Cellar/libuv/1.44.2/lib/libuv.1.dylib]54: 0x104d518a8 node::SpinEventLoopInternal(node::Environment*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]55: 0x104e60068 node::NodeMainInstance::Run(node::ExitCode*, node::Environment*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]56: 0x104e5fde8 node::NodeMainInstance::Run() [/opt/homebrew/Cellar/node/19.8.1/bin/node]57: 0x104de6654 node::LoadSnapshotDataAndRun(node::SnapshotData const**, node::InitializationResultImpl const*) [/opt/homebrew/Cellar/node/19.8.1/bin/node]58: 0x104de67d8 node::Start(int, char**) [/opt/homebrew/Cellar/node/19.8.1/bin/node]59: 0x1a2cd7e50 start [/usr/lib/dyld]

@jasnell
Copy link
MemberAuthor

Oops, yeah just a bug. Will take a look in the next day or so.

@jimmywartingjimmywarting mentioned this pull requestMar 20, 2023
@targos
Copy link
Member

I also noticed that the new method is onfs, notfs.promises, which is weird since it returns a promise.

codebytere added a commit to electron/electron that referenced this pull requestNov 6, 2023
codebytere added a commit to electron/electron that referenced this pull requestNov 14, 2023
codebytere added a commit to electron/electron that referenced this pull requestNov 15, 2023
codebytere added a commit to electron/electron that referenced this pull requestNov 16, 2023
codebytere added a commit to electron/electron that referenced this pull requestNov 21, 2023
codebytere added a commit to electron/electron that referenced this pull requestNov 22, 2023
codebytere added a commit to electron/electron that referenced this pull requestNov 28, 2023
codebytere added a commit to electron/electron that referenced this pull requestNov 29, 2023
jkleinsc pushed a commit to electron/electron that referenced this pull requestNov 30, 2023
* chore: upgrade to Node.js v20* src: allow embedders to override NODE_MODULE_VERSIONnodejs/node#49279* src: fix missing trailing ,nodejs/node#46909* src,tools: initialize cppgcnodejs/node#45704* tools: allow passing absolute path of config.gypi in js2cnodejs/node#49162* tools: port js2c.py to C++nodejs/node#46997* doc,lib: disambiguate the old term, NativeModulenodejs/node#45673* chore: fixup Node.js BSSL tests*nodejs/node#49492*nodejs/node#44498* deps: upgrade to libuv 1.45.0nodejs/node#48078* deps: update V8 to 10.7nodejs/node#44741* test: use gcUntil() in test-v8-serialize-leaknodejs/node#49168* module: make CJS load from ESM loadernodejs/node#47999* src: make BuiltinLoader threadsafe and non-globalnodejs/node#45942* chore: address changes to CJS/ESM loading* module: make CJS load from ESM loader (nodejs/node#47999)* lib: improve esm resolve performance (nodejs/node#46652)* bootstrap: optimize modules loaded in the built-in snapshotnodejs/node#45849* test: mark test-runner-output as flakynodejs/node#49854* lib: lazy-load deps in modules/run_main.jsnodejs/node#45849* url: use private properties for brand checknodejs/node#46904* test: refactor `test-node-output-errors`nodejs/node#48992* assert: deprecate callTrackernodejs/node#47740* src: cast v8::Object::GetInternalField() return value to v8::Valuenodejs/node#48943* test: adapt test-v8-stats for V8 updatenodejs/node#45230* tls: ensure TLS Sockets are closed if the underlying wrapclosesnodejs/node#49327* test: deflake test-tls-socket-closenodejs/node#49575* net: fix crash due to simultaneous close/shutdown on JS Stream Socketsnodejs/node#49400* net: use asserts in JS Socket Stream to catch races in futurenodejs/node#49400* lib: fix BroadcastChannel initialization locationnodejs/node#46864* src: create BaseObject with node::Realmnodejs/node#44348* src: implement DataQueue and non-memory resident Blobnodejs/node#45258* sea: add support for V8 bytecode-only cachingnodejs/node#48191* chore: fixup patch indices* gyp: put filenames in variablesnodejs/node#46965* build: modify js2c.py into GN executable* fix: (WIP) handle string replacement of fs -> original-fs* [v20.x] backport vm-related memoryfixesnodejs/node#49874* src: make BuiltinLoader threadsafe and non-globalnodejs/node#45942* src: avoid copying string in fs_permissionnodejs/node#47746* look upon my works ye mightyand dispair* chore: patch cleanup* [api] Remove AllCan Read/Writehttps://chromium-review.googlesource.com/c/v8/v8/+/5006387* fix: missing include for NODE_EXTERN* chore: fixup patch indices* fix: fail properly when js2c fails in Node.js* build: fix js2c root_gen_dir* fix: lib/fs.js -> lib/original-fs.js* build: fix original-fs file xforms* fixup! module: make CJS load from ESM loader* build: get rid of CppHeap for now* build: add patch to prevent extra fs lookup on esm load* build: greatly simplify js2c modificationsMoves our original-fs modifications back into a super simple python script action, wires up the output of that action into our call to js2c* chore: update to handle moved internal/modules/helpers file* test: update @types/node test* feat: enable preventing cppgc heap creation* feat: optionally prevent calling V8::EnableWebAssemblyTrapHandler* fix: no cppgc initialization in the renderer* gyp: put filenames in variablesnodejs/node#46965* test: disable single executable tests* fix: nan tests failing on node headers missing file* tls,http2: send fatal alert on ALPN mismatchnodejs/node#44031* test: disable snapshot tests*nodejs/node#47887*nodejs/node#49684*nodejs/node#44193* build: use deps/v8 for v8/toolsNode.js hard depends on these in their builtins* test: fix edge snapshot stack tracesnodejs/node#49659* build: remove js2c //base dep* build: use electron_js2c_toolchain to build node_js2c* fix: don't create SafeSet outside packageResolveFixes failure in parallel/test-require-delete-array-iterator:=== release test-require-delete-array-iterator ===Path: parallel/test-require-delete-array-iteratornode:internal/per_context/primordials:426    constructor(i) { super(i); } // eslint-disable-line no-useless-constructor                     ^TypeError: object is not iterable (cannot read property Symbol(Symbol.iterator))    at new Set (<anonymous>)    at new SafeSet (node:internal/per_context/primordials:426:22)* fix: failing crashReporter tests on LinuxThese were failing because our change from node::InitializeNodeWithArgs tonode::InitializeOncePerProcess meant that we now inadvertently calledPlatformInit, which reset signal handling. This meant that our intentionalcrash function ElectronBindings::Crash no longer worked and the renderer processno longer crashed when process.crash() was called. We don't want to use Node.js'default signal handling in the renderer process, so we disable it by passingkNoDefaultSignalHandling to node::InitializeOncePerProcess.* build: only create cppgc heap on non-32 bit platforms* chore: clean up util:CompileAndCall* src: fix compatility with upcoming V8 12.1 APIsnodejs/node#50709* fix: use thread_local BuiltinLoader* chore: fixup v8 patch indices---------Co-authored-by: Keeley Hammond <vertedinde@electronjs.org>Co-authored-by: Samuel Attard <marshallofsound@electronjs.org>
MrHuangJser pushed a commit to MrHuangJser/electron that referenced this pull requestDec 11, 2023
* chore: upgrade to Node.js v20* src: allow embedders to override NODE_MODULE_VERSIONnodejs/node#49279* src: fix missing trailing ,nodejs/node#46909* src,tools: initialize cppgcnodejs/node#45704* tools: allow passing absolute path of config.gypi in js2cnodejs/node#49162* tools: port js2c.py to C++nodejs/node#46997* doc,lib: disambiguate the old term, NativeModulenodejs/node#45673* chore: fixup Node.js BSSL tests*nodejs/node#49492*nodejs/node#44498* deps: upgrade to libuv 1.45.0nodejs/node#48078* deps: update V8 to 10.7nodejs/node#44741* test: use gcUntil() in test-v8-serialize-leaknodejs/node#49168* module: make CJS load from ESM loadernodejs/node#47999* src: make BuiltinLoader threadsafe and non-globalnodejs/node#45942* chore: address changes to CJS/ESM loading* module: make CJS load from ESM loader (nodejs/node#47999)* lib: improve esm resolve performance (nodejs/node#46652)* bootstrap: optimize modules loaded in the built-in snapshotnodejs/node#45849* test: mark test-runner-output as flakynodejs/node#49854* lib: lazy-load deps in modules/run_main.jsnodejs/node#45849* url: use private properties for brand checknodejs/node#46904* test: refactor `test-node-output-errors`nodejs/node#48992* assert: deprecate callTrackernodejs/node#47740* src: cast v8::Object::GetInternalField() return value to v8::Valuenodejs/node#48943* test: adapt test-v8-stats for V8 updatenodejs/node#45230* tls: ensure TLS Sockets are closed if the underlying wrapclosesnodejs/node#49327* test: deflake test-tls-socket-closenodejs/node#49575* net: fix crash due to simultaneous close/shutdown on JS Stream Socketsnodejs/node#49400* net: use asserts in JS Socket Stream to catch races in futurenodejs/node#49400* lib: fix BroadcastChannel initialization locationnodejs/node#46864* src: create BaseObject with node::Realmnodejs/node#44348* src: implement DataQueue and non-memory resident Blobnodejs/node#45258* sea: add support for V8 bytecode-only cachingnodejs/node#48191* chore: fixup patch indices* gyp: put filenames in variablesnodejs/node#46965* build: modify js2c.py into GN executable* fix: (WIP) handle string replacement of fs -> original-fs* [v20.x] backport vm-related memoryfixesnodejs/node#49874* src: make BuiltinLoader threadsafe and non-globalnodejs/node#45942* src: avoid copying string in fs_permissionnodejs/node#47746* look upon my works ye mightyand dispair* chore: patch cleanup* [api] Remove AllCan Read/Writehttps://chromium-review.googlesource.com/c/v8/v8/+/5006387* fix: missing include for NODE_EXTERN* chore: fixup patch indices* fix: fail properly when js2c fails in Node.js* build: fix js2c root_gen_dir* fix: lib/fs.js -> lib/original-fs.js* build: fix original-fs file xforms* fixup! module: make CJS load from ESM loader* build: get rid of CppHeap for now* build: add patch to prevent extra fs lookup on esm load* build: greatly simplify js2c modificationsMoves our original-fs modifications back into a super simple python script action, wires up the output of that action into our call to js2c* chore: update to handle moved internal/modules/helpers file* test: update @types/node test* feat: enable preventing cppgc heap creation* feat: optionally prevent calling V8::EnableWebAssemblyTrapHandler* fix: no cppgc initialization in the renderer* gyp: put filenames in variablesnodejs/node#46965* test: disable single executable tests* fix: nan tests failing on node headers missing file* tls,http2: send fatal alert on ALPN mismatchnodejs/node#44031* test: disable snapshot tests*nodejs/node#47887*nodejs/node#49684*nodejs/node#44193* build: use deps/v8 for v8/toolsNode.js hard depends on these in their builtins* test: fix edge snapshot stack tracesnodejs/node#49659* build: remove js2c //base dep* build: use electron_js2c_toolchain to build node_js2c* fix: don't create SafeSet outside packageResolveFixes failure in parallel/test-require-delete-array-iterator:=== release test-require-delete-array-iterator ===Path: parallel/test-require-delete-array-iteratornode:internal/per_context/primordials:426    constructor(i) { super(i); } // eslint-disable-line no-useless-constructor                     ^TypeError: object is not iterable (cannot read property Symbol(Symbol.iterator))    at new Set (<anonymous>)    at new SafeSet (node:internal/per_context/primordials:426:22)* fix: failing crashReporter tests on LinuxThese were failing because our change from node::InitializeNodeWithArgs tonode::InitializeOncePerProcess meant that we now inadvertently calledPlatformInit, which reset signal handling. This meant that our intentionalcrash function ElectronBindings::Crash no longer worked and the renderer processno longer crashed when process.crash() was called. We don't want to use Node.js'default signal handling in the renderer process, so we disable it by passingkNoDefaultSignalHandling to node::InitializeOncePerProcess.* build: only create cppgc heap on non-32 bit platforms* chore: clean up util:CompileAndCall* src: fix compatility with upcoming V8 12.1 APIsnodejs/node#50709* fix: use thread_local BuiltinLoader* chore: fixup v8 patch indices---------Co-authored-by: Keeley Hammond <vertedinde@electronjs.org>Co-authored-by: Samuel Attard <marshallofsound@electronjs.org>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@addaleaxaddaleaxaddaleax left review comments

@jimmywartingjimmywartingjimmywarting left review comments

@mcollinamcollinamcollina approved these changes

Assignees
No one assigned
Labels
buildIssues and PRs related to build files or the CI.c++Issues and PRs that require attention from people who are familiar with C++.experimentalIssues and PRs related to experimental features.needs-ciPRs that need a full CI run.semver-minorPRs that contain new features and should be released in the next minor version.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

7 participants
@jasnell@mcollina@nodejs-github-bot@addaleax@juanarbol@jimmywarting@targos
[8]ページ先頭
©2009-2025 Movatter.jp