- Notifications
You must be signed in to change notification settings - Fork664
Scaffold IPC-based API#711
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Uh oh!
There was an error while loading.Please reload this page.
Merged
Changes from68 commits
Commits
Show all changes
71 commits Select commitHold shift + click to select a range
af278cc Example working
andrewbranch2ffb88e Refactor API into own package
andrewbranchc563027 WIP
andrewbranch8867ecb WIP
andrewbranchb6c10fe WIP
andrewbranch8ccc215 WIP
andrewbranch75191c1 Switch to new libsyncrpc protocol
andrewbranch04b0181 Switch to integer project ids
andrewbranch386fff2 Custom binary encoding for symbols and types
andrewbranch29b04b6 Revert "Custom binary encoding for symbols and types"
andrewbranch057c6cb Use pnpm
andrewbranchf932949 Split AST and API packages
andrewbranche65d5d5 Clean up
andrewbranch5819b1a Add client benchmarks
andrewbranch8257e8e Batch symbol requests more efficiently
andrewbranchdb7ce0f Switch to byte indexes in encoder
andrewbranch8131c9b Transfer source file text
andrewbranchdb1d249 Merge branch 'main' into api
andrewbranch518a5b0 Delete async API for now
andrewbranch56d7e67 More pnpm things after merge
andrewbranchbd92afa Benchmark client-hosted file system
andrewbranch7a07c5c Implement extended data table
andrewbranche454aeb Document binary encoding
andrewbranch9602b07 Include Strada for comparison in benchmarks
andrewbranch9c9c255 Support client-side object equality
andrewbranche60c60f Add tests to hereby/CI
andrewbranch4ceae39 Merge branch 'main' into api
andrewbranch7a8005c Delete unused TextLength tracking
andrewbranch15776e4 Delete DS_Store
andrewbranchdf0968d Fix parseConfigFile and update libsyncrpc
andrewbranchc880885 Add tests for getting symbols and types
andrewbranchde52e45 Wrap protocol in MessagePack
andrewbranch8130058 Switch back to npm
andrewbranch44328e7 Use extended node data for SourceFile, include fileName property
andrewbranch5ce3cb7 Switch to explicit-only resource management
andrewbranchf01b3e1 Change batching, add node id handling
andrewbranch39121ea Fix getSymbolAtLocation, materializing children, etc
andrewbranch2eefe62 Merge branch 'main' into api
andrewbranch17c625b Format
andrewbranch66fe030 Add vfs.FS.Remove to api.Server
andrewbranch8c785bd Add libsyncrpc as a git dependency
andrewbranchf9655ab Add encoder baseline
andrewbranchd985ac2 Type check JS before testing, use --experimental-strip-types
andrewbranchd76c2f6 Kind is 16-bit
andrewbranchef3fe0f lint
andrewbranch3630c83 Add dependencies to test:all
andrewbranch866518c Fix @typescript/ast exports
andrewbranch7fd8f5f Explicitly install rust toolchain on runners
andrewbranchb4a48fe Add .exe to tsserverPath on Windows
andrewbranche9099b5 Fix windows paths
andrewbranch5bcf558 Update benchmarks
andrewbranch4f87998 Split hereby test:all
andrewbranch8f36dc0 Add Rust toolchain to README
andrewbranch8c31046 Delete DS_Store
andrewbranch5580a3a Move env to job level
andrewbranch81d1574 Preserve same CI testing order
andrewbranch4fbc66a Update _packages/api/src/api.ts
andrewbranch9c25263 Fix overload implementation signature
andrewbranchdf6851f Merge branch 'main' into api
andrewbranche1d90df Merge branch 'main' into api
andrewbranchefc981f Use atomic id getter functions
andrewbranch9f8dffc tsconfig hierarchy
andrewbranchf5d829f Bring back hereby test:all
andrewbranch072d696 Use constants for handle prefixes
andrewbranch15600c8 Move benchmarks into tests, run single iteration as part of tests
andrewbranch5efde45 Skip benchmarks when submodule isn’t cloned
andrewbranch9274c93 Merge branch 'main' into api
andrewbrancha7d15d0 Merge branch 'main' into api
andrewbrancha212d0e Merge branch 'main' into api
andrewbranchc93f782 Install rust on merge-queue workflow
andrewbranchae749fb Fix test:all dependencies
andrewbranchFile filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
There are no files selected for viewing
19 changes: 18 additions & 1 deletion.github/workflows/ci.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
19 changes: 19 additions & 0 deletionsHerebyfile.mjs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -314,18 +314,37 @@ async function runTestTools() { | ||
| await $test({ cwd: path.join(__dirname, "_tools") })`${gotestsum("tools")} ./...`; | ||
| } | ||
| async function runTestAPI() { | ||
| await $`npm run -w @typescript/api test`; | ||
| } | ||
| export const testTools = task({ | ||
| name: "test:tools", | ||
| run: runTestTools, | ||
| }); | ||
| export const buildAPITests = task({ | ||
| name: "build:api:test", | ||
| run: async () => { | ||
| await $`npm run -w @typescript/api build:test`; | ||
| }, | ||
| }); | ||
| export const testAPI = task({ | ||
| name: "test:api", | ||
| dependencies: [tsgo, buildAPITests], | ||
| run: runTestAPI, | ||
| }); | ||
| export const testAll = task({ | ||
| name: "test:all", | ||
andrewbranch marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| dependencies: [buildAPITests], | ||
| run: async () => { | ||
| // Prevent interleaving by running these directly instead of in parallel. | ||
| await runTests(); | ||
| await runTestBenchmarks(); | ||
| await runTestTools(); | ||
| await runTestAPI(); | ||
| }, | ||
| }); | ||
2 changes: 1 addition & 1 deletionREADME.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
51 changes: 51 additions & 0 deletions_packages/api/package.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| { | ||
| "private": true, | ||
| "name": "@typescript/api", | ||
| "version": "1.0.0", | ||
| "type": "module", | ||
| "imports": { | ||
| "#symbolFlags": { | ||
| "@typescript/source": { | ||
| "types": "./src/symbolFlags.enum.ts", | ||
| "default": "./src/symbolFlags.ts" | ||
| }, | ||
| "types": "./dist/symbolFlags.enum.d.ts", | ||
| "default": "./dist/symbolFlags.js" | ||
| }, | ||
| "#typeFlags": { | ||
| "@typescript/source": { | ||
| "types": "./src/typeFlags.enum.ts", | ||
| "default": "./src/typeFlags.ts" | ||
| }, | ||
| "types": "./dist/typeFlags.enum.d.ts", | ||
| "default": "./dist/typeFlags.js" | ||
| } | ||
| }, | ||
| "exports": { | ||
| ".": { | ||
| "@typescript/source": "./src/api.ts", | ||
| "default": "./dist/api.js" | ||
| }, | ||
| "./fs": { | ||
| "@typescript/source": "./src/fs.ts", | ||
| "default": "./dist/fs.js" | ||
| }, | ||
| "./proto": { | ||
| "@typescript/source": "./src/proto.ts", | ||
| "default": "./dist/proto.js" | ||
| } | ||
| }, | ||
| "scripts": { | ||
| "build": "tsc -b", | ||
| "build:test": "tsc -b test", | ||
| "bench": "node --experimental-strip-types --no-warnings --conditions @typescript/source test/api.bench.ts", | ||
| "test": "node --test --experimental-strip-types --no-warnings --conditions @typescript/source ./test/**/*.test.ts" | ||
| }, | ||
| "devDependencies": { | ||
| "tinybench": "^3.1.1" | ||
| }, | ||
| "dependencies": { | ||
| "@typescript/ast": "1.0.0", | ||
| "libsyncrpc": "github:microsoft/libsyncrpc#bb02d84" | ||
| } | ||
| } |
179 changes: 179 additions & 0 deletions_packages/api/src/api.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,179 @@ | ||
| /// <reference path="./node.ts" preserve="true" /> | ||
| import { SymbolFlags } from "#symbolFlags"; | ||
| import { TypeFlags } from "#typeFlags"; | ||
| import type { | ||
| Node, | ||
| SourceFile, | ||
| } from "@typescript/ast"; | ||
| import { Client } from "./client.ts"; | ||
| import type { FileSystem } from "./fs.ts"; | ||
| import { RemoteSourceFile } from "./node.ts"; | ||
| import { ObjectRegistry } from "./objectRegistry.ts"; | ||
| import type { | ||
| ConfigResponse, | ||
| ProjectResponse, | ||
| SymbolResponse, | ||
| TypeResponse, | ||
| } from "./proto.ts"; | ||
| export { SymbolFlags, TypeFlags }; | ||
| export interface APIOptions { | ||
| tsserverPath: string; | ||
| cwd?: string; | ||
| logFile?: string; | ||
| fs?: FileSystem; | ||
| } | ||
| export class API { | ||
| private client: Client; | ||
| private objectRegistry: ObjectRegistry; | ||
| constructor(options: APIOptions) { | ||
| this.client = new Client(options); | ||
| this.objectRegistry = new ObjectRegistry(this.client); | ||
| } | ||
| parseConfigFile(fileName: string): ConfigResponse { | ||
| return this.client.request("parseConfigFile", { fileName }); | ||
| } | ||
| loadProject(configFileName: string): Project { | ||
| const data = this.client.request("loadProject", { configFileName }); | ||
| return this.objectRegistry.getProject(data); | ||
| } | ||
| echo(message: string): string { | ||
| return this.client.echo(message); | ||
| } | ||
| echoBinary(message: Uint8Array): Uint8Array { | ||
| return this.client.echoBinary(message); | ||
| } | ||
| close(): void { | ||
| this.client.close(); | ||
| } | ||
| } | ||
| export class DisposableObject { | ||
| private disposed: boolean = false; | ||
| protected objectRegistry: ObjectRegistry; | ||
| constructor(objectRegistry: ObjectRegistry) { | ||
| this.objectRegistry = objectRegistry; | ||
| } | ||
| [globalThis.Symbol.dispose](): void { | ||
| this.objectRegistry.release(this); | ||
| this.disposed = true; | ||
| } | ||
| dispose(): void { | ||
| this[globalThis.Symbol.dispose](); | ||
| } | ||
| isDisposed(): boolean { | ||
| return this.disposed; | ||
| } | ||
| ensureNotDisposed(): this { | ||
| if (this.disposed) { | ||
| throw new Error(`${this.constructor.name} is disposed`); | ||
| } | ||
| return this; | ||
| } | ||
| } | ||
| export class Project extends DisposableObject { | ||
| private decoder = new TextDecoder(); | ||
| private client: Client; | ||
| id: string; | ||
| configFileName!: string; | ||
| compilerOptions!: Record<string, unknown>; | ||
| rootFiles!: readonly string[]; | ||
| constructor(client: Client, objectRegistry: ObjectRegistry, data: ProjectResponse) { | ||
| super(objectRegistry); | ||
| this.id = data.id; | ||
| this.client = client; | ||
| this.loadData(data); | ||
| } | ||
| loadData(data: ProjectResponse): void { | ||
| this.configFileName = data.configFileName; | ||
| this.compilerOptions = data.compilerOptions; | ||
| this.rootFiles = data.rootFiles; | ||
| } | ||
| reload(): void { | ||
| this.ensureNotDisposed(); | ||
| this.loadData(this.client.request("loadProject", { configFileName: this.configFileName })); | ||
| } | ||
| getSourceFile(fileName: string): SourceFile | undefined { | ||
| this.ensureNotDisposed(); | ||
| const data = this.client.requestBinary("getSourceFile", { project: this.id, fileName }); | ||
| return data ? new RemoteSourceFile(data, this.decoder) as unknown as SourceFile : undefined; | ||
| } | ||
| getSymbolAtLocation(node: Node): Symbol | undefined; | ||
| getSymbolAtLocation(nodes: readonly Node[]): (Symbol | undefined)[]; | ||
| getSymbolAtLocation(nodeOrNodes: Node | readonly Node[]): Symbol | (Symbol | undefined)[] | undefined { | ||
| this.ensureNotDisposed(); | ||
| if (Array.isArray(nodeOrNodes)) { | ||
| const data = this.client.request("getSymbolsAtLocations", { project: this.id, locations: nodeOrNodes.map(node => node.id) }); | ||
| return data.map((d: SymbolResponse | null) => d ? this.objectRegistry.getSymbol(d) : undefined); | ||
| } | ||
| const data = this.client.request("getSymbolAtLocation", { project: this.id, location: (nodeOrNodes as Node).id }); | ||
| return data ? this.objectRegistry.getSymbol(data) : undefined; | ||
| } | ||
| getSymbolAtPosition(fileName: string, position: number): Symbol | undefined; | ||
| getSymbolAtPosition(fileName: string, positions: readonly number[]): (Symbol | undefined)[]; | ||
| getSymbolAtPosition(fileName: string, positionOrPositions: number | readonly number[]): Symbol | (Symbol | undefined)[] | undefined { | ||
| this.ensureNotDisposed(); | ||
| if (typeof positionOrPositions === "number") { | ||
| const data = this.client.request("getSymbolAtPosition", { project: this.id, fileName, position: positionOrPositions }); | ||
| return data ? this.objectRegistry.getSymbol(data) : undefined; | ||
| } | ||
| const data = this.client.request("getSymbolsAtPositions", { project: this.id, fileName, positions: positionOrPositions }); | ||
| return data.map((d: SymbolResponse | null) => d ? this.objectRegistry.getSymbol(d) : undefined); | ||
| } | ||
| getTypeOfSymbol(symbol: Symbol): Type | undefined; | ||
| getTypeOfSymbol(symbols: readonly Symbol[]): (Type | undefined)[]; | ||
| getTypeOfSymbol(symbolOrSymbols: Symbol | readonly Symbol[]): Type | (Type | undefined)[] | undefined { | ||
| this.ensureNotDisposed(); | ||
| if (Array.isArray(symbolOrSymbols)) { | ||
| const data = this.client.request("getTypesOfSymbols", { project: this.id, symbols: symbolOrSymbols.map(symbol => symbol.ensureNotDisposed().id) }); | ||
| return data.map((d: TypeResponse | null) => d ? this.objectRegistry.getType(d) : undefined); | ||
| } | ||
| const data = this.client.request("getTypeOfSymbol", { project: this.id, symbol: (symbolOrSymbols as Symbol).ensureNotDisposed().id }); | ||
| return data ? this.objectRegistry.getType(data) : undefined; | ||
| } | ||
| } | ||
| export class Symbol extends DisposableObject { | ||
| private client: Client; | ||
| id: string; | ||
| name: string; | ||
| flags: SymbolFlags; | ||
| checkFlags: number; | ||
| constructor(client: Client, objectRegistry: ObjectRegistry, data: SymbolResponse) { | ||
| super(objectRegistry); | ||
| this.client = client; | ||
| this.id = data.id; | ||
| this.name = data.name; | ||
| this.flags = data.flags; | ||
| this.checkFlags = data.checkFlags; | ||
| } | ||
| } | ||
| export class Type extends DisposableObject { | ||
| private client: Client; | ||
| id: string; | ||
| flags: TypeFlags; | ||
| constructor(client: Client, objectRegistry: ObjectRegistry, data: TypeResponse) { | ||
| super(objectRegistry); | ||
| this.client = client; | ||
| this.id = data.id; | ||
| this.flags = data.flags; | ||
| } | ||
| } |
Oops, something went wrong.
Uh oh!
There was an error while loading.Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.