NotificationsYou must be signed in to change notification settings
Fork7.7k
Star48k

Improve variable type inference#19830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

iSazonov merged 28 commits intoPowerShell:masterfromMartinGC94:ImproveVarInference

Mar 13, 2025

Merged

Improve variable type inference#19830

iSazonov merged 28 commits intoPowerShell:masterfromMartinGC94:ImproveVarInference

Mar 13, 2025

Conversation

Copy link

Contributor

MartinGC94 commentedJun 20, 2023•
edited
Loading

PR Summary

Improves the type inference of variables in the following ways:

Infer type from "ref" assignments, like:[System.Management.Automation.Language.Parser]::ParseFile("ls", [ref] $Tokens, [ref] $null)
Infer type from the *Variable common parameters, like:New-Guid -WarningVariable MyVar
Infer type for variables set by New/Set-Variable
Infer type of variables set with the Data statement
More accurate inference of variables assigned with the PipelineVariable common parameter (Now it correctly limits the type inference to the pipeline scope)
Fix issue where the type of parameters and foreach statement variables were inferred based on the first assignment, rather than the closest
Fix type inference crash when the inferred variable had a scope assigned, like:[AstTypeInference]::InferTypeOf({$Global:true}.Ast)
Infer type of variables assigned with an attribute:[ValidateNotNull()]$Var1 = "Hello"
Infer type of variables assigned with multiple type constraints:[int] [string] $Var1 = "10"
Infers type of variables assigned with redirection:New-Guid 1>variable:RedirectedVar; $RedirectedVar

PR Context

Fixes#18759
Fixes#24205

PR Checklist

PR has a meaningful title
- Use the present tense and imperative mood when describing your changes
Summarized changes
Make sure all.h,.cpp,.cs,.ps1 and.psm1 files have the correct copyright header
This PR is ready to merge and is notWork in Progress.
- If the PR is work in progress, please add the prefixWIP: or[ WIP ] to the beginning of the title (theWIP bot will keep its status check atPending while the prefix is present) and remove the prefix when the PR is ready.
Breaking changes
- None
- OR
- Experimental feature(s) needed
  - Experimental feature name(s):
User-facing changes
- Not Applicable
- OR
- Documentation needed
  - Issue filed:
Testing - New and feature
- N/A or can only be tested interactively
- OR
- Make sure you've added a new test if existing tests do not effectively test the code changed
Tooling
- I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
- OR
- I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
  - Impact onPowerShell Editor Services which is used in thePowerShell extension for VSCode
    (which runs in a different PS Host).
    - Issue filed:
  - Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
    - Issue filed:
  - Impact onPSScriptAnalyzer (which provides linting & formatting in the editor extensions).
    - Issue filed:
  - Impact onEditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
    - Issue filed:

Improve variable type inference

c91003c

MartinGC94 requested a review fromdaxian-dbw as acode owner

June 20, 2023 22:05

pull-request-quantifier-deprecatedbot added the Extra Large label

Jun 20, 2023

ghost added the Review - NeededThe PR is being reviewed label

Jun 28, 2023

Copy link

ghost commentedJun 28, 2023

This pull request has been automatically marked as Review Needed because it has been there has not been any activity for7 days.
Maintainer, please provide feedback and/or mark it asWaiting on Author

StevenBucher98 added the PowerShell-Docs not neededThe PR was reviewed and doesn't appear to require a PowerShell Docs update label

Jul 10, 2023

Merge branch 'PowerShell:master' into ImproveVarInference

691cd63

Copy link

pull-request-quantifier-deprecatedbot commentedJan 23, 2024

This PR has405 quantified lines of changes. In general, a change size of upto200 lines is ideal for the best PR experience!

Quantification details

Label      : Extra LargeSize       : +289 -116Percentile : 80.17%Total files changed: 3Change summary by file extension:.cs : +229 -116.ps1 : +60 -0

Change counts above are quantified counts, based on thePullRequestQuantifier customizations.

Why proper sizing of changes matters

Optimal pull request sizes drive a better predictable PR flow as they strike a
balance between between PR complexity and PR review overhead. PRs within the
optimal size (typical small, or medium sized PRs) mean:

Fast and predictable releases to production:
- Optimal size changes are more likely to be reviewed faster with fewer
  iterations.
- Similarity in low PR complexity drives similar review times.
Review quality is likely higher as complexity is lower:
- Bugs are more likely to be detected.
- Code inconsistencies are more likely to be detected.
Knowledge sharing is improved within the participants:
- Small portions can be assimilated better.
Better engineering practices are exercised:
- Solving big problems by dividing them in well contained, smaller problems.
- Exercising separation of concerns within the code changes.

What can I do to optimize my changes

Use the PullRequestQuantifier to quantify your PR accurately
- Create a context profile for your repo using thecontext generator
- Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out theExcluded section from yourprquantifier.yaml context profile.
- Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in yourprquantifier.yaml context profile.
- Only use the labels that matter to you,see context specification to customize yourprquantifier.yaml context profile.
Change your engineering behaviors
- For PRs that fall outside of the desired spectrum, review the details and check if:
  - Your PR could be split in smaller, self-contained PRs instead
  - Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

One line was added:+1 -0
One line was deleted:+0 -1
One line was modified:+1 -1 (git diff doesn't know about modified, it will
interpret that line like one addition plus one deletion)
Change percentiles: Change characteristics (addition, deletion, modification)
of this PR in relation to all other PRs within the repository.

Was this comment helpful?👍 :ok_hand: :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.

MartinGC94 mentioned this pull request

Jan 26, 2024

Improve assignment type inference#21143

Merged

22 tasks

Merge branch 'PowerShell:master' into ImproveVarInference

e833446

Copy link

pull-request-quantifier-deprecatedbot commentedFeb 10, 2024

This PR has405 quantified lines of changes. In general, a change size of upto200 lines is ideal for the best PR experience!

Quantification details

Label      : Extra LargeSize       : +289 -116Percentile : 80.17%Total files changed: 3Change summary by file extension:.cs : +229 -116.ps1 : +60 -0

Change counts above are quantified counts, based on thePullRequestQuantifier customizations.

Why proper sizing of changes matters

Fast and predictable releases to production:
- Optimal size changes are more likely to be reviewed faster with fewer
  iterations.
- Similarity in low PR complexity drives similar review times.
Review quality is likely higher as complexity is lower:
- Bugs are more likely to be detected.
- Code inconsistencies are more likely to be detected.
Knowledge sharing is improved within the participants:
- Small portions can be assimilated better.
Better engineering practices are exercised:
- Solving big problems by dividing them in well contained, smaller problems.
- Exercising separation of concerns within the code changes.

What can I do to optimize my changes

Use the PullRequestQuantifier to quantify your PR accurately
- Create a context profile for your repo using thecontext generator
- Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out theExcluded section from yourprquantifier.yaml context profile.
- Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in yourprquantifier.yaml context profile.
- Only use the labels that matter to you,see context specification to customize yourprquantifier.yaml context profile.
Change your engineering behaviors
- For PRs that fall outside of the desired spectrum, review the details and check if:
  - Your PR could be split in smaller, self-contained PRs instead
  - Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

One line was added:+1 -0
One line was deleted:+0 -1
One line was modified:+1 -1 (git diff doesn't know about modified, it will
interpret that line like one addition plus one deletion)
Change percentiles: Change characteristics (addition, deletion, modification)
of this PR in relation to all other PRs within the repository.

Was this comment helpful?👍 :ok_hand: :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.

MartinGC94 added2 commits

April 17, 2024 17:32

Merge branch 'PowerShell:master' into ImproveVarInference

a5356b7

Merge branch 'PowerShell:master' into ImproveVarInference

e57665a

MartinGC94 mentioned this pull request

Aug 25, 2024

Completion type inference in foreach item infers from the first declaration in a document rather than the iterated collection#24205

Closed

5 tasks

MartinGC94 added2 commits

September 7, 2024 19:59

Merge branch 'PowerShell:master' into ImproveVarInference

9220460

Merge branch 'master' into ImproveVarInference

69688d2

MartinGC94 requested a review fromSeeminglyScience as acode owner

February 18, 2025 01:37

Copy link

Collaborator

iSazonov commentedFeb 18, 2025

@MartinGC94 Is it possible to split the PR to some small ones? This would significantly speed up the review.

microsoft-github-policy-servicebot removed the Review - NeededThe PR is being reviewed label

Feb 18, 2025

Copy link

ContributorAuthor

MartinGC94 commentedFeb 18, 2025

Not in a meaningful way. It's a new Ast visitor so we kinda need the whole thing for it to make sense.

Merge branch 'master' into ImproveVarInference

80ef9b0

This comment was marked as outdated.

This comment was marked as outdated.

iSazonov added the CL-GeneralIndicates that a PR should be marked as a general cmdlet change in the Change Log label

Feb 18, 2025

MartinGC94and others added2 commits

February 19, 2025 15:51

Merge branch 'PowerShell:master' into ImproveVarInference

09db4e3

Handle type inference for attributed assignments

e2a4a57

iSazonov reviewed

Feb 19, 2025

View reviewed changes

src/System.Management.Automation/engine/parser/TypeInferenceVisitor.cs OutdatedShow resolvedHide resolved

Replace "var" with actual type

4630c8c

This comment was marked as outdated.

iSazonov reviewed

Mar 4, 2025

View reviewed changes

test/powershell/engine/Api/TypeInference.Tests.ps1 OutdatedShow resolvedHide resolved

MartinGC94and others added2 commits

March 4, 2025 18:56

Add missing test

abe50cb

Update test/powershell/engine/Api/TypeInference.Tests.ps1

df646e0

Co-authored-by: Ilya <darpa@yandex.ru>

This comment was marked as outdated.

iSazonov self-assigned this

Mar 5, 2025

iSazonov reviewed

Mar 5, 2025

View reviewed changes

src/System.Management.Automation/engine/parser/TypeInferenceVisitor.csShow resolvedHide resolved

src/System.Management.Automation/engine/parser/TypeInferenceVisitor.cs OutdatedShow resolvedHide resolved

Apply suggestions from code review

24d76b8

Co-authored-by: Ilya <darpa@yandex.ru>

iSazonov reviewed

Mar 6, 2025

View reviewed changes

src/System.Management.Automation/engine/parser/TypeInferenceVisitor.cs OutdatedShow resolvedHide resolved

src/System.Management.Automation/engine/parser/TypeInferenceVisitor.cs

		// The condition itself may not be interesting if it's after the cursor, but the statement block could be.
		return ast is PipelineBaseAst && ast.Parent is DoUntilStatementAst or DoWhileStatementAst
		? AstVisitAction.SkipChildren
		: AstVisitAction.StopVisit;

Copy link

Collaborator

iSazonovMar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I wonder why not use VisitDoWhileStatement/VisitDoUntilStatement method directly? It would be more clear.

If SkipChildren is for condition - it is clear that we skip it, what is StopVisit for?

Copy link

ContributorAuthor

MartinGC94Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It's not possible to do it from those methods because we can only tell it to continue, skip children, or stop completely. We want to skip the condition part of the loop (if it's after the variable being inferred) but we still want to process the body.

Copy link

Collaborator

iSazonovMar 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

My idea is that if we come in the methods we can return StopVisit but before directly call VisitStatementBlock for StatementBlockAst from Body.

Copy link

ContributorAuthor

MartinGC94Mar 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

That doesn't work either. When calling VisitStatementBlock it will hit DefaultVisit which would return "Continue". Then it'd move on to the condition and hit the DefaultVisit again, and this time it will return "StopVisit" because it's after the cursor.

Copy link

Collaborator

iSazonovMar 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Then it'd move on to the condition

If we run forBody it cannot move toCondition.

Copy link

ContributorAuthor

MartinGC94Mar 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I tried it on a local build and it behaved the way I described.

Copy link

Collaborator

iSazonovMar 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

After further thought, I believe I can take this as a step forward. Although I am inclined to believe that a direct bypass would be more convenient, simple and extensible. I'm even inclined to think that this was originally intended.

src/System.Management.Automation/engine/parser/TypeInferenceVisitor.csShow resolvedHide resolved

MartinGC94 added4 commits

March 6, 2025 16:09

Address feedback

2e2eec1

Merge conflict

86328b7

Add missing assignment

4aa929c

Fix another "as" statement

8216056

This comment was marked as outdated.

iSazonov reviewed

Mar 8, 2025

View reviewed changes

test/powershell/engine/Api/TypeInference.Tests.ps1

Comment on lines +1521 to +1534

		It 'Ignores type constraint defined outside of scope' {
		$res = [AstTypeInference]::InferTypeOf(({
		function Outer
		{
		[string] $Test = "Hello"
		function Inner
		{
		$Test = 2
		$Test
		}
		}
		}.Ast.FindAll({param($Ast) $Ast -is [Language.VariableExpressionAst]}, $true) \| Select-Object -Last 1 ))
		$res.Name \| Should -Be 'System.Int32'
		}

Copy link

Collaborator

iSazonovMar 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Will the inference work after follow swap?

It 'Ignores typeconstraint definedoutside ofscope'{        $res=[AstTypeInference]::InferTypeOf(({function Outer{                function Inner{                    $Test=2                    $Test}[string] $Test="Hello"                Inner}        ...        $res.Name|Should-Be 'System.Int32'}

Copy link

ContributorAuthor

MartinGC94Mar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Yes,$Test inside theInner function will also be inferred as an int32 in this scenario because there's an assignment insideInner so outer scope type constraints/assignments don't matter.

Copy link

Collaborator

iSazonovMar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

And if we remove $Test = 2 the inference will stop?

Copy link

ContributorAuthor

MartinGC94Mar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Yes. It would stop the search before it would reach the assignment because it's after the start of theInner scope. This is not accurate to how PowerShell works, but trying to keep track of the variable values before each command invocation is quite difficult.