- Notifications
You must be signed in to change notification settings - Fork4
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 8b1d1eabe32ba781ffcce2867333dfdc53bdd635 |
| Compare | e0c94f84b47fdd48435a625446123f89268b9c20 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Vector4
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AddFunctionBenchmark - Duration of single invocation | 1.48 ns | 0.07 ns | 0.05 | 0.79 | False | |||||
| SquareRootBenchmark - Duration of single invocation | 1.37 ns | 0.01 ns | 0.01 | 0.85 | False | |||||
| AddOperatorBenchmark - Duration of single invocation | 1.58 ns | 0.05 ns | 0.03 | 0.77 | False | |||||
| LengthSquaredBenchmark - Duration of single invocation | 1.91 ns | 0.05 ns | 0.03 | 0.56 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'
Payloads
Histogram
System.Numerics.Tests.Perf_Vector4.AddFunctionBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.07128859655855821 < 1.4135000269061486.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 7.523425909582359 (T) = (0 -0.04142762713608473) / Math.Sqrt((0.35675457239358704 / (299)) + (0.0012865140425237685 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8760547754050163 = (0.3342414140718848 - 0.04142762713608473) / 0.3342414140718848 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector4.SquareRootBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.012016776945270227 < 1.2274759944297544.IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 3.553885472237078 (T) = (0 -0.07451690404570571) / Math.Sqrt((0.3187142661333788 / (299)) + (0.013843166247593734 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.7624017754701529 = (0.31362567709905126 - 0.07451690404570571) / 0.31362567709905126 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector4.AddOperatorBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.05015263512458719 < 0.7119560207878848.IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 8.077905498643204 (T) = (0 -0.033125246813771936) / Math.Sqrt((0.30112160184478726 / (299)) + (0.0007398684792661226 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8938378878455158 = (0.31202512969569574 - 0.033125246813771936) / 0.31202512969569574 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector4.LengthSquaredBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.05377036458562206 < 1.6455587220759984.IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 4/3/2023 6:15:03 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 6.518202478907093 (T) = (0 -0.056152861318788126) / Math.Sqrt((0.5101567274398501 / (299)) + (0.0014711934765790841 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8384966018336937 = (0.34768841991154475 - 0.056152861318788126) / 0.34768841991154475 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 8b1d1eabe32ba781ffcce2867333dfdc53bdd635 |
| Compare | e0c94f84b47fdd48435a625446123f89268b9c20 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Vector2
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| LengthSquaredBenchmark - Duration of single invocation | 1.85 ns | 0.03 ns | 0.02 | 0.64 | False | |||||
| AddOperatorBenchmark - Duration of single invocation | 1.62 ns | 0.11 ns | 0.07 | 0.80 | False | |||||
| SquareRootBenchmark - Duration of single invocation | 1.60 ns | 0.16 ns | 0.10 | 0.69 | False | |||||
| AddFunctionBenchmark - Duration of single invocation | 1.61 ns | 0.05 ns | 0.03 | 0.76 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'
Payloads
Histogram
System.Numerics.Tests.Perf_Vector2.LengthSquaredBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.03080131800536622 < 1.8079028801041945.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 8.446974697686565 (T) = (0 -0.015408674133142342) / Math.Sqrt((0.5471487264371615 / (299)) + (0.0003165698227013366 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9599235141897791 = (0.38448166853024246 - 0.015408674133142342) / 0.38448166853024246 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector2.AddOperatorBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.11188700205000704 < 1.7277761434827401.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 5.635544692976003 (T) = (0 -0.10853212331016368) / Math.Sqrt((0.5228163439553707 / (299)) + (0.003850534474399378 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.7299990652071842 = (0.4019694353778641 - 0.10853212331016368) / 0.4019694353778641 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector2.SquareRootBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.1553015082661257 < 1.7269523112595755.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 5.410216170389057 (T) = (0 -0.043772210662626584) / Math.Sqrt((0.4624767173454978 / (299)) + (0.005550192759029865 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8700499878730028 = (0.3368388347655481 - 0.043772210662626584) / 0.3368388347655481 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector2.AddFunctionBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.05365361835945301 < 1.7358395826421784.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 8.090846211283663 (T) = (0 -0.054926783171732836) / Math.Sqrt((0.46328247183356847 / (299)) + (4.9864769084865775E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8534053423581908 = (0.3746847535599929 - 0.054926783171732836) / 0.3746847535599929 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 8b1d1eabe32ba781ffcce2867333dfdc53bdd635 |
| Compare | e0c94f84b47fdd48435a625446123f89268b9c20 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CountBenchmark - Duration of single invocation | 1.89 ns | 0.01 ns | 0.01 | 0.73 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Byte>*'
Payloads
Histogram
System.Numerics.Tests.Perf_VectorOf<Byte>.CountBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.012496986717106324 < 1.7991680820637201.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 8.777794395759967 (T) = (0 -0.008885081071813176) / Math.Sqrt((0.5981554481953305 / (299)) + (1.3306412702778504E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.977887758169694 = (0.4018172892644326 - 0.008885081071813176) / 0.4018172892644326 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 8b1d1eabe32ba781ffcce2867333dfdc53bdd635 |
| Compare | e0c94f84b47fdd48435a625446123f89268b9c20 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| NewJsonSerializerContext - Duration of single invocation | 14.02 μs | 1.33 μs | 0.10 | 0.29 | True |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>*'
Payloads
Histogram
System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewJsonSerializerContext
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 1.3325636373747811 < 12.580745397833125.IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 277.582718170339 (T) = (0 -1309.3650189924456) / Math.Sqrt((323114.0830337199 / (299)) + (2416.5089126425223 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8969250066213115 = (12703.03277325425 - 1309.3650189924456) / 12703.03277325425 is greater than 0.05.IsChangeEdgeDetector: Marked as regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 8b1d1eabe32ba781ffcce2867333dfdc53bdd635 |
| Compare | e0c94f84b47fdd48435a625446123f89268b9c20 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.MemoryMarshal<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryGetArray - Duration of single invocation | 5.16 ns | 2.53 ns | 0.49 | 0.43 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.MemoryMarshal<Byte>*'
Payloads
Histogram
System.Memory.MemoryMarshal<Byte>.TryGetArray
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 2.53194615643591 < 4.848274230557065.IsChangePoint: Marked as a change because one of 2/1/2023 10:12:25 AM, 2/2/2023 8:18:56 AM, 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 5.103296393437463 (T) = (0 -2.8422068953523594) / Math.Sqrt((0.976699219113439 / (299)) + (0.16420107062493694 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.2743049112761248 = (3.9165304265050778 - 2.8422068953523594) / 3.9165304265050778 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 8b1d1eabe32ba781ffcce2867333dfdc53bdd635 |
| Compare | e0c94f84b47fdd48435a625446123f89268b9c20 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Plane
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| DotCoordinateBenchmark - Duration of single invocation | 2.46 ns | 0.01 ns | 0.01 | 0.75 | False | |||||
| TransformByMatrix4x4Benchmark - Duration of single invocation | 48.82 ns | 42.82 ns | 0.88 | 0.01 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*'
Payloads
Histogram
System.Numerics.Tests.Perf_Plane.DotCoordinateBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.012586381909878867 < 1.5321251133917855.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 9.92558775319431 (T) = (0 -0.05011460113677147) / Math.Sqrt((0.41448723869696075 / (299)) + (0.0006405275026858656 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8862139640088499 = (0.4404283944004272 - 0.05011460113677147) / 0.4404283944004272 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Plane.TransformByMatrix4x4Benchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 42.81789912313856 < 47.23487818672364.IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 3/31/2023 8:29:50 AM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 54.58447919530533 (T) = (0 -42.53085006379626) / Math.Sqrt((5.517113701388062 / (299)) + (0.09773817306135504 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.16990081318741299 = (51.23586523088418 - 42.53085006379626) / 51.23586523088418 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 8b1d1eabe32ba781ffcce2867333dfdc53bdd635 |
| Compare | e0c94f84b47fdd48435a625446123f89268b9c20 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Vector3
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AddFunctionBenchmark - Duration of single invocation | 1.50 ns | 0.06 ns | 0.04 | 0.77 | False | |||||
| LengthSquaredBenchmark - Duration of single invocation | 1.77 ns | 0.07 ns | 0.04 | 0.81 | False | |||||
| SquareRootBenchmark - Duration of single invocation | 2.23 ns | 0.06 ns | 0.03 | 0.80 | False | |||||
| AddOperatorBenchmark - Duration of single invocation | 1.75 ns | 0.00 ns | 0.00 | 0.81 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'
Payloads
Histogram
System.Numerics.Tests.Perf_Vector3.AddFunctionBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.061821709136556095 < 1.5300671336047953.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 9.155077803232874 (T) = (0 -0.03429372000774648) / Math.Sqrt((0.415080862167176 / (299)) + (0.0006106858764703142 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9128871588470522 = (0.3936700898956508 - 0.03429372000774648) / 0.3936700898956508 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector3.LengthSquaredBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.0701497024355996 < 1.6335517535776272.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 8.464499370285884 (T) = (0 -0.032566634226230476) / Math.Sqrt((0.4385468887075337 / (299)) + (0.001018287237184963 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9151346589379846 = (0.38374481052792053 - 0.032566634226230476) / 0.38374481052792053 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.0630964058679144 < 1.5047961827524299.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 7.800212543467743 (T) = (0 -0.06495422693421984) / Math.Sqrt((0.39599323180932233 / (299)) + (6.660704573829131E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8147341358585088 = (0.35060008078235616 - 0.06495422693421984) / 0.35060008078235616 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.System.Numerics.Tests.Perf_Vector3.AddOperatorBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 0.0003596367865697028 < 1.6413366484671834.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 10.547049286560318 (T) = (0 -0.005523188981689868) / Math.Sqrt((0.4819315408884285 / (299)) + (3.871256918344225E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9871622060173676 = (0.43022882195818707 - 0.005523188981689868) / 0.43022882195818707 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 8b1d1eabe32ba781ffcce2867333dfdc53bdd635 |
| Compare | e0c94f84b47fdd48435a625446123f89268b9c20 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Random
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Next_unseeded - Duration of single invocation | 7.58 ns | 4.54 ns | 0.60 | 0.38 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'
Payloads
Histogram
System.Tests.Perf_Random.Next_unseeded
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 4.541093357519626 < 7.435160538382805.IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 7.507126190320924 (T) = (0 -4.430162019919515) / Math.Sqrt((2.388827441377345 / (299)) + (0.024130969447175233 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.16712485098196575 = (5.319118988173327 - 4.430162019919515) / 5.319118988173327 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository








_1.png&f=jpg&w=240)
_1.png&f=jpg&w=240)
_1.png&f=jpg&w=240)






