- Notifications
You must be signed in to change notification settings - Fork4
Closed
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 2323fc8513c9f5e60683cf4aa90475b44a550170 |
| Compare | 390c2d5df324139c716c12bedf5a8dee737fe994 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dequeue_And_Enqueue - Duration of single invocation | 12.05 μs | 7.97 μs | 0.66 | 0.01 | True | |||||
| HeapSort - Duration of single invocation | 4.39 μs | 2.78 μs | 0.63 | 0.00 | True | |||||
| Dequeue_And_Enqueue - Duration of single invocation | 490.23 ns | 454.29 ns | 0.93 | 0.00 | True | |||||
| K_Max_Elements - Duration of single invocation | 665.21 ns | 589.63 ns | 0.89 | 0.00 | True | |||||
| K_Max_Elements - Duration of single invocation | 4.73 μs | 4.48 μs | 0.95 | 0.00 | True |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>*'
Payloads
Histogram
System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 100)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 7.967306827457535 < 11.285030517377674.IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 79.24799807705467 (T) = (0 -8208.800240700797) / Math.Sqrt((16040.23127028316 / (299)) + (28011.830016928216 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.3043881463031513 = (11800.834326032396 - 8208.800240700797) / 11800.834326032396 is greater than 0.05.IsChangeEdgeDetector: Marked as regression because Edge Detector said so.System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 100)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 2.7799610742020078 < 4.1676825576928795.IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 247.2804970894706 (T) = (0 -2788.688024534845) / Math.Sqrt((2776.4642724456958 / (299)) + (477.75856837926835 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.3687829206193311 = (4417.954006046575 - 2788.688024534845) / 4417.954006046575 is greater than 0.05.IsChangeEdgeDetector: Marked as regression because Edge Detector said so.System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 10)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 454.2908128265757 < 466.1408138899135.IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 93.35836937079901 (T) = (0 -454.9508879329665) / Math.Sqrt((13.097931810189205 / (299)) + (1.6400022506649259 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07606324827629488 = (492.40479619866386 - 454.9508879329665) / 492.40479619866386 is greater than 0.05.IsChangeEdgeDetector: Marked as regression because Edge Detector said so.System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 100)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 589.6269140477093 < 628.7773395634234.IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 48.097793582561245 (T) = (0 -591.3952234804864) / Math.Sqrt((11.553800882896383 / (299)) + (32.31142185587683 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.11078334627915937 = (665.0743899204435 - 591.3952234804864) / 665.0743899204435 is greater than 0.05.IsChangeEdgeDetector: Marked as regression because Edge Detector said so.System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 1000)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 4.4836960122478375 < 4.494138835129538.IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 112.11511632845693 (T) = (0 -4476.280416549036) / Math.Sqrt((591.3992045047202 / (299)) + (49.30084490487852 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.055477556478664036 = (4739.199631785055 - 4476.280416549036) / 4739.199631785055 is greater than 0.05.IsChangeEdgeDetector: Marked as regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 2323fc8513c9f5e60683cf4aa90475b44a550170 |
| Compare | 390c2d5df324139c716c12bedf5a8dee737fe994 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Decimal
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Subtract - Duration of single invocation | 8.08 ns | 6.97 ns | 0.86 | 0.41 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'
Payloads
Histogram
System.Tests.Perf_Decimal.Subtract
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 6.973876518930619 < 7.72427474858055.IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 3/31/2023 8:29:50 AM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 9.771103730352266 (T) = (0 -7.037252816928418) / Math.Sqrt((0.37319453509596706 / (299)) + (0.20821520253132236 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.14987072466703819 = (8.277861992427217 - 7.037252816928418) / 8.277861992427217 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 2323fc8513c9f5e60683cf4aa90475b44a550170 |
| Compare | 390c2d5df324139c716c12bedf5a8dee737fe994 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Matrix4x4
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| InvertBenchmark - Duration of single invocation | 41.17 ns | 36.46 ns | 0.89 | 0.01 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.gitpython3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'
Payloads
Histogram
System.Numerics.Tests.Perf_Matrix4x4.InvertBenchmark
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.IsImprovementWindowed: Marked as improvement because 36.462682827576614 < 38.82669603002287.IsChangePoint: Marked as a change because one of 1/6/2023 8:18:17 PM, 3/31/2023 8:29:50 AM, 4/5/2023 11:58:06 PM falls between 3/28/2023 10:24:48 AM and 4/5/2023 11:58:06 PM.IsImprovementStdDev: Marked as improvement because 63.00520862135902 (T) = (0 -36.25487066782566) / Math.Sqrt((0.21554010716325858 / (299)) + (0.06460429670049198 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.112640768140949 = (40.85703891519823 - 36.25487066782566) / 40.85703891519823 is greater than 0.05.IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
_1.png&f=jpg&w=240)
_2.png&f=jpg&w=240)
_3.png&f=jpg&w=240)
_4.png&f=jpg&w=240)

