- Notifications
You must be signed in to change notification settings - Fork4
Closed
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2af0051c894aac4d1a9d6479ad6dd7547a688363 |
| Compare | 8db3cab69e3afe8c25d295edb8b1d599518e4eca |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono |
Regressions in System.Collections.ContainsKeyFalse<String, String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 18.26 μs | 19.80 μs | 1.08 | 0.02 | False | ||||
| 13.66 μs | 16.89 μs | 1.24 | 0.06 | False | ||||
| 16.94 μs | 19.18 μs | 1.13 | 0.01 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifactsor build instructions:Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions:CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions:MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
#Set $RunDir to the runtime directoryRunDir=`pwd`#Set the OS, arch, and OSIdRunOS='linux'RunOSId='linux'RunArch='x64'# Create mono dotnetmkdir -p $RunDir/artifacts/dotnet-mono$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=monocp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rfcp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -rcp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun# Create Core Root$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release# Clone performance git clone --branch main --depth1 --quiet https://github.com/dotnet/performance.git $RunDir/performance# One line run:python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.ContainsKeyFalse<String, String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands:# Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Builddotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Rundotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Collections.ContainsKeyFalse<String, String>*' --anyCategories Libraries Runtime" --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
#Set $RunDir to the runtime directory$RunDir="FullPathHere"#Set the OS, arch, and OSIdRunOS='windows'RunOSId='win'RunArch='x64'# Create mono dotnetmkdir -p $RunDir/artifacts/dotnet-mono$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=monoxcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /yxcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /yxcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y# Create Core Root$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release# Clone performance git clone --branch main --depth1 --quiet https://github.com/dotnet/performance.git $RunDir\performance# One line run:python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.ContainsKeyFalse<String, String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands:# Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Builddotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Rundotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Collections.ContainsKeyFalse<String, String>*' --anyCategories Libraries Runtime" --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
System.Collections.ContainsKeyFalse<String, String>.IDictionary(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Collections.ContainsKeyFalse<String, String>.ConcurrentDictionary(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Collections.ContainsKeyFalse<String, String>.Dictionary(Size: 512)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2af0051c894aac4d1a9d6479ad6dd7547a688363 |
| Compare | 8db3cab69e3afe8c25d295edb8b1d599518e4eca |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono |
Regressions in System.Memory.Span<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 43.54 ns | 57.22 ns | 1.31 | 0.28 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifactsor build instructions:Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions:CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions:MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
#Set $RunDir to the runtime directoryRunDir=`pwd`#Set the OS, arch, and OSIdRunOS='linux'RunOSId='linux'RunArch='x64'# Create mono dotnetmkdir -p $RunDir/artifacts/dotnet-mono$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=monocp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rfcp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -rcp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun# Create Core Root$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release# Clone performance git clone --branch main --depth1 --quiet https://github.com/dotnet/performance.git $RunDir/performance# One line run:python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands:# Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Builddotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Rundotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Byte>*' --anyCategories Libraries Runtime" --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
#Set $RunDir to the runtime directory$RunDir="FullPathHere"#Set the OS, arch, and OSIdRunOS='windows'RunOSId='win'RunArch='x64'# Create mono dotnetmkdir -p $RunDir/artifacts/dotnet-mono$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=monoxcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /yxcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /yxcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y# Create Core Root$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release# Clone performance git clone --branch main --depth1 --quiet https://github.com/dotnet/performance.git $RunDir\performance# One line run:python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands:# Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Builddotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Rundotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Memory.Span<Byte>*' --anyCategories Libraries Runtime" --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
System.Memory.Span<Byte>.SequenceCompareTo(Size: 512)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2af0051c894aac4d1a9d6479ad6dd7547a688363 |
| Compare | 8db3cab69e3afe8c25d295edb8b1d599518e4eca |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono |
Regressions in System.Collections.IterateForEach<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 5.61 μs | 6.49 μs | 1.16 | 0.05 | False | ||||
| 3.92 μs | 4.64 μs | 1.18 | 0.09 | False | ||||
| 5.62 μs | 6.43 μs | 1.15 | 0.10 | False | ||||
| 48.40 μs | 52.15 μs | 1.08 | 0.06 | False | ||||
| 1.59 μs | 1.85 μs | 1.17 | 0.05 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifactsor build instructions:Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions:CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions:MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
#Set $RunDir to the runtime directoryRunDir=`pwd`#Set the OS, arch, and OSIdRunOS='linux'RunOSId='linux'RunArch='x64'# Create mono dotnetmkdir -p $RunDir/artifacts/dotnet-mono$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=monocp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rfcp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -rcp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun# Create Core Root$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release# Clone performance git clone --branch main --depth1 --quiet https://github.com/dotnet/performance.git $RunDir/performance# One line run:python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.IterateForEach<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands:# Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Builddotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Rundotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Collections.IterateForEach<String>*' --anyCategories Libraries Runtime" --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
#Set $RunDir to the runtime directory$RunDir="FullPathHere"#Set the OS, arch, and OSIdRunOS='windows'RunOSId='win'RunArch='x64'# Create mono dotnetmkdir -p $RunDir/artifacts/dotnet-mono$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=monoxcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /yxcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /yxcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y# Create Core Root$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release# Clone performance git clone --branch main --depth1 --quiet https://github.com/dotnet/performance.git $RunDir\performance# One line run:python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.IterateForEach<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands:# Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Builddotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Rundotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Collections.IterateForEach<String>*' --anyCategories Libraries Runtime" --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
System.Collections.IterateForEach<String>.ConcurrentQueue(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Collections.IterateForEach<String>.Dictionary(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Collections.IterateForEach<String>.SortedList(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Collections.IterateForEach<String>.ImmutableDictionary(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Collections.IterateForEach<String>.ImmutableArray(Size: 512)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2af0051c894aac4d1a9d6479ad6dd7547a688363 |
| Compare | 8db3cab69e3afe8c25d295edb8b1d599518e4eca |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono |
Regressions in System.Collections.TryGetValueFalse<String, String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 17.96 μs | 20.06 μs | 1.12 | 0.01 | False | ||||
| 17.17 μs | 19.44 μs | 1.13 | 0.02 | False |
Repro
General Docs link:https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to
runtime/artifactsor build instructions:Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions:CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - Mono Runtime build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions:MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
#Set $RunDir to the runtime directoryRunDir=`pwd`#Set the OS, arch, and OSIdRunOS='linux'RunOSId='linux'RunArch='x64'# Create mono dotnetmkdir -p $RunDir/artifacts/dotnet-mono$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=monocp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rfcp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -rcp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun# Create Core Root$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release# Clone performance git clone --branch main --depth1 --quiet https://github.com/dotnet/performance.git $RunDir/performance# One line run:python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.TryGetValueFalse<String, String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" # Individual Commands:# Restore dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Builddotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Rundotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Collections.TryGetValueFalse<String, String>*' --anyCategories Libraries Runtime" --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages
Windows
#Set $RunDir to the runtime directory$RunDir="FullPathHere"#Set the OS, arch, and OSIdRunOS='windows'RunOSId='win'RunArch='x64'# Create mono dotnetmkdir -p $RunDir/artifacts/dotnet-mono$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=monoxcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /yxcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /yxcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y# Create Core Root$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release# Clone performance git clone --branch main --depth1 --quiet https://github.com/dotnet/performance.git $RunDir\performance# One line run:python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.TryGetValueFalse<String, String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" # Individual Commands:# Restore dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Builddotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1# Rundotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Collections.TryGetValueFalse<String, String>*' --anyCategories Libraries Runtime" --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages
System.Collections.TryGetValueFalse<String, String>.IDictionary(Size: 512)
ETL Files
Histogram
JIT Disasms
System.Collections.TryGetValueFalse<String, String>.Dictionary(Size: 512)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository