Benchmark Tests
Benchmarks were conducted on a Lenovo ThinkPad T450s - Results on the actual server environment might be better! Therefore, an important factor to look at, is the scaling between different Amount-parameters and the delta between each method! (These results should be somehow 'stable' across devices)
ExerciseService
These Benchmarks were collected from the ExerciseService.CheckAnswer() Method, where in the setup (not included in the benchmark time) Amount-many questions get pulled from the Questions-catalogue or randomly generated by the Factory.
// * Summary *
BenchmarkDotNet=v0.13.5, OS=Windows 10 (10.0.19045.2965/22H2/2022Update)
Intel Core i7-5600U CPU 2.60GHz (Broadwell), 1 CPU, 4 logical and 2 physical cores
.NET SDK=7.0.201
[Host] : .NET 6.0.14 (6.0.1423.7309), X64 RyuJIT AVX2
DefaultJob : .NET 6.0.14 (6.0.1423.7309), X64 RyuJIT AVX2
| Method | Amount | Mean | Error | StdDev | Allocated |
|------------------------------------------- |------- |------------------:|---------------:|---------------:|----------:|
| CheckAnswer_For_Mental_Correct | 1 | 376.0694 ns | 2.8835 ns | 2.6972 ns | 88 B |
| CheckAnswer_For_Analysis_Correct | 1 | 46.7278 ns | 0.3879 ns | 0.3240 ns | 88 B |
| CheckAnswer_For_Geometry_Correct | 1 | 204.2387 ns | 1.1517 ns | 1.0773 ns | 88 B |
| CheckAnswer_For_RandomMental_Correct | 1 | 17,243.4191 ns | 148.2749 ns | 138.6965 ns | 7863 B |
| CheckAnswer_For_RandomMentalInsane_Correct | 1 | 191,751.7404 ns | 1,661.9532 ns | 1,473.2783 ns | 122548 B |
| CheckAnswer_For_Mental_Wrong | 1 | 393.4626 ns | 2.6921 ns | 2.3864 ns | 88 B |
| CheckAnswer_For_Analysis_Wrong | 1 | 81.1610 ns | 1.1441 ns | 1.0702 ns | 88 B |
| CheckAnswer_For_Geometry_Wrong | 1 | 253.0405 ns | 3.7406 ns | 3.3160 ns | 88 B |
| CheckAnswer_For_RandomMental_Wrong | 1 | 16,222.5817 ns | 131.2528 ns | 122.7740 ns | 7863 B |
| CheckAnswer_For_RandomMentalInsane_Wrong | 1 | 192,425.2668 ns | 2,522.1894 ns | 2,235.8552 ns | 122395 B |
| CheckAnswer_For_Mental_Correct | 10 | 3,943.0722 ns | 25.7309 ns | 24.0687 ns | 880 B |
| CheckAnswer_For_Analysis_Correct | 10 | 720.9757 ns | 5.0698 ns | 4.2335 ns | 880 B |
| CheckAnswer_For_Geometry_Correct | 10 | 2,568.8426 ns | 16.2514 ns | 14.4064 ns | 880 B |
| CheckAnswer_For_RandomMental_Correct | 10 | 351,417.5255 ns | 5,453.3028 ns | 4,553.7548 ns | 193421 B |
| CheckAnswer_For_RandomMentalInsane_Correct | 10 | 350,652.7138 ns | 3,072.0548 ns | 3,414.5808 ns | 195013 B |
| CheckAnswer_For_Mental_Wrong | 10 | 4,220.4692 ns | 44.7595 ns | 37.3762 ns | 880 B |
| CheckAnswer_For_Analysis_Wrong | 10 | 804.2267 ns | 9.8240 ns | 9.1894 ns | 880 B |
| CheckAnswer_For_Geometry_Wrong | 10 | 2,732.1326 ns | 6.7042 ns | 5.5983 ns | 880 B |
| CheckAnswer_For_RandomMental_Wrong | 10 | 170,571.0955 ns | 1,039.2961 ns | 921.3089 ns | 79381 B |
| CheckAnswer_For_RandomMentalInsane_Wrong | 10 | 694,795.1562 ns | 8,611.6278 ns | 8,055.3220 ns | 425114 B |
| CheckAnswer_For_RandomMental_Correct | 100 | 2,095,574.8197 ns | 19,278.3336 ns | 16,098.2816 ns | 1018883 B |
| CheckAnswer_For_RandomMentalInsane_Correct | 100 | 5,133,289.5833 ns | 27,084.9770 ns | 25,335.3043 ns | 2987918 B |
| CheckAnswer_For_RandomMental_Wrong | 100 | 2,613,226.7448 ns | 38,666.1526 ns | 36,168.3431 ns | 1354488 B |
| CheckAnswer_For_RandomMentalInsane_Wrong | 100 | 5,784,581.7007 ns | 79,705.1139 ns | 66,557.3799 ns | 3437303 B |
// * Legends *
Amount : Value of the 'Amount' parameter
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
StdDev : Standard deviation of all measurements
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
1 ns : 1 Nanosecond (0.000000001 sec)
// ***** BenchmarkRunner: End *****
Run time: 00:10:52 (652.45 sec), executed benchmarks: 30
Global total time: 00:11:18 (678.57 sec), executed benchmarks: 30
// * Artifacts cleanup *
Mental Exercise Factory
These Benchmark were collected from the MentalExerciseFactory.GenerateRandomQuestion() Method, where Amount-many questions get generated.
// * Summary *
BenchmarkDotNet=v0.13.5, OS=Windows 10 (10.0.19045.2965/22H2/2022Update)
Intel Core i7-5600U CPU 2.60GHz (Broadwell), 1 CPU, 4 logical and 2 physical cores
.NET SDK=7.0.201
[Host] : .NET 6.0.14 (6.0.1423.7309), X64 RyuJIT AVX2
DefaultJob : .NET 6.0.14 (6.0.1423.7309), X64 RyuJIT AVX2
| Method | Amount | Mean | Error | StdDev | Allocated |
|------------------------------- |------- |-------------:|-----------:|-----------:|------------:|
| GenerateRandomExercises | 1 | 33.28 us | 0.357 us | 0.298 us | 11.02 KB |
| GenerateRandomExercises_Insane | 1 | 46.10 us | 0.340 us | 0.284 us | 14.83 KB |
| GenerateRandomExercises | 10 | 314.87 us | 5.853 us | 5.475 us | 110.27 KB |
| GenerateRandomExercises_Insane | 10 | 543.27 us | 7.763 us | 6.882 us | 149.27 KB |
| GenerateRandomExercises | 100 | 3,222.10 us | 42.776 us | 35.720 us | 1103.91 KB |
| GenerateRandomExercises_Insane | 100 | 4,714.66 us | 69.032 us | 64.573 us | 1480.3 KB |
| GenerateRandomExercises | 1000 | 32,455.74 us | 644.823 us | 716.719 us | 11025.3 KB |
| GenerateRandomExercises_Insane | 1000 | 45,743.09 us | 728.414 us | 645.720 us | 14839.44 KB |
// * Legends *
Amount : Value of the 'Amount' parameter
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
StdDev : Standard deviation of all measurements
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
1 us : 1 Microsecond (0.000001 sec)
// ***** BenchmarkRunner: End *****
Run time: 00:02:31 (151.58 sec), executed benchmarks: 8
Global total time: 00:02:57 (177.41 sec), executed benchmarks: 8
// * Artifacts cleanup *
Last update:
2023-06-09