Using BenchmarkDotNet
I had been introduced to this package last year, and was impressed after using it to profile some new code my team had written. BenchmarkDotNet is an open source profiling framework which allows you to build performance measuring projects into your solutions very easily. You simply add a new project, pull in the NuGet package, and start writing tests. A neat feature of this package is that it allows you to also get memory diagnostics, so you can understand the GC footprint of your code.Name that Smell
Here is method that piqued my curiosity.
public IList<ResultMessage> CheckLoadFederalStimulusRules(int code, AccountStatusReasonCode reason)
{
var allowedLockedReasons = new List<accountstatusreasoncode>
{
AccountStatusReasonCode.PASSWORD_ATTEMPTS,
AccountStatusReasonCode.PIN_ATTEMPTS,
AccountStatusReasonCode.UNLOCK_ATTEMPTS
};
var messages = new List<ResultMessage>();
switch (code)
{
case (int)AccountStatusCode.Closed:
case (int)AccountStatusCode.PermanentlyClosed:
messages.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Closed));
break;
case (int)AccountStatusCode.Locked:
if (!allowedLockedReasons.Contains(reason))
{
messages.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Locked));
}
break;
}
return messages;
}
This is a really simple method. It creates a list of allowable reasons, and then verifies if the operation is allowed. Knowing that the method will get called a lot, the memory allocations just didn't sit right. So I dig in to see if I can make it faster.
Build the Benchmark Harness
Typically you would want to build this right into your solution, but I use Visual Studio for Mac at home and this solution doesn't play nice without IIS, so I go an alternate route and setup a new .NET Core console app.public class Program
{
TestClass test = new TestClass();
static void Main(string[] args)
{
Console.WriteLine(@"Let's test some code!");
var summary = BenchmarkRunner.Run<Program>();
}
[Benchmark]
[Arguments(0, AccountStatusReasonCode.NONE)]
[Arguments(0, AccountStatusReasonCode.UNLOCK_ATTEMPTS)]
[Arguments(1, AccountStatusReasonCode.UNLOCK_ATTEMPTS)]
[Arguments(1, AccountStatusReasonCode.NONE)]
[Arguments(3, AccountStatusReasonCode.UNLOCK_ATTEMPTS)]
[Arguments(3, AccountStatusReasonCode.NONE)]
public void TestOriginal(int code, AccountStatusReasonCode reason)
{
var messages = test.CheckLoadFederalStimulusRules(code, reason);
}
}
The first thing I do is tear the method out of it's home, throw it into the test harness, and setup a benchmark to exercise the various code paths. The attributes drive the benchmarks and allow the developer to control how any particular method is tested. I wanted each code path evaluated to make sure I get a full idea of how the method behaves. Running the application kicks off the profiling process and dumps the diagnostics out to the console. Since I don't have anything to compare it to, I run through some simple optimizations just to see how much better we can get.
Let's Tune Some Code
I wrote three different versions of the method to get a feel for how each optimization affects the performance. I'll throw the code for all of them at the bottom of the page.
UpdatedCheckLoadFederalStimulusRules
I moved the List<AccountStatusReasonCode> object to a static class variable, so we only instantiate and initialize the list once.
OptimizedCheckLoadFederalStimulusRules
This version was a lot more aggressive. I noticed I could take advantage of the calling class where it treats a null response object as a success condition. Next, I unwound the switch block and turned it into if/then statements.
FinalCheckLoadFederalStimulusRules
This version also uses the null return to prevent creating an empty response list for success conditions. It also happens to be much more readable than the Optimized version above.
I moved the List<AccountStatusReasonCode> object to a static class variable, so we only instantiate and initialize the list once.
OptimizedCheckLoadFederalStimulusRules
This version was a lot more aggressive. I noticed I could take advantage of the calling class where it treats a null response object as a success condition. Next, I unwound the switch block and turned it into if/then statements.
FinalCheckLoadFederalStimulusRules
This version also uses the null return to prevent creating an empty response list for success conditions. It also happens to be much more readable than the Optimized version above.
Benchmarking & Results
BenchmarkDotNet produces a bunch of statistical analysis graphs and files in the application folder, but I am really just interested in the console output for my purposes. The output breaks down each method and each test case and shows how it performed under load.
The results were actually a bit surprising and show just how expensive memory allocations are. For the fastest path, the final version produced a 91% decrease in runtime with no memory allocations. For the slowest path, we still produced a 26% decrease in runtime, and cut the amount of garbage collection almost in half.
The difference between the Optimized version and the Final version showed that they are almost equivalent, but to have the code be much more readable and maintainable is worth the tiny performance hit in the fastest code paths. For a little more insight into the underlying difference between these two methods, refer to this article on the differences between if and switch at the IL level.
Final Thoughts and Code
This simple exercise just goes to show how small items can eat into the performance of your application, and also how expensive memory allocations are. With just a little more care and attention by the original coder, we could have had a version that performed much better under load. The hope is that you can utilize this package to test the code you are writing and experiment with various constructs to better understand how to optimize your application and identify performance problems before your code hits the field.Here is the code file in it's final tested form.
using System;
using System.Collections.Generic;
using Serve.Internal.Shared.DataContracts;
namespace Apr2020
{
public class TestClass
{
static List<AccountStatusReasonCode> AllowedLockReasons;
#region Constructors
public TestClass()
{
}
static TestClass()
{
AllowedLockReasons = new List<AccountStatusReasonCode>
{
AccountStatusReasonCode.PASSWORD_ATTEMPTS,
AccountStatusReasonCode.PIN_ATTEMPTS,
AccountStatusReasonCode.UNLOCK_ATTEMPTS
};
}
#endregion
// This is the original version of the method
public IList<ResultMessage> CheckLoadFederalStimulusRules(int code, AccountStatusReasonCode reason)
{
var allowedLockedReasons = new List<AccountStatusReasonCode>
{
AccountStatusReasonCode.PASSWORD_ATTEMPTS,
AccountStatusReasonCode.PIN_ATTEMPTS,
AccountStatusReasonCode.UNLOCK_ATTEMPTS
};
var messages = new List<ResultMessage>();
switch (code)
{
case (int)AccountStatusCode.Closed:
case (int)AccountStatusCode.PermanentlyClosed:
messages.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Closed));
break;
case (int)AccountStatusCode.Locked:
if (!allowedLockedReasons.Contains(reason))
{
messages.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Locked));
}
break;
}
return messages;
}
// We use the static collection for checking lock reasons and lessen memory allocations
public IList<ResultMessage> UpdatedCheckLoadFederalStimulusRules(int code, AccountStatusReasonCode reason)
{
var messages = new List<ResultMessage>(1);
switch (code)
{
case (int)AccountStatusCode.Closed:
case (int)AccountStatusCode.PermanentlyClosed:
messages.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Closed));
break;
case (int)AccountStatusCode.Locked:
if (!TestClass.AllowedLockReasons.Contains(reason))
{
messages.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Locked));
}
break;
}
return messages;
}
// Unwind the case statement and try and take advantage of the null
// return handling here to shortcut processing
// Less readable, but very optimized
public IList<ResultMessage> OptimizedCheckLoadFederalStimulusRules(int code, AccountStatusReasonCode reason)
{
if ((int)AccountStatusCode.Open == code)
{
return null;
}
if ((int)AccountStatusCode.Locked == code)
{
if (!TestClass.AllowedLockReasons.Contains(reason))
{
var messages = new List<ResultMessage>(1);
messages.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Locked));
return messages;
}
}
else if ((int)AccountStatusCode.Closed == code || (int)AccountStatusCode.PermanentlyClosed == code)
{
var messages = new List<ResultMessage>(1);
messages.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Closed));
return messages;
}
return null;
}
// Back to the Switch construct, optimized for readability and speed
public IList<ResultMessage> FinalCheckLoadFederalStimulusRules(int code, AccountStatusReasonCode reason)
{
switch (code)
{
case (int)AccountStatusCode.Closed:
case (int)AccountStatusCode.PermanentlyClosed:
{
var result = new List<ResultMessage>(1);
result.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Closed));
return result;
}
case (int)AccountStatusCode.Locked:
{
if (!TestClass.AllowedLockReasons.Contains(reason))
{
var result = new List<ResultMessage>(1);
result.Add(new ResultMessage(ResultStatus.Fail, AccountStatusCode.Locked));
return result;
}
return null;
}
default:
return null;
}
}
}
}
