C# High Performance .NET String Split
I recently worked on some heavy optimisations for BunnySpeed and noticed just how slow the .NET implementation of String.Split() can get when calling hundreds of thousands of times per second. I assume this happens due to re-creation and resizing of the result string arrays.
Not only does this produce additional work for GC, it also uses extra CPU cycles, which is especially noticable when doing a huge amount of text processing. On top of that it seems the .NET implementation uses some additional checking while splitting strings and introducing even further unnecessary CPU load. Even MSDN warns about this.
With this in mind, I set to create a string splitter that would use an internal buffer similarly to a StringBuilder. It uses very simple logic of going through the string and saving the value parts into the buffer as it goes along.
public int Split(string value, char separator) { int resultIndex = 0; int startIndex = 0; // Find the mid-parts for (int i = 0; i < value.Length; i++) { if (value[i] == separator) { this.buffer[resultIndex] = value.Substring(startIndex, i - startIndex); resultIndex++; startIndex = i + 1; } } // Find the last part this.buffer[resultIndex] = value.Substring(startIndex, value.Length - startIndex); resultIndex++; return resultIndex; }
The StringSplitter class is incredibly simple to use as you can see in the example below. Just be careful to reuse the StringSplitter object and not create a new instance of it in loops or for a single time use. In this case it would be better to juse use the built in String.Split.
var splitter = new StringSplitter(2); splitter.Split("Hello World", ' '); if (splitter.Results[0] == "Hello" && splitter.Results[1] == "World") { Console.WriteLine("It works!"); }
the Split methods returns the number of items found, so you can easily iterate through the results like this:
var splitter = new StringSplitter(2); var len = splitter.Split("Hello World", ' '); for (int i = 0; i < len; i++) { Console.WriteLine(splitter.Results[i]); }
The new code proved to work close to 40% faster than the built in implementation with a simple 10 million iterations test.
.NET String.Split: ~4450 ms
StringSplitter.Split: ~2760 ms
Multiple character split proved to be even more effecient, being almost 50% faster. Once again with 10 million iterations
.NET String.Split: ~4380 ms
StringSplitter.Split: ~2360 ms
There is a problem with the method above though. What happens when we reach the end of the buffer? To tackle this I introduced a new set of methods called SafeSplit that will automatically resize the buffer when needed for double the size.
public int SafeSplit(string value, char separator) { int resultIndex = 0; int startIndex = 0; // Find the mid-parts for (int i = 0; i < value.Length; i++) { if (value[i] == separator) { // Check if the array needs to be resized if (this.buffer.Length == resultIndex) { Array.Resize(ref this.buffer, this.buffer.Length * 2); } this.buffer[resultIndex] = value.Substring(startIndex, i - startIndex); resultIndex++; startIndex = i + 1; } } // Check if the array needs to be resized if (this.buffer.Length == resultIndex) { Array.Resize(ref this.buffer, this.buffer.Length * 2); } // Find the last part this.buffer[resultIndex] = value.Substring(startIndex, value.Length - startIndex); return resultIndex + 1; }
The two method sets are separated for cases when you are 100% sure about your data and want to achieve the maximum performance possible.
You can download the full class from my OneDrive account here.
I am sure there are further optimizations that can be made to the class to make it work even faster. Also, I would like to point out that at the moment some of the methods were not thoroughly tested. I am however writing unit tests for this, but until they are finished, use it at your own risk.
