Open
Description
What version of protobuf and what language are you using?
Version: v3.21.9
Language: C#
What operating system (Linux, Windows, ...) and version?
Windows 11
What runtime / compiler are you using (e.g., python version or gcc version)
csharp
What did you do?
unittest_deep_dependencies.zip
Steps to reproduce the behavior:
- [Optional] Take a structure of highly nested of .proto files that we have anonymized and flattened for testing purposes (see attached example: unittest_deep_dependencies folder)
- [Optional] Generate the classes for the .proto files (also included in the attached example: csharp folder)
- Copy the
csharp\UnittestDeepDependencies
folder into\csharp\src\Google.Protobuf.Test
- Open solution in csharp\src\Google.Protobuf.sln
- Create an additional Unit Test with the following content in
Google.Protobuf.Test\Reflection\DescriptorsTest.cs
:
[Test]
public void FileDescriptor_CreateMessageWithDeepDependencies_BuildFromByteString()
{
Console.WriteLine("Running performance test for extensions");
var stopwatchCached = Stopwatch.StartNew();
var example = new UnittestDeepDependencies.Example();
Console.WriteLine(example.ToString());
stopwatchCached.Stop();
Console.WriteLine($"Time elapsed: {stopwatchCached.Elapsed}");
// actually test the elapsed time
Assert.Less(stopwatchCached.Elapsed, TimeSpan.FromSeconds(2), "Extension performance was not as low as expected - please improve performance of extension resolution");
}
- Run the unit test. The unit test takes, in my case more than 1 minute.
What did you expect to see
A similar behavior as in v3.5.2, where printing a message would take milliseconds.
What did you see instead?
The print of a message takes > 1 minute
Anything else we should know about your project / environment
- The performance is affected by the change introduced to support
extend
in the runtime: Pull 8220 - The recursive iteration through the classes to search for extensions creates a runtime performance penalty.See FileDescriptor.cs
- This performance impact increases with the increased depth of the hierarchy.
- The test set provided is based on an actual data set where the following processing has been done:
- Data has been anonymized.
- Folder layout has been flattened.
- Hierarchy has been reduced to have a considerable runtime delay during unit testing.
- Stripped of additional data, leaving only dependencies between .proto files (as this seems to be the major source of the delay). This may trigger
warning
duringprotoc
execution due to unused dependencies. - Observe that the only existing Message is contained in the File117.cs
We have devised a proposal that we will bring as a PR.
Activity