Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for issue 20371 related to deep depencies #20392

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fgarciacorona
Copy link

The goal of this PR is to provide a fix for issue #20371 and improve the performance of csharp runtime when dealing with deep nested dependencies of .proto files.

  • The fix consists in caching the recursive calls for the search of extensions.

  • A test dataset has been created with deeply nested .proto files that showcases the performance penalty.

  • A set of benchmark metrics have been created to evaluate the performance impact before and after the fix.

  • Unfortunately due to not being able to unload all the existing objects inside the same testcase, it is only possible to run the test before and after enabling the caching mechanism. I am open to any other idea on how to improve this.

The test results before and after the fix are the following:
Before the proposed fix:

FileDescriptor_CreateMessageWithDeepDependencies_BuildFromByteString
   Source: DescriptorsTest.cs line 40
   Duration: 2,1 min

  Message: 
  Expected: 399
  But was:  76306932


  Stack Trace: 
DescriptorsTest.FileDescriptor_CreateMessageWithDeepDependencies_BuildFromByteString() line 61
1)    at Google.Protobuf.Reflection.DescriptorsTest.FileDescriptor_CreateMessageWithDeepDependencies_BuildFromByteString() in D:\Git\protobuf\csharp\src\Google.Protobuf.Test\Reflection\DescriptorsTest.cs:line 61

  Standard Output: 
Running performance test for extension registry caching: With caching
{ }
GetAllExtensionsCount: 402
GetAllGeneratedExtensionsCount: 573
GetAllDependedExtensionsCount: 76306932
GetAllDependedExtensionsFromMessageCount: 644966683
TotalReturnedExtensionsCount: 118
w/ cache elapsed: 00:02:04.6190917

After the proposed fix:

 FileDescriptor_CreateMessageWithDeepDependencies_BuildFromByteString
   Source: DescriptorsTest.cs line 40
   Duration: 117 ms

  Standard Output: 
Running performance test for extension registry caching: With caching
{ }
GetAllExtensionsCount: 402
GetAllGeneratedExtensionsCount: 573
GetAllDependedExtensionsCount: 399
GetAllDependedExtensionsFromMessageCount: 39
TotalReturnedExtensionsCount: 118
w/ cache elapsed: 00:00:00.0700036

Improve performance of Extension search

Profiling/Benchmarking code still included and enabled !

Add assertions to unit test and enable caching by default

Added .proto files based on a deeper hierarchy
@fgarciacorona fgarciacorona requested a review from a team as a code owner February 19, 2025 10:51
@fgarciacorona fgarciacorona requested review from jskeet and removed request for a team February 19, 2025 10:51
Copy link

google-cla bot commented Feb 19, 2025

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@jskeet jskeet removed their request for review February 19, 2025 11:05
@tonyliaoss
Copy link
Member

Hello @fgarciacorona, we won't be able to start reviewing your PR until the CLA has been signed.

Also, I suspect that we won't accept the newly added test protos, but we can discuss that after we start our review process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants