C# Proto2 feature : Finale

[ObsidianMinor]

This wraps up all the previous feature PRs. Changes include:

• Many tests for previous feature PRs
• The proto3 constraint on C# code-gen has been removed
• Conformance tests for proto2 are ran instead of skipped
• GetOption APIs have been added to descriptors
• CustomOptions are now deprecated
• The generated descriptor code has been made public

When pulled, this PR will fix numerous proto2 feature issues, including: #5759, #4591, #4407, #3487, #2991

[ObsidianMinor]

We're open for review @anandolee. I'll continue to add tests in places I see lacking but it's all nearly there. Feel free to point out anywhere you feel needs more tests.

cc: @jskeet if you want to review as well

Is there anything else we need to do for proto2 support like docs?

[anandolee]

Yes, we also need to update the docs as well
https://developers.google.com/protocol-buffers/docs/reference/csharp-generated?

But this can be done later after the support is released

[jskeet]

I'm absolutely not going to have time to review, I'm afraid :(

[anandolee]

@jtattermusch , can you help to review the final PR for proto2 support from C# language's view. My feedback may more focus on protobuf's view.

We didn't make the other proto2 PRs( #5759, #4591, #4407, #3487, #2991 ) usable for users, so related APIs can still be changed

[ObsidianMinor]

We didn't make the other proto2 PRs( #5759, #4591, #4407, #3487, #2991 ) usable for users, so related APIs can still be changed

*those issues listed are not the other proto2 PRs. These are the other PRs (#4642, #5183, #5350)

[jtattermusch]

I will review once I have a bit of free time.

[gonzedge]

Hey all, I'm working on a project at the moment that has a mixture of proto2 and proto3 definitions and trying (unsuccessfully) to generate C# objects. It looks like this PR would solve my problems. So a couple of questions:

• Is this planned to be merged/released soon? How can I help?
• Is there a way to install the binaries available on this branch locally for me to try it out on my project?
• What do you recommend I do in the meantime? Is there something out there that supports both syntaxes?

[ObsidianMinor]

• I believe Google wants it released soon, however it needs to be reviewed first and the two people that would be best for that are busy, so I don't know when it'll be released.
• You can checkout my fork branch csharp/proto2-feature/finale to try it out. Of course you'll have to compile protoc yourself and you may want to rebase on master.
• In the meantime you can use another protobuf lib like mgravell/protobuf-net @gonzedge

[jtattermusch]

I'm in still in progress of reviewing (it's a lot of code), but I have a few initial comments.

[jtattermusch]

@ObsidianMinor can you also resolve the conflict?

[jtattermusch]

I merged both this PR and the the docs #6499.

@anandolee when is the planned branch cut / release date for 3.10?

[arthur-tacca]

Any idea why this want mentioned in the 3.10.0rc1 release notes? Did it not make it on? Or is it being "secretly" released to allow for testing before it's official?

[jtattermusch]

@rafi-kamal are #6499 and #5936 part of the v3.10.x branch? If so, we should update the release notes.

If not, seems like the PRs haven't made it on time for the v3.10.x branch cut (and they will be in the next release).

[rafi-kamal]

Unfortunately the branch cut happened about a week ago, before the PRs were merged. It will be included in the next release (ETA: 6 weeks), unless this is something critical and requires cherry-picking.

[ObsidianMinor]

Cherry-picking it should fix #6509 and #6510 from the 3.9 release but I don't know if that's critical or not.

[ravgupms]

@ObsidianMinor given the release happened only a few days back can you cherry pick your changes into current release branch?

I'm trying to implement a GRPC External Auth service for Envoy in C# and I'm currently blocked since a number of protos which Envoy uses are in proto2 version. And I'm sure there are many like me who are waiting for this feature for quite a long time.

[ObsidianMinor]

@ravgupms If I worked at Google, I might. But I don't, so I can't.

Maybe we could create a 3.11 beta branch with the cherry-picked changes and put up a 3.11 beta version of Google.Protobuf on nuget so people could test the proto2 features. Could that work? @anandolee @jtattermusch

[ravgupms]

@ObsidianMinor that would be great. Thanks!

[jtattermusch]

We could cherry-pick this PR and #6499 (the corresponding docs) into 3.10.x branch, but it would be a very large cherry-pick (this PR is 34 commits), so even though I understand that folks are waiting for this feature and i'd like to get it tested sooner than later, backporting such a large (non-critical) change is not something we usually do. I'm open to backporting into 3.10.x branch if @rafi-kamal is ok with that (as an exception to the normal process to facilitate early testing).

Releasing 3.11.x is not a good option because that would require cutting the 3.11.x ahead of schedule and that's not a good idea for process reasons (the branch involves all other languages too).

[rafi-kamal]

I agree with @jtattermusch, this is a pretty large change and I would prefer to release it within our normal release schedule instead of cherry-picking it to 3.10.

[tgeng]

Thanks for the awesome work! Any idea when 3.11 will be released?

[ObsidianMinor]

Next week @tgeng

[tgeng]

Thank you so much ObsidianMinor for driving this! This unblocks us and saves us from migrating some huge protos to proto3! 👍 😆