Simplify build systems, consider removing GNU Makefiles

[cpu]

In #253 there was Cmake support integrated, but I'm not sure I understand the motivation. It's described as being a cross-platform solution, but we only test it for Windows (in addition to the hand-rolled Makefile that already supports Windows). Was there downstream demand for this that I'm not aware of? Is it used by any integrating projects?

This repository doesn't have an abundance of maintainer activity and I think it would be better to simplify, either removing the existing Makefile and embracing cmake, removing cmake and embracing the manual Makefile, or making progress with cargo-c via #274 (my personal preference, and the preference of multiple downstream packagers).

[jsha]

I'm definitely in favor of simplifying our build system. The original goal for integrating CMake was that we were having trouble maintaining our hand-rolled GNU-style Makefile, and in particular we would make changes that were causing tests to break on Windows. What I learned from folks wiser in Windows than myself is that most Windows projects use MSVC, it's hard to make MSVC work well with GNU Make (and many folks won't have it), and CMake, as complex and confusing as it can be, is the best and most popular option there.

Our Windows support is pretty thin and is thanks largely to great contributions from volunteers. But it's probably no surprise that I don't regularly develop on Windows so haven't had the expertise to really make it shine. For instance you can see the README doesn't even have specific instructions for a Windows build.

I think we can straightforwardly delete Makefile.Windows in preference for our CMakeLists.txt.

I'm convinced by the preference of downstream maintainers that we should support cargo-c. Can we get it to build the client/server binaries on Windows, using MSVC, and then run the integration test? Thanks to your work bringing that test into Rust, that part can be done with Cargo alone (cargo test --locked --test client_server client_server_integration -- --ignored --exact).

[cpu]

Thanks for the extra historical context 👍 I also have no significant experience with Windows, but trust the assessment that MSVC + cmake is the paved road for that platform.

I think we can straightforwardly delete Makefile.Windows in preference for our CMakeLists.txt.

That sounds good to me and seems like an immediate improvement 🎉

Do you have thoughts on the future of the Makefile used on Linux/MacOS? I think we should decide on converting it to cmake or removing it if cargo-c can fill the need.

Can we get it to build the client/server binaries on Windows, using MSVC, and then run the integration test?

I will try to tilt at that 👍

[cpu]

I've thought more about this & propose we go all-in on cargo-c for building and installing the library, and all in on cmake for building the C test applications and running integration tests.

I think in the downstream case where you want to spit out a rustls-ffi .a/.lib or .so/.dll then cargo c is your best bet. It has practically no overhead for us to maintain, supports both static/dynamic outputs well, and also handles installation well (including pkg-config considerations). I think we should make this the blessed method for installation going forward. Users already have rust/cargo and installing cargo c is minimal additional overhead IMO. In the extreme if a downstream user balks at needing an extra tool it remains possible to build the library with just cargo and additional manual work.

The remaining consideration is for the client/server test code. This isn't important to downstream consumers, but is an important aspect of how we test the crate. For this, cargo c isn't a great fit and a more capable C-friendly build system seems more natural (and serves as a better example for integration for existing C projects).

Since cmake is well positioned to support Windows I think it's clear we should keep it in the mix for the test code. Right now it uses cargo to generate a static .lib, I think we should use cargo c here as well so that we can test the pkg-config & dynamic linking workflow from cmake.

It's relatively straight-forward to update the existing CMakeList.txt files to support Linux/MacOS operating systems in addition to Windows. My new position is we should drop the GNU Makefiles and do that. It'll be one consistent way to build our test binaries and will make ongoing development & CI simpler. I think I'm also reaching the limit of traditional Makefiles by jamming in more and more environment variable based feature toggles.

What do folks think?

[cpu]

Our Windows support is pretty thin and is thanks largely to great contributions from volunteers. But it's probably no surprise that I don't regularly develop on Windows so haven't had the expertise to really make it shine.

Based on what I've learned this past week fiddling with Windows development I think the gold standard for supporting modern Windows dev would be to have rustls-ffi in vcpkg, which will indeed be easiest with cmake. More research to confirm this is workable for a Rust crate would be required.

Also I'm 1000% a UNIX person playing pretend Windows developer. Input from folks actually shipping software on Windows day to day would be helpful.

Since cmake is well positioned to support Windows I think it's clear we should keep it in the mix for the test code. Right now it uses cargo to generate a static .lib, I think we should use cargo c here as well so that we can test the pkg-config & dynamic linking workflow from cmake.

Another note on this topic: I experimented a bit with a project called corrosion to try and find a more native integration for the Rust compilation step compared to the existing approach of using cargo directly to represent a CMake external project.

Pros:

• It was less CMake for us to maintain.
• It has nice support for customizing the cargo invocation.
• It supports propagating bits of output from the cargo build process, e.g. to tell later compilation targets what linker args are required without needing to configure that explicitly ourselves.

Cons:

• It's an extra dep - I was fetching it from remote git at build time, an alternative would be to vendor it. Both have pros/cons.
• It doesn't expose a way to run test targets. This is annoying because we want to run cargo test .. -- --ignored for integration tests after building the test binaries. Doing that outside of corrosion means we build the whole crate twice, I couldn't manage to find a way to get them to share build artifacts.
• It doesn't expose a (stable) way to install generated libraries and the header file. There's an experimental feature, but it isn't stablized.
• The promise of being able to drop some manually curated linker arguments based on the cargo build output being propagated through to the Cmake binary targets worked OK for Linux, but on MacOS and Windows I found I still had to specify many of the existing linker args.
• If we get install, dynamic linking, etc working with cmake/corrosion we're duplicating work cargo c already handles for us.

Some of the above might be me holding the tool wrong but in sum it felt like we're better off migrating the external project config that we have now to using cargo c, or keeping it as is using cargo.