Abseil overhaul, again

[h-vetinari]

So, as most people know by now, abseil's ABI (by default) depends on the C++ version used to compile it¹.

This is problematic because we can only have one ABI per shared library (trying to load two different ones would lead to lots of ODR violations at best, segfaults at worst).

This left us with the choice of:

1. use C++17 for everyone (though some packages downstream are not ready yet, especially on win)
2. use C++11 for everyone (but some packages already require C++17, so no-go)
3. try to support several ABIs at once using build strings

I tried 3. in #35, because it seemed like it would have the least painful contraints (and was nicely explicit in making people aware of the ABI issue in the build string as well as having to match using the C++ version of consuming packages). The idea was that shared builds stay on C++17 for those who can, and those who're stuck with C++11/14 can use a static lib, with the intention to encapsulate that ABI dependence at the package using the static lib (because a run-export for the same C++ version would then conflict with the C++17 shared builds).

This worked for grpc (or seemed to at least) and some other packages, but the encapsulation is leakier than I had hoped. For example, for building a static library (which we have a few of, especially on windows), even if specifying target_link_libraries(<target> PRIVATE absl::something), this effectively gets turned into public linkage, which makes libabseil-static a runtime requirement, and runs into the same virality issue where stuff that's run-depending on libabseil-static=*=cxx14* cannot be co-installed with libabseil[=*=cxx17*]. There are also packages which need public linkage anyway (and thus a run-dependence), because abseil types are part of their public API.

In practice it's been messy to try to work through this, but so far this was the only solution (given that e.g. gRPC bumped its required C++ version to 14, and moving to C++17 seemed too aggressive on windows given that some consumers weren't yet ready to change). I've tried as best as I could so far, but I'm getting to the point where I don't think it's really feasible as an overall approach. The remaining options are:

1. use C++17
2. use C++11 ABI
3. try to support several ABIs at once using build strings
4. keep using the compat-types no matter which C++ version (see Customize options.h for ABI consistency #43)

The last option is something I hadn't been aware of. It works by forcing abseil to never replace its own types with the ones from the stdlib, which means the ABI stays the same regardless of the C++ version that a consuming package uses, but also that now there's a lot more symbols and that the compiler will have a harder time optimizing things because those symbols live behind a DLL boundary (and aren't as transparent to the compiler as stdlib types).

[update] option 5 appeared:
5. support C++17 & compat ABI by changing the inline namespace for symbols used by the latter. This should mean that two shared builds with different ABIs could co-exist at runtime without stuff blowing up.

Some advantages & disadvantages as I see them:

option	work necessary	fragility / complexity	performance / footprint	comment
1.	❌	✔️	✔️	needs to move all feedstocks to C++17; some code bases might not be ready yet
3.	❌	❌	✔️	needs to juggle several incompatible ABIs that conflict
4.	✔️	❔/ ➖	➖	would break packages relying on interop between stdlib & abseil for newer standards; potential performance impact
5.	✔️	✔️	✔️	best of all worlds?

I don't think 3. is really salvageable (except through moving stuff that breaks the encapsulation to C++17), and I don't think 1. is reasonable from the work involved for a migration (and already, the migrators for abseil, grpc & protobuf are piled on top of each other).

obsoleted section about going with 4.

So I'm planning to move forward with 4., unless there are other opinions. In keeping with the spirit of marking the ABI in the build-string, I'm planning to do:

libabseil=*=compat_abi*
libabseil-static=*=compat_abi*

This intentionally doesn't refer to a standard version (which would IMO be confusing vis-à-vis abseil default behaviour), and AFAICT, there's no guarantee that this has to match the ABI of one of C++11/14/17, because abseil might have more compat types than are necessary even in the oldest C++ version they support (in particular it'll likely change once abseil drops C++11 in the next release).

I think 4 sounds doable, but might still run into problems as pointed out by Isuru, and 5. just sounds way better overall. More detailed exposition on option 5 here.

Thankfully, the universe has kindly provided us with a new minor version (#44), so that the transition from the current setup can happen sanely (rerun another migration against 20220623.1).

Please comment if you have thoughts
@conda-forge/abseil-cpp @conda-forge/core

Footnotes

1. basically because it depends whether ABI will use the types from the C++ standard library (if the targetted standard version is high enough), or it's own internal emulations of those types (which aren't ABI-compatible with the stdlib-types). ↩

[hmaarrfk]

For example, for building a static library (which we have a few of, especially on windows), even if specifying target_link_libraries( PRIVATE absl::something), this effectively gets turned into public linkage, which makes libabseil-static a runtime requirement, and runs into the same virality issue where stuff that's run-depending on libabseil-static==cxx14 cannot be co-installed with libabseil[==cxx17]

Can you provide a specific example where this was true?

[isuruf]

but also that now there's a lot more symbols and that the compiler will have a harder time optimizing things because those symbols live behind a DLL boundary (and aren't as transparent to the compiler as stdlib types).

How is it any different from using stdlib implementations which live in libstdc++.so etc?

performance/footprint

Why do you claim that it would have a performance issue?

One disadvantage you forgot was interoperability with std types. Some codes using C++17 might assume that abseil::any and std::any are interoperable.

[coryan]

Tagging abseil/abseil-cpp#696

If you want to hear terrible, no-good, very-bad ideas:

• create a separate package for Abseil with C++11, say abseil-cpp-cxx14

• patch that package to only support the absl::{any,variant,string_view} types, via the patches to absl/base/options.h

• Change the inline namespace to something like lts_20220623_cxx14

  • https://github.com/abseil/abseil-cpp/blob/8c0b94e793a66495e0b1f34a5eb26bd7dc672db0/absl/base/options.h#L209

• change the standard abseil-cpp package to always require C++17, i.e., add cxx_std_17 to the target_compile_requires():

  • https://github.com/abseil/abseil-cpp/blob/4a1ccf16ed98c876bf1e1985afb080baeff5101f/CMake/AbseilHelpers.cmake#L282
  • https://github.com/abseil/abseil-cpp/blob/4a1ccf16ed98c876bf1e1985afb080baeff5101f/CMake/AbseilHelpers.cmake#L330

Now, most libraries can link with the standard abseil-cpp package. They will be forced to adopt C++17, but they kind of wanted to. If a library is not ready, it can link with abseil-cpp-cxx14.

[h-vetinari]

Can you provide a specific example where this was true?

Grpc itself on windows, see conda-forge/grpc-cpp-feedstock#237.

(Public linkage obviously fails without a rundep, removing the linkage fails with missing symbols - unsurprisingly - and private linkage still fails at link time, because the grpc builds on windows are static.)

[h-vetinari]

How is it any different from using stdlib implementations which live in libstdc++.so etc?

performance/footprint

Why do you claim that it would have a performance issue?

My impression is that the stdlib does better than what abseil can do (it's not a level playing field, the compiler knows the stdlib better than a third-party lib), so falling back on abseil types where it wouldn't strictly be necessary might have an impact.

I don't have numbers beyond this impression though, so I'm proposing already in the OP to move to the lowest denominator ABI - not least to keep the effort manageable and the migrations moving forward.

One disadvantage you forgot was interoperability with std types. Some codes using C++17 might assume that abseil::any and std::any are interoperable.

Is that a valid (=supported) assumption when using abseil?

[hmaarrfk]

Can you provide a specific example where this was true?

Grpc itself on windows, see conda-forge/grpc-cpp-feedstock#237.

(Public linkage obviously fails without a rundep, removing the linkage fails with missing symbols - unsurprisingly - and private linkage still fails at link time, because the grpc builds on windows are static.)

I delt with something similar with pytorch. It is very likely that somebody didn't declare abseil as a dependency to one of the submodules and it succeeded transitively. I don't think this is an issue of "even private symbols leak". If configured correctly, I don't think I've seen this before.

We may have to find the package in question, and add a dependency to abseil ourselves, and eventually upstream.

[h-vetinari]

I don't think this is an issue of "even private symbols leak". If configured correctly, I don't think I've seen this before.

I know the explanation about this from the cmake mailing list is long, but I think it's worth a read, e.g.:

[...] when A links in B as PRIVATE, the include directories of B never propagate to something linking to A, but if A is a static library, then the linking of B behaves as though the relationship was PUBLIC. This PRIVATE-becomes-PUBLIC behaviour for static libraries only applies to the linking, not to the other dependencies (compiler options/flags and include search paths).

In particular, building a static target will not be self-contained (as in: bring along all the symbols of its dependencies), and so linking a static target still needs the libs that are "privately" linked to be present. Here's another ML post that lays out why such self-contained static libs are hard to achieve (and some workarounds that I don't think are scalable for us).

It could still be that I'm somehow misunderstanding something, but the stuff I found when researching this is consistent with the behaviour of grpc at link time (happily, conda-forge/grpc-cpp-feedstock#237 now contains a test, so we can catch this on the feedstock itself, and not in downstream builds).

Not sure if someone has a different way of achieving the "bundling" of necessary-symbols-for-static-targets that would give us back the encapsulation which would make option 3. work out better. This SO question has some answers, but not sure how applicable/scalable this is for us.

[h-vetinari]

One disadvantage you forgot was interoperability with std types. Some codes using C++17 might assume that abseil::any and std::any are interoperable.

Is that a valid (=supported) assumption when using abseil?

Do you happen to know some projects that are relying on this explicitly or implicitly? I'd like to get a feel how this looks in practice - it feels risky to me because that assumption depends on something outside of the control of that project (how abseil is built).

But you're right, if code depends on interop between abseil and standard types based on using the C++17 ABI, then moving to the lowest common denominator might break something. It's another constraint in an already very tangled web, but not sure how to weight it relative to the others, resp. if we'd still have a workable solution left at all if we take the lowest common denominator off the table.

[h-vetinari]

Thanks for joining in @coryan! (For context, Carlos is the top committer of https://github.com/googleapis/google-cloud-cpp & also maintains the feedstock on cf-side. and he proposed the lowest-common-denominator ABI independently of @pitrou on the same day).

If you want to hear terrible, no-good, very-bad ideas:

I'm not sure how what to read out of this, TBH 😅
Is this poking fun at the whole situation, or intended to show other things that would be possible?

[pitrou]

If you want to hear terrible, no-good, very-bad ideas:

That would actually sound like a nice way out of this quagmire.

[h-vetinari]

If you want to hear terrible, no-good, very-bad ideas:

OK, the intro (and mix of C++11 with cxx14) threw me off, but

Change the inline namespace

sounds like a really good idea to have symbols for two different ABIs that can coexist without blowing up. 🥳

[h-vetinari]

Change the inline namespace

sounds like a really good idea to have symbols for two different ABIs that can coexist without blowing up. 🥳

I updated the OP to include this option, and I'm trying to think this through, as it seems like the best of the options by far.

If I understood the idea correctly, it would mean that:

• libabseil [C++17] and (say) libabseil-compat [only abseil compat-types] could co-exist at runtime.
• For the host dependencies, one should choose one of the two.
• However, both might be present in host (e.g. through the run-dep of another lib).
  • In this case, the headers will clobber each other (but they're identical, so it's fine), and the linker would find the symbols with the respective ABI in either of the shared libraries.
  • I don't know what the linker does for symbols that aren't affected by the ABI and are therefore duplicate between the SOs, presumably it'll choose the first one it finds. This should not be a problem in general, but might be in the following case:

# extract of recipe for package libxyz
requirements:
  host:
    - libabseil-compat
  run:
    - libabseil-compat [implicitly through the run-export]

# extract of recipe for package ABC
requirements:
  host:
    - libabseil
    - libxyz
  run:
    - libabseil [implicitly through the run-export of libabseil]

This means that at build-time, there will be both libabseil & libabseil-compat in the host-env of ABC, and if we're unlucky, some of the duplicated symbols might be picked up from both symbol namespaces (= different packages). This isn't a problem if libxyz has a run-export for itself (because then both abseil flavours would be present in the final runtime environment), but if libxyz for some reason does not have a run-export, then libabseil-compat would not be present anymore at runtime -> boom?

[coryan]

If you want to hear terrible, no-good, very-bad ideas:

I'm not sure how what to read out of this, TBH 😅 Is this poking fun at the whole situation, or intended to show other things that would be possible?

Poking fun at myself mostly, and since I do not have full context on the conda ecosystem, there is a good chance my ideas are bad. I am willing to share them.

[hmaarrfk]

Is grpc static on windows?