Investigate alternatives to use of nightly features (and other Gentoo build issues)

[Kangie]

Gentoo has some requirements for building packages that are causing issues with my attempt to update our package to 0.12.1.

1. The use of nightly features to build a shared library

Gentoo users compile software locally, and our package manager will use the installed Rust to build this package.

As per #345 this fails when using cbindgen unless you use a nightly comipler or disable stability guarantees, but maybe we don't need to run cbindgen for release builds?

We vendor the cbindgen generated rustls.h in-repo, which would avoid needing cbindgen or nightly rustc, but I think the cargo-c build process is regenerating it as part of its build. Possibly one fix is figuring out if that's necessary at all.

2. Compilation outside the src_compile phase

If we attempt to work around this with RUSTC_BOOTSTRAP=1, I see compilation when cargo cinstall is run if the src_test phase has been called (i.e. if cargo ctest has been run between cargo cbuild and cargo cinstall)

 abi_x86_64.amd64: running multilib-minimal_abi_src_install
       Dirty cfg-if v1.0.0: the profile configuration changed
   Compiling cfg-if v1.0.0
       Dirty untrusted v0.9.0: the profile configuration changed
   Compiling untrusted v0.9.0
       Dirty rustls-pki-types v1.3.1: the profile configuration changed
   Compiling rustls-pki-types v1.3.1
       Dirty spin v0.9.8: the profile configuration changed
   Compiling spin v0.9.8
       Dirty log v0.4.21: the profile configuration changed
   Compiling log v0.4.21
       Dirty zeroize v1.7.0: the profile configuration changed
   Compiling zeroize v1.7.0
       Dirty base64 v0.21.5: the profile configuration changed
   Compiling base64 v0.21.5
       Dirty subtle v2.5.0: the profile configuration changed
   Compiling subtle v2.5.0

We don't allow this in Gentoo - compilation should happen in the appropriate src_compile phase.

Current state of the Gentoo Ebuild:

# Copyright 2022-2024 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

EAPI=8

CRATES="
        aho-corasick@1.1.1
        base64@0.21.5
        cc@1.0.83
        cfg-if@1.0.0
        getrandom@0.2.11
        libc@0.2.153
        log@0.4.21
        memchr@2.6.4
        regex@1.9.6
        regex-automata@0.3.9
        regex-syntax@0.7.5
        ring@0.17.5
        rustls@0.22.0
        rustls-pemfile@2.1.1
        rustls-pki-types@1.3.1
        rustls-webpki@0.102.0
        rustversion@1.0.14
        spin@0.9.8
        subtle@2.5.0
        untrusted@0.9.0
        wasi@0.11.0+wasi-snapshot-preview1
        windows-sys@0.48.0
        windows-targets@0.48.5
        windows_aarch64_gnullvm@0.48.5
        windows_aarch64_msvc@0.48.5
        windows_i686_gnu@0.48.5
        windows_i686_msvc@0.48.5
        windows_x86_64_gnu@0.48.5
        windows_x86_64_gnullvm@0.48.5
        windows_x86_64_msvc@0.48.5
        zeroize@1.7.0
"

inherit cargo multilib-minimal rust-toolchain

DESCRIPTION="C-to-rustls bindings"
HOMEPAGE="https://github.com/rustls/rustls-ffi"
SRC_URI="https://github.com/rustls/rustls-ffi/archive/refs/tags/v${PV}.tar.gz -> ${P}.tar.gz"
SRC_URI+=" $(cargo_crate_uris)"

# From cargo-ebuild (note that webpki is also just ISC)
LICENSE="|| ( MIT Apache-2.0 ) BSD Boost-1.0 ISC MIT MPL-2.0 Unicode-DFS-2016"
# Dependent crate licenses
LICENSE+=" ISC MIT Unicode-DFS-2016"
# For Ring (see its LICENSE)
LICENSE+=" ISC openssl SSLeay MIT"
SLOT="0/${PV}"
KEYWORDS="~amd64"

BDEPEND="dev-util/cargo-c"

QA_FLAGS_IGNORED="usr/lib.*/librustls.*"

src_prepare() {
        default

        multilib_copy_sources
}

multilib_src_compile() {
        local cargoargs=(
                --library-type=cdylib
                --prefix=/usr
                --libdir="/usr/$(get_libdir)"
                --target="$(rust_abi)"
                $(usev !debug '--release')
        )

        cargo cbuild "${cargoargs[@]}" || die "cargo cbuild failed"
}

multilib_src_test() {
        local cargoargs=(
                --prefix=/usr
                --libdir="/usr/$(get_libdir)"
                --target="$(rust_abi)"
                $(usex debug '--debug' '--release')
        )

        cargo ctest "${cargoargs[@]}" || die "cargo ctest failed"
}

multilib_src_install() {
        local cargoargs=(
                --library-type=cdylib
                --prefix=/usr
                --libdir="/usr/$(get_libdir)"
                --target="$(rust_abi)"
                --destdir="${ED}"
                $(usex debug '--debug' '--release')
        )

        cargo cinstall "${cargoargs[@]}" || die "cargo cinstall failed"
}

pinging @thesamesam as the actual Gentoo package maintainer, I just touched some curl-y stuff related to rustls detection and realised that cURL 8.7.1 needs rustls >=0.12.0 :)

[Kangie]

We've worked around the first issue via RUSTC_BOOTSTRAP=1, but I'm still interested in seeing if we can avoid that altogether, at least for release builds.

It's been suggested that the use of Meson build will enable rustls-ffi to build the tests without clobbering the release crates, addressing the second issue.

[cpu]

I'm hoping that if you can avoid the RUSTC_BOOTSTRAP workaround entirely with #398 the related complications will fall away.

I'm successfully building the .so with a WIP Nix flake using cargo-c & stable rust with that branch. I'm 0% familiar with Gentoo but Nix's build environment is also fairly picky so I hope the success translates.

[cpu]

cpu reopened this now

Going to leave this open until it's confirmed that #398's updated form works for you folks.

[thesamesam]

Thanks! It's looking good now.

[Kangie]

I suspect that the test phase changing compile phase output is still an issue. Will check.

[cpu]

Thanks! It's looking good now.

Great, thank you! I'll close this out.

I suspect that the test phase changing compile phase output is still an issue.

My understanding of cargo ctest is that it's a combination build+test phase. The docs say (emphasis mine):

build the library, create the .h header, create the .pc file, build and run the tests

We're not (at least presently) using cargo-c to do anything fancy with tests. If this step is causing you issues, I suspect running cargo test should be equivalent without any unusual build side effects. Or perhaps you can look at what other Gentoo packaged software using cargo-c is doing (assuming there is some)?

In any case, if there's still trouble I think a second issue focused on that problem with as much detail as you can provide would be helpful.

Thanks!