Build error: thread 'main' panicked, No such file or directory

[adamjstewart]

Describe the bug

I'm trying to build kornia-rs, which uses maturin and cargo to build a bunch of dependencies, including rav1e. The rav1e build in particular fails for me on our CI.

To Reproduce

I'm unable to reproduce the issue locally, I can only reproduce it on a bare-bones CI image. The Linux and macOS logs can be found at these links. It should be possible to reproduce this issue using docker with the instructions at the bottom of the log.

Expected behavior

I would expect rav1e to build without issue. This happens locally but not in CI, so maybe I'm missing some dependency?

Required Information

This is in CI, so running custom commands is a little tricky, but here is what I know at least:

Toolchain:

$ cargo --version
$ rustc --version
1.76.0
$ nasm --version # if on x86_64

Version:

$ rav1e --version
0.7.1

Operating system:

$ uname -a
linux-ubuntu22.04-x86_64_v3/gcc-11.4.0
darwin-ventura-aarch64/apple-clang-15.0.0

Console Output

  error: failed to run custom build command for `rav1e v0.7.1`
  Caused by:
    process didn't exit successfully: `/private/var/folders/pc/qn0ft6pn27j3l3zn1kqm30y80000gv/T/gitlab-runner-1/spack-stage/spack-stage-py-kornia-rs-0.1.1-hgmbxj2hrvuo5fj56jw7zs6jxmspor33/spack-src/py-kornia/target/release/build/rav1e-2dc2f3837c638f7d/build-script-build` (exit status: 101)
    --- stderr
    thread 'main' panicked at /Users/gitlab-runner-1/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rav1e-0.7.1/build.rs:250:29:
    Failed to acquire build-time information: Os { code: 2, kind: NotFound, message: "No such file or directory" }
    stack backtrace:
       0:        0x10086edb4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha4abc3487e9247fb
       1:        0x100895890 - core::fmt::write::he8af205789bcd90b
       2:        0x10088295c - std::io::Write::write_fmt::hbd26f67f402b371c
       3:        0x10086ec0c - std::sys_common::backtrace::print::h1b35ea6986770f20
       4:        0x100884044 - std::panicking::default_hook::{{closure}}::had4704b216e37822
       5:        0x100883dc4 - std::panicking::default_hook::hde7681e53e69c647
       6:        0x100884468 - std::panicking::rust_panic_with_hook::h9bdbdfbbd8d8edb6
       7:        0x10086f2a8 - std::panicking::begin_panic_handler::{{closure}}::hc3e1a0567f0adab2
       8:        0x10086efe8 - std::sys_common::backtrace::__rust_end_short_backtrace::h7169dfbe5386f664
       9:        0x1008841f8 - _rust_begin_unwind
      10:        0x10089eaf0 - core::panicking::panic_fmt::hf114074f3520ee77
      11:        0x10089ef5c - core::result::unwrap_failed::h8e7e9a14fdc25fcb
      12:        0x1008413cc - core::result::Result<T,E>::expect::h91bf0f754c76c416
                                   at /private/var/folders/3k/c4s_3_q91m75pmb5jz2xc2fm0000gw/T/gitlab-runner-p-0/spack-stage/spack-stage-rust-1.76.0-dlt5z7dlyndtsdcrmuzqu3khafb2q6ix/spack-src/library/core/src/result.rs:1030:23
      13:        0x1008415a8 - build_script_build::main::h18c3ed6971800364
                                   at /Users/gitlab-runner-1/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rav1e-0.7.1/build.rs:250:3
      14:        0x1008412dc - core::ops::function::FnOnce::call_once::h7321ff834bce2967
                                   at /private/var/folders/3k/c4s_3_q91m75pmb5jz2xc2fm0000gw/T/gitlab-runner-p-0/spack-stage/spack-stage-rust-1.76.0-dlt5z7dlyndtsdcrmuzqu3khafb2q6ix/spack-src/library/core/src/ops/function.rs:250:5
      15:        0x100841008 - std::sys_common::backtrace::__rust_begin_short_backtrace::h1834d967a89ef6f6
                                   at /private/var/folders/3k/c4s_3_q91m75pmb5jz2xc2fm0000gw/T/gitlab-runner-p-0/spack-stage/spack-stage-rust-1.76.0-dlt5z7dlyndtsdcrmuzqu3khafb2q6ix/spack-src/library/std/src/sys_common/backtrace.rs:155:18
      16:        0x10084109c - std::rt::lang_start::{{closure}}::he8d570b16dafd830
                                   at /private/var/folders/3k/c4s_3_q91m75pmb5jz2xc2fm0000gw/T/gitlab-runner-p-0/spack-stage/spack-stage-rust-1.76.0-dlt5z7dlyndtsdcrmuzqu3khafb2q6ix/spack-src/library/std/src/rt.rs:166:18
      17:        0x1008840e4 - std::panicking::try::h660103e6070608c5
      18:        0x100870da8 - std::rt::lang_start_internal::h767d9065edff9d33
      19:        0x100841068 - std::rt::lang_start::hd62fa27af620c20e
                                   at /private/var/folders/3k/c4s_3_q91m75pmb5jz2xc2fm0000gw/T/gitlab-runner-p-0/spack-stage/spack-stage-rust-1.76.0-dlt5z7dlyndtsdcrmuzqu3khafb2q6ix/spack-src/library/std/src/rt.rs:165:17
      20:        0x100841a10 - _main
  warning: build failed, waiting for other jobs to finish...

P.S. I know next to nothing about rust/cargo/rav1e, I'm just an ignorant Python user 😄

@edgarriba
@ducha-aiki
@lferraz
@shijianjian
@cjpurackal
@johnnv1

[FreezyLemon]

Failed to acquire build-time information: Os { code: 2, kind: NotFound, message: "No such file or directory" }

This means that the following line in the build script failed:

built::write_built_file().expect("Failed to acquire build-time information");

built writes some information to a file, so NotFound here indicates that this writing failed. Is it possible that the docker container is configured in a way that could prevent file writes? It might also be a bug in built.

[adamjstewart]

Do you know where it would be trying to write to? Is there a way to control that?

[adamjstewart]

Also, is this an error in writing an error message, in which case we still need to solve the "Failed to acquire build-time information" message?

[FreezyLemon]

Also, is this an error in writing an error message, in which case we still need to solve the "Failed to acquire build-time information" message?

No, the error is "Failed to acquire build-time information". The message after that is the inner error message, which tells us why it failed to acquire build-time information: IO error OS { kind: NotFound, ... }.

Do you know where it would be trying to write to?

It tries to write to a temporary directory created for the build. It's the OUT_DIR (more info here and here, ctrl-f for OUT_DIR). The name follows the pattern <project root>/target/release/build/rav1e-<some hex>/out, but changes between compilations.

Is there a way to control that?

The out dir is inside the target folder used during compilation. I don't think you can set the OUT_DIR directly, but moving the target folder (e.g. by setting CARGO_TARGET_DIR) should move everything including OUT_DIR.

[adamjstewart]

Okay, apparently our docker images have no read-only partitions, so that shouldn't be an issue.

I'm still not sure what build-time information it failed to acquire though. Is it possible there is a missing dependency? If I'm able to reproduce locally with docker, what steps can I take to debug this? The error message doesn't even tell me what file it's trying to read/write...

[adamjstewart]

Possibly related to lukaslueg/built#14, I wish the error message was a little more helpful...

[adamjstewart]

Found a fix. If I build rust with developer tools (clippy, rustdoc, rustfmt, rust-analyzer) then the build succeeds. Does that sound right?

[FreezyLemon]

Ah. It's probably rustdoc then. I think built collects version information by executing rustdoc -V, so not having it installed could cause that error too.

[adamjstewart]

Thanks, that clears that up! I'm new to the rust build system so I'm not sure if/where the dependency on rustdoc is listed. I'll chalk this one up to rust needing better error messages. Thanks again for the help!

[lukaslueg]

Does this need a bump in built right now?

[lu-zero]

I'm afraid so