cmd/go: build cache does not check #included headers for changes

[FlorianUekermann]

go 1.10 linux/amd64

The go build and go test commands don't rebuild packages if included cgo files changed. I guess a solution would be to run the preprocessor first or disable caching for packages that use cgo altogether.

[andybons]

@ianlancetaylor

[ianlancetaylor]

Can you show us a small standalone example? I'm not clear on what you mean by "included cgo files". Do you mean explicitly files included using #include?

[FlorianUekermann]

Do you mean explicitly files included using #include?

Yes.

[ianlancetaylor]

One possibility would be to use -MD when running the C compiler, and add the listed files to the hash.

[FlorianUekermann]

use -MD when running the C compiler, and add the listed files to the hash.

Just to make sure I'm following. Did you mean add the contents of all listed files or just the list?

I had running with -E and hashing the output in mind, but I guess your idea may be more efficient (is it?).

[ianlancetaylor]

I meant the contents of the listed files, as in cache.FileHash in cmd/go/internal/cache/hash.go. The idea is to be able to use the cache to detect whether we can skip running the compiler. If we use -E then we have to run the compiler anyhow to see whether the cache is up to date.

This is also affecting me in a more general sense that with 1.10 there is no way to model the build dependency to statically linked libraries anymore. If the library changes, the cached files are still reused and I silently end up using an older version of the library unless I do go clean -cache -i <package> before the build. With go versions previous to 1.10 I had cmake touching my cgo wrappers to have them rebuilt.

[rsc]

I think this is basically working as expected.
If you change the underlying C code, or you change the compiler,
then you have to rebuild with -a. I'll leave it open in case there is
a simple fix but I don't think there is.

[navytux]

There is a way to make it work with the help of gcc -MD & friends:

(hello.h)

#define WORLDNUM        3

(hello.c)

#include <stdio.h>
#include "hello.h"

int main() {
        printf("Hello world (%d)!\n", WORLDNUM);
}

$ gcc -c -MMD -MT cdeps hello.c
$ cat hello.d 
cdeps: hello.c hello.h

$ gcc -c -MD -MT cdeps hello.c 
$ cat hello.d 
cdeps: hello.c /usr/include/stdc-predef.h /usr/include/stdio.h \
 /usr/include/x86_64-linux-gnu/bits/libc-header-start.h \
 /usr/include/features.h /usr/include/x86_64-linux-gnu/sys/cdefs.h \
 /usr/include/x86_64-linux-gnu/bits/wordsize.h \
 /usr/include/x86_64-linux-gnu/bits/long-double.h \
 /usr/include/x86_64-linux-gnu/gnu/stubs.h \
 /usr/include/x86_64-linux-gnu/gnu/stubs-64.h \
 /usr/lib/gcc/x86_64-linux-gnu/7/include/stddef.h \
 /usr/include/x86_64-linux-gnu/bits/types.h \
 /usr/include/x86_64-linux-gnu/bits/typesizes.h \
 /usr/include/x86_64-linux-gnu/bits/types/__FILE.h \
 /usr/include/x86_64-linux-gnu/bits/types/FILE.h \
 /usr/include/x86_64-linux-gnu/bits/libio.h \
 /usr/include/x86_64-linux-gnu/bits/_G_config.h \
 /usr/include/x86_64-linux-gnu/bits/types/__mbstate_t.h \
 /usr/lib/gcc/x86_64-linux-gnu/7/include/stdarg.h \
 /usr/include/x86_64-linux-gnu/bits/stdio_lim.h \
 /usr/include/x86_64-linux-gnu/bits/sys_errlist.h hello.h

This way if there is something like

// #include "mycode.cinc"
import "C"

Cgo could see the dependency on mycode.cinc and other files mycode.cinc includes.