New Update can't make

[namecloudz]

root@filter:/XDP-Firewall# make
mkdir -p build/
clang -O2 -c -o build/config.o src/config.c
clang -O2 -c -o build/cmdline.o src/cmdline.c
make -C modules/xdp-tools
make[1]: Entering directory '/root/XDP-Firewall/modules/xdp-tools'
sh configure
Found clang binary 'clang' with version 11 (from 'Ubuntu clang version 11.1.0-++20211011094159+1fdec59bffc1-1exp120211011214622.5')
bpftool doesn't support skeleton generation
make[1]: *** [Makefile:49: config.mk] Error 1
make[1]: Leaving directory '/root/XDP-Firewall/modules/xdp-tools'
make: *** [Makefile:75: libxdp] Error 2
root@filter:/XDP-Firewall# make libxdp
make -C modules/xdp-tools
make[1]: Entering directory '/root/XDP-Firewall/modules/xdp-tools'
sh configure
Found clang binary 'clang' with version 11 (from 'Ubuntu clang version 11.1.0-++20211011094159+1fdec59bffc1-1exp120211011214622.5')
bpftool doesn't support skeleton generation
make[1]: *** [Makefile:49: config.mk] Error 1
make[1]: Leaving directory '/root/XDP-Firewall/modules/xdp-tools'
make: *** [Makefile:75: libxdp] Error 2
root@filter:~/XDP-Firewall#

[gamemann]

Hey! What Linux distro are you using when building the program and can you give the output of the following?

# Kernel version
uname -r

# Release file
cat /etc/*-release

I've ran into this error before, but I don't remember what I did to fix it and I'm pretty sure it's related to LibXDP (missing a dependency).

If you want something quickly, you can try checking out the commit before moving to LibXDP which is detailed here. There haven't been many changes to the XDP code itself, so you should be running the same program with the exception of using LibXDP to load the BPF/XDP program.

[namecloudz]

root@filter:~# uname -r

5.4.0-139-generic

root@filter:~# cat /etc/*-release

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

[pettai]

See xdp-project/xdp-tools#370

[gamemann]

I spun up a vanilla Ubuntu 20.04 VM with the same kernel version as you and ran into the same issue. I couldn't even compile bpftool from source and the issue for me ended up being the kernel being too outdated. I was able to get it working by installing kernel image linux-image-5.15.0-86-generic which ships with Ubuntu 20.04 and its headers via the following command.

sudo apt install linux-image-5.15.0-86-generic linux-headers-5.15.0-86-generic linux-tools-5.15.0-86-generic

I tried installing linux-tools-generic and linux-tools-5.15.0-86-generic, but I had the same error. However, I was able to compile bpftool successfully from source which fixed the issue.

# Clone BPFTool and its submodules.
git clone --recurse-submodules https://github.com/libbpf/bpftool.git

# Change directory to source.
cd bpftool/src

# Make and install.
make -j 2
sudo make install

I also installed version 14 of clang and had to manually create a symbolic link for llc. I'm not entirely sure version >= 14 is needed. I'm going to reinstall the VM soon and see if the kernel change plus version 12 of clang which ships with Ubuntu 20.04 is fine enough.

# Download and use LVM 14 install script.
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 14

# Replace older version of Clang with 14.
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-14 100
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-14 100

# Remove old LLC link and use LLVM/LLC-14 instead.
sudo rm -f /usr/bin/llc
sudo ln -s /usr/lib/llvm-14/bin/llc /usr/bin/llc

I hope this helps and I'll let you know once I reinstall the VM!

[gamemann]

After a full reinstall of the VM, I was able to get away with installing kernel image 5.15.0-86-generic and Clang 12. Clang 10 was too outdated, but it's possible Clang 11 may work (I didn't test Clang 11).

I didn't need to build bpftool from source as well, so I installed the kernel 5.15.0-86-generic tools as suggested in @pettai's linked issue 😄

Here are the commands I performed from the start of the vanilla VM:

# Update and upgrade packages
sudo apt update
sudo apt upgrade

# Install kernel 5.15.0-86-generic and its tools for bpftool
sudo apt install -y linux-image-5.15.0-86-generic linux-headers-5.15.0-86-generic linux-tools-5.15.0-86-generic

# Reboot to load new kernel
sudo reboot

# Confirm kernel version
uname -r

# Install dependencies for XDP Firewall
sudo apt install -y libconfig-dev llvm clang libelf-dev build-essential
sudo apt install -y libpcap-dev m4 gcc-multilib

# Install Clang 12
sudo apt install -y clang-12

# Update primary clang binaries to use version 12
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-12 100
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-12 100

# Confirm Clang version
clang -v

# Replace LLVM/LLC 10 with version 14 (otherwise an error occurs when building).
# Note - I'm not sure what to use with `update-alternatives` since `llc` and `llvm` doesn't work with it. So instead, I manually remove the old symbolic link and create a new one pointed towards LLVM 12.
sudo rm -f /usr/bin/llc
sudo ln -s /usr/lib/llvm-12/bin/llc /usr/bin/llc

# Confirm LLVM/LLC version
llc --version

# Clone XDP Firewall and its sub-modules
git clone --recursive https://github.com/gamemann/XDP-Firewall.git

# Change directory
cd XDP-Firewall

# Make and install LibXDP
make libxdp

# Make XDP Firewall
make

# Install Firewall
sudo make install

# Edit /etc/xdpfw/xdpfw.conf if needed (e.g. using different interface name)

# Run Firewall
sudo xdpfw

When building, I do receive a warning, but I believe it's harmless.

clang -lconfig -lelf -lz -I modules/xdp-tools/lib/libbpf/src -I /usr/include -I /usr/local/include -o build/xdpfw modules/xdp-tools/lib/libbpf/src/staticobjs/bpf_prog_linfo.o modules/xdp-tools/lib/libbpf/src/staticobjs/bpf.o modules/xdp-tools/lib/libbpf/src/staticobjs/btf_dump.o modules/xdp-tools/lib/libbpf/src/staticobjs/btf.o modules/xdp-tools/lib/libbpf/src/staticobjs/gen_loader.o   modules/xdp-tools/lib/libbpf/src/staticobjs/hashmap.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf_errno.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf_probes.o  modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf.o modules/xdp-tools/lib/libbpf/src/staticobjs/linker.o modules/xdp-tools/lib/libbpf/src/staticobjs/netlink.o  modules/xdp-tools/lib/libbpf/src/staticobjs/nlattr.o modules/xdp-tools/lib/libbpf/src/staticobjs/relo_core.o modules/xdp-tools/lib/libbpf/src/staticobjs/ringbuf.o modules/xdp-tools/lib/libbpf/src/staticobjs/str_error.o modules/xdp-tools/lib/libbpf/src/staticobjs/strset.o modules/xdp-tools/lib/libbpf/src/staticobjs/usdt.o modules/xdp-tools/lib/libxdp/sharedobjs/xsk.o modules/xdp-tools/lib/libxdp/sharedobjs/libxdp.o build/config.o build/cmdline.o src/xdpfw.c
In file included from src/xdpfw.c:20:
modules/xdp-tools/lib/libbpf/src/libbpf.h:70:54: warning: declaration of 'enum bpf_link_type' will not be visible outside of this function [-Wvisibility]
LIBBPF_API const char *libbpf_bpf_link_type_str(enum bpf_link_type t);
                                                     ^
1 warning generated.

I hope this helps and if you have any questions or run into other issues, please let me know!

[GermanKingYT]

XDP-Firewall# make && sudo make install
mkdir -p build/
clang -O2 -c -o build/config.o src/config.c
clang -O2 -c -o build/cmdline.o src/cmdline.c
make -C libbpf/src
make[1]: Entering directory '/root/XDP-Firewall/libbpf/src'
make[1]: ../scripts/check-reallocarray.sh: Permission denied
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/root/XDP-Firewall/libbpf/src'
mkdir -p build/
clang -lconfig -lelf -lz -I libbpf/src -o build/xdpfw libbpf/src/staticobjs/bpf_prog_linfo.o libbpf/src/staticobjs/bpf.o libbpf/src/staticobjs/btf_dump.o libbpf/src/staticobjs/btf.o libbpf/src/staticobjs/hashmap.o libbpf/src/staticobjs/libbpf_errno.o libbpf/src/staticobjs/libbpf_probes.o libbpf/src/staticobjs/libbpf.o libbpf/src/staticobjs/netlink.o libbpf/src/staticobjs/nlattr.o libbpf/src/staticobjs/str_error.o libbpf/src/staticobjs/xsk.o build/config.o build/cmdline.o src/xdpfw.c
mkdir -p build/
clang -I libbpf/src -D__BPF__ -O2 -emit-llvm -c -o build/xdpfw_kern.bc src/xdpfw_kern.c
llc -march=bpf -filetype=obj -o build/xdpfw_kern.o build/xdpfw_kern.bc
llc: error: llc: build/xdpfw_kern.bc: error: Unknown attribute kind (70) (Producer: 'LLVM13.0.1' Reader: 'LLVM 11.0.1')
make: *** [Makefile:37: xdpfw_filter] Error 1

any ideas ?

[gamemann]

@GermanKingYT Hey! It appears you have mismatched Clang and LLVM versions. Can you provide the output of the following commands please?

llc --version
clang --version

Can you also provide the Linux distro you're using and its release version/kernel? You can do that by providing the outputs of the following commands.

cat /etc/*-release

uname -a

I believe it'll be best if we started a separate issue related to this issue as well since it looks like the error you received is unrelated to the initial issue's error. I plan on closing this one soon.

[namecloudz]

i use old kernel,

clang 11
and update xdp-tools 1.4.1

but

clang -lconfig -lelf -lz -I modules/xdp-tools/lib/libbpf/src -I /usr/include -I /usr/local/include -o build/xdpfw modules/xdp-tools/lib/libbpf/src/staticobjs/bpf_prog_linfo.o modules/xdp-tools/lib/libbpf/src/staticobjs/bpf.o modules/xdp-tools/lib/libbpf/src/staticobjs/btf_dump.o modules/xdp-tools/lib/libbpf/src/staticobjs/btf.o modules/xdp-tools/lib/libbpf/src/staticobjs/gen_loader.o modules/xdp-tools/lib/libbpf/src/staticobjs/hashmap.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf_errno.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf_probes.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf.o modules/xdp-tools/lib/libbpf/src/staticobjs/linker.o modules/xdp-tools/lib/libbpf/src/staticobjs/netlink.o modules/xdp-tools/lib/libbpf/src/staticobjs/nlattr.o modules/xdp-tools/lib/libbpf/src/staticobjs/relo_core.o modules/xdp-tools/lib/libbpf/src/staticobjs/ringbuf.o modules/xdp-tools/lib/libbpf/src/staticobjs/str_error.o modules/xdp-tools/lib/libbpf/src/staticobjs/strset.o modules/xdp-tools/lib/libbpf/src/staticobjs/usdt.o modules/xdp-tools/lib/libxdp/sharedobjs/xsk.o modules/xdp-tools/lib/libxdp/sharedobjs/libxdp.o build/config.o build/cmdline.o src/xdpfw.c
In file included from src/xdpfw.c:19:
modules/xdp-tools/lib/libbpf/src/bpf.h:471:60: warning: declaration of 'struct bpf_link_info' will not be visible outside of this function [-Wvisibility]
LIBBPF_API int bpf_link_get_info_by_fd(int link_fd, struct bpf_link_info *info, __u32 *info_len);
^
In file included from src/xdpfw.c:20:
modules/xdp-tools/lib/libbpf/src/libbpf.h:70:54: warning: declaration of 'enum bpf_link_type' will not be visible outside of this function [-Wvisibility]
LIBBPF_API const char *libbpf_bpf_link_type_str(enum bpf_link_type t);
^
2 warnings generated.
/usr/bin/ld: modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf.o: in function elf_find_func_offset_from_archive': /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10893: undefined reference to zip_archive_open'
/usr/bin/ld: /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10900: undefined reference to zip_archive_find_entry' /usr/bin/ld: /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10934: undefined reference to zip_archive_close'
/usr/bin/ld: /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10934: undefined reference to zip_archive_close' /usr/bin/ld: /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10934: undefined reference to zip_archive_close'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:59: xdpfw] Error 1

how to fix? i can't change kernel but some interface issue

[gamemann]

@namecloudz The errors you're receiving are related to ZIP functions and references missing in the version of LibBPF you're using. The latest version of LibBPF appears to compile a new zip.c file into a shared/static object while this firewall currently uses an outdated version of LibBPF that does not. Since we still use raw LibBPF functionality in the XDP Firewall loader (which I should definitely look into eliminating when I have the time so that we only rely on xdp-tools independently), we manually link the LibBPF's static objects here. The new ZIP object is not linked here which is causing the error you're receiving most likely.

There are two options that will most likely resolve this issue.

The first option is downgrading the xdp-tools submodule to commit 81ea740 and the libbpf submodule (within the xdp-tools submodule) to commit a032540 (if you reinitialize the XDP firewall repository with the --recursive flag via git clone --recursive https://github.com/gamemann/XDP-Firewall.git or use git submodule update --init in a vanilla repository , it should automatically checkout these commits that compile properly for XDP Firewall).

The second option is to edit the Makefile and add $(LIBBPFSRC)/staticobjs/zip.o and any other new static objects to the LIBBPFOBJS variable here so that they are linked.

I hope this helps!

Edit
I just relooked at xdp-tools's code and wanted to note that we won't be able to rely on xdp-tools separately since xdp-tools doesn't include the BPF functionality the XDP Firewall loader requires. So we'll have to still link libbpf in the XDP Firewall loader separately for now, but we do use the LibBPF library that is cloned inside of the xdp-tools submodule. This doesn't impact the solutions above and they should still work fine, I just wanted to clarify this in the case anybody wonders why we can't rely on xdp-tools for all BPF functionality in the project.

[gamemann]

I've updated the project's xdp-tools submodule to its latest commit, cae9c91, which included linking the zip static object from libbpf. Try pulling the recent changes and building again. libbpf is now on commit fbd60db within the xdp-tools submodule.

I tested these changes in a dev environment and they appear to be working fine. The GitHub workflow I made also succeeded in building the project.

[namecloudz]

thank you,

and i will try test in 22.04

libbpf: elf: skipping unrecognized data section(7) .xdp_run_config
libbpf: elf: skipping unrecognized data section(8) xdp_metadata
libbpf: elf: skipping unrecognized data section(23) .eh_frame
libbpf: elf: skipping relo section(24) .rel.eh_frame for section(23) .eh_frame
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: prog 'xdp_pass': BPF program load failed: Invalid argument
libbpf: prog 'xdp_pass': failed to load: -22
libbpf: failed to load object '/usr/local/lib/bpf/xdp-dispatcher.o'
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
Loaded XDP program on mode DRV/native.

[gamemann]

@namecloudz You're welcome! It looks like it's working from the output you gave. Can you confirm this? If so, would it be alright to close this issue?

[pettai]

Edit I just relooked at xdp-tools's code and wanted to note that we won't be able to rely on xdp-tools separately since xdp-tools doesn't include the BPF functionality the XDP Firewall loader requires. So we'll have to still link libbpf in the XDP Firewall loader separately for now, but we do use the LibBPF library that is cloned inside of the xdp-tools submodule. This doesn't impact the solutions above and they should still work fine, I just wanted to clarify this in the case anybody wonders why we can't rely on xdp-tools for all BPF functionality in the project.

What you mean is that XDP Firewall can't be built nor use the packages version of xdp-tools at run-time?

[gamemann]

Edit I just relooked at xdp-tools's code and wanted to note that we won't be able to rely on xdp-tools separately since xdp-tools doesn't include the BPF functionality the XDP Firewall loader requires. So we'll have to still link libbpf in the XDP Firewall loader separately for now, but we do use the LibBPF library that is cloned inside of the xdp-tools submodule. This doesn't impact the solutions above and they should still work fine, I just wanted to clarify this in the case anybody wonders why we can't rely on xdp-tools for all BPF functionality in the project.

What you mean is that XDP Firewall can't be built nor use the packages version of xdp-tools at run-time?

The firewall does use xdp-tools, but I was initially hoping that we could eliminate the need of using LibBPF directly in the xdpfw.c file and replacing it with functionality from xdp-tools. However, xdp-tools doesn't provide the BPF functionality we'd need from what I've seen (e.g. adding/deleting entries from a BPF map). Therefore, we still need to link the LibBPF objects here in this firewall's Makefile.

For example, if we didn't need to link the LibBPF objects directly in our Makefile, the errors from this reply probably wouldn't have occurred since xdp-tools would have handled linking the LibBPF objects which would have included the newer zip.o object file.

I hope this clarifies what I meant, but if not, feel free to open a new issue (this one was closed, so I don't think you can reply)! 😄