Can't access k3s - any suggestions?

[LMSSonos]

Hello

I have upgraded kube-vip on my k3s cluster (4 nodes, all master) - I don't have anything important on them and can easily restart from scratch. But I'm curios what the right way is to gain access to my cluster in this situation - because I made the same upgrade a few weeks ago and exactly the same happened

So when I upgraded kube-vip from v0.6.4 to the latest verison I lost access to all nodes.

running on that node kubectl get nodes I get this error

CONTAINER           IMAGE               CREATED             STATE               NAME                           ATTEMPT             POD ID              POD
2cfb525160179       4d7d4280598e3       26 minutes ago      Exited              csi-provisioner                1                   bbfa7b4860837       csi-provisioner-664cb5bdd5-brlhb
416fd6336fed2       cfc5cc58b333f       26 minutes ago      Exited              csi-snapshotter                1                   56dbeb8291ab9       csi-snapshotter-7b7db78f9-rxpmm
6242761582d74       0d52f790b5b3f       26 minutes ago      Exited              csi-attacher                   1                   5695ba6860cf4       csi-attacher-79866cdcf8-776cf
fdf8ebdd03ec7       c61bb1c17f18b       26 minutes ago      Exited              csi-resizer                    1                   19c042ce933b2       csi-resizer-64f6fb4459-7gfcs

and if I run systemctl status k3s

{"level":"info","ts":"2025-02-09T17:39:17.936024+0100","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"d52a0050b4fdb958 has received 2 MsgPreVoteResp votes and 0 vote reject>
{"level":"info","ts":"2025-02-09T17:39:18.759725+0100","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"d52a0050b4fdb958 [logterm: 17, index: 12559060, vote: d52a0050b4fdb958>
W0209 17:39:20.980877    1814 reflector.go:561] k8s.io/[email protected]/tools/cache/reflector.go:243: failed to list *v1.Node: apiserver not ready
E0209 17:39:20.980911    1814 reflector.go:158] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:243: Failed to watch *v1.Node: failed to list *v1.Node: api>
{"level":"warn","ts":"2025-02-09T17:39:21.694443+0100","caller":"etcdserver/server.go:2163","msg":"failed to publish local member to cluster through raft","local-member-id":"d52a0050b4f>
time="2025-02-09T17:39:21+01:00" level=info msg="Failed to test data store connection: failed to get etcd status: rpc error: code = Unavailable desc = connection error: desc = \"transpo>
{"level":"warn","ts":"2025-02-09T17:39:22.512531+0100","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","r>
{"level":"warn","ts":"2025-02-09T17:39:22.521959+0100","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE>

What I see in all logs is that the API server can't be reached. Should I see a api related pod when I run crictl ps?

I think kube-vip messed up the networking, because I can ping all nodes on ipv6, but not on ipv4. I can ssh all on ipv6, but not on ipv4. I saw moments, where all nodes showed the same mac address when running arp -a

Thanks for any suggestions