[Linux] `psutil.pids()` return an empty list

[BlueCat-de]

Summary

• OS: { ubuntu }
• Architecture: { 64bit }
• Psutil version: { 6.1.0 }
• Python version: { Python 3.9.21 }
• Type: { core }

Description

{{{
_psplatform.pids() = [], raising the following error:

Traceback (most recent call last):
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
    return _target_(*args, **kwargs)
  File "/root/projects/E2E/DiffusionDrive/navsim/planning/utils/multithreading/worker_ray_no_torch.py", line 125, in __init__
    super().__init__(self.initialize())
  File "/root/projects/E2E/DiffusionDrive/navsim/planning/utils/multithreading/worker_ray_no_torch.py", line 137, in initialize
    return initialize_ray(
  File "/root/projects/E2E/DiffusionDrive/navsim/planning/utils/multithreading/worker_ray_no_torch.py", line 78, in initialize_ray
    ray.init(
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/ray/_private/worker.py", line 1633, in init
    bootstrap_address = services.canonicalize_bootstrap_address(address, _temp_dir)
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/ray/_private/services.py", line 548, in canonicalize_bootstrap_address
    addr = get_ray_address_from_environment(addr, temp_dir)
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/ray/_private/services.py", line 409, in get_ray_address_from_environment
    gcs_addrs = find_gcs_addresses()
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/ray/_private/services.py", line 380, in find_gcs_addresses
    return _find_address_from_flag("--gcs-address")
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/ray/_private/services.py", line 343, in _find_address_from_flag
    pids = psutil.pids()
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/psutil/__init__.py", line 1467, in pids
    _LOWEST_PID = ret[0]
IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/projects/E2E/DiffusionDrive/navsim/planning/script/run_dataset_caching.py", line 105, in <module>
    main()
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/main.py", line 90, in decorated_main
    _run_hydra(
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/utils.py", line 389, in _run_hydra
    _run_app(
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/utils.py", line 452, in _run_app
    run_and_report(
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/utils.py", line 216, in run_and_report
    raise ex
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/utils.py", line 213, in run_and_report
    return func()
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/utils.py", line 453, in <lambda>
    lambda: hydra.run(
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/root/projects/E2E/DiffusionDrive/navsim/planning/script/run_dataset_caching.py", line 77, in main
    worker: WorkerPool = instantiate(cfg.worker)
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 222, in instantiate
    return instantiate_node(
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 339, in instantiate_node
    return _call_target(_target_, partial, args, kwargs, full_key)
  File "/high_perf_store/surround-view/cqm/envs/diffusiondrive/lib/python3.9/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error in call to target 'navsim.planning.utils.multithreading.worker_ray_no_torch.RayDistributedNoTorch':
IndexError('list index out of range')
full_key: worker

}}}

[giampaolo]

That's weird. It means psutil.pids() return an empty list. Pids list is get by doing a listdir of files under /proc. I assume on your system this directory is empty. Can you verify it with ls -la /proc?