Raspberry PI's ISP (libcamera/picamera2) crops IMX500's inference detection region when cropping parameters of feed are inputted with cropping coordinates.

[vikasmalitest]

Hello,

I am using picamera2 library (Raspberry Pi's ISP) to crop and show video feed of IMX500 based on people detection (person detected coordinates) obtained from AI inference on IMX500 using object detection model.

Ideal situation, when I am using ISP cropping method using picamera2 for showing cropped video feed output (x,y,w,h) of custom coordinates, it should show cropped video feed and at the same time, AI detection running on the full resolution which is 4056 * 3040. But this is not the case. When I feed (x,y,w,h) coordiantes for showing cropped video streaming, it runs AI inferencing on (x,y,w,h) coordinates rather than running inference on full resolution (4056*3040).

I am using picamera2.setcontrols("ScalerCrop":(x,y,w,h,))
Please let me know if I am missing something or API dedictaed for above application or this is a missing functionality in using ISP of PI.

[naushir]

As you've said, ScalerCrop will setup the crop on the ISP frame, but you can also setup the IMX500 inference crop window using the IMX500.set_inference_roi_abs API call:

picamera2/picamera2/devices/imx500/imx500.py

Line 494 in fb031cb

def set_inference_roi_abs(self, roi: tuple):

The ROI tuple used here would be relative to the full sensor resolution (0,0,4056,3040). Hopefully this should give you are after?

[naushir]

Also moving this issue to the picamera2 repo.