Description
Reminder
- I have read the README and searched the existing issues.
System Info
基于llava1.5-7b-chat微调
CUDA_VISIBLE_DEVICES=0,1 API_PORT=7864 llamafactory-cli api /data/rich/project/LLaMA-Factory-main/examples/inference/vllm.yaml
vllm.yaml
model_name_or_path: /data/rich/model/Med-v1.0
template: vicuna
infer_backend: vllm
vllm_enforce_eager: true
这种部署可以文本对话 传图片报错
Reproduction
llamafactory-cli webchat --model_name_or_path /data/rich/model/Med-v1.0 --template vicuna --visual_inputs
这种方式界面是可以的
api的方式怎么实现
llamafactory-cli api config/llava1_5_full_sft_infer.yaml 后台报错 422
{
"model": "llava-v1.5",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "请告诉我这张图像里有什么，中文回复我"
},
{
"type": "image_url",
"image_url": {
"url": "https://th.bing.com/th/id/R.6b5df1bfe0e4778a44dba0753cd169c8?rik=QRQIMqvjWRCO5Q&riu=http%3a%2f%2fpic39.nipic.com%2f20140321%2f8857347_232251363165_2.jpg&ehk=7oAaMo6LCHJc%2bqpQ0IPvcH7v69jGRQhb2vDz%2fOd5720%3d&risl=&pid=ImgRaw&r=0"
}
}
]
}
],
"temperature": 0.7,
"top_p": 1,
"logprobs": false,
"top_logprobs": 0,
"n": 1,
"max_tokens": null,
"stop": null,
"stream": false,
"presence_penalty": 0,
"frequency_penalty": 0,
"user": "string",
"repetition_penalty": 1,
"session_id": -1,
"ignore_eos": false,
"skip_special_tokens": true,
"top_k": 40
}### Expected behavior
No response
Others
No response
Activity