max_tokens args doesn't work on stanford_town

[Fuyubai]

Bug description
max_tokens args doesn't work on STAction._run_gpt35_max_tokens()

class STAction(Action):
    async def _aask(self, prompt: str) -> str:
        return await self.llm.aask(prompt)

    async def _run_gpt35_max_tokens(self, prompt: str, max_tokens: int = 50, retry: int = 3):
        for idx in range(retry):
            try:
                tmp_max_tokens_rsp = getattr(config.llm, "max_token", 1500)
                setattr(config.llm, "max_token", max_tokens)
                self.llm.use_system_prompt = False  # to make it behave like a non-chat completions

                llm_resp = await self._aask(prompt)

                setattr(config.llm, "max_token", tmp_max_tokens_rsp)
                logger.info(f"Action: {self.cls_name} llm _run_gpt35_max_tokens raw resp: {llm_resp}")
                if self._func_validate(llm_resp, prompt):
                    return self._func_cleanup(llm_resp, prompt)
            except Exception as exp:
                logger.warning(f"Action: {self.cls_name} _run_gpt35_max_tokens exp: {exp}")
                time.sleep(5)
        return self.fail_default_resp

Actually, self.llm module use self.llm.config rather than config.llm, even though they are both same class type (id is not same)

Bug solved method
Simply, just replace config.llm to self.llm.config, max_tokens can work

But I think self.llm.config has same class type with config.llm, they should be same thing (id is same), maybe something wrong in initialization these class

• LLM type and model name: openai api deployed by vllm
• Python version: 3.10
• MetaGPT version or branch: main (4954729)

[iorisa]

The Action class inherits from the ContextMixin class.
The usage of the ContextMixin class is somewhat complex. It is recommended that you use a dual LLM object.
Here is the example:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from typing import Optional

import pytest

from metagpt.actions import Action
from metagpt.provider.base_llm import BaseLLM
from metagpt.provider.llm_provider_registry import create_llm_instance


class MockAction(Action):
    llm1: Optional[BaseLLM] = None

    def get_llm_max_token(self) -> int:
        return self.llm.config.max_token

    def get_llm1_max_token(self, default_max_token:int = 50) -> int:
        if not self.llm1:
            llm_config = self.llm.config.model_copy(deep=True)
            llm_config.max_token = default_max_token
            self.llm1 = create_llm_instance(llm_config)
        return self.llm1.config.max_token

@pytest.mark.asyncio
async def test_args():
    action = MockAction()
    assert action.get_llm_max_token() != action.get_llm1_max_token()

if __name__ == "__main__":
    pytest.main([__file__, "-s"])

This is the effect when it is running:

[Fuyubai]

Yeah, i get what you mean.

But the STAction is from official example of MetaGPT

from metagpt.config2 import config

In this example, they import global config to control self.llm, and set max_tokens through config.llm, but actually self.llm is controlled by self.llm.config

It will lead that llm output many many tokens even though I set max_tokens = 50, so I recommend you can fix this bug

[iorisa]

Sorry, I misunderstood. I'll fix it.