Integration for Azure OpenAI Endpoints IDEA (And generic models with parameter injection)

[BubuDavid]

What I wanted and what I knew

Hi there. As a lot of people on this community I wondered if I could connect to an Azure OpenAI Endpoint and create a fast-agent that interacts with it. It is easy to get a chat completion using the AzureOpenAI specialized client from openai python sdk library, but given the fact that on fast-agent, currently there is no way to specify custom clients, I experimented a little and discover that the way of get chat completion from an Azure Endpoint using the default OpenAI client was doing something like this:

def main():
    client = OpenAI(base_url=azure_base_url, api_key=azure_api_key)

    response = client.chat.completions.create(
        messages=[{"role": "user", "content": "Hi there"}],
        model="gpt-4o",
        extra_query={"api-version": "<actual-api-version>"}, # 👈 This is an important part for Azure apparently
    )

The problem and my solution

Looking at the current code 0.2.14 I was not able to find a way to "inject" that extra parameter called extra_query with the default config files builder, so what I did was add this code to be able to do it:

# src/mcp_agent/llm/providers/augmented_llm_openai.py.generate_internal:172
if openai_config := self.context.config.openai:
  if hasattr(openai_config, "extra_params"):
    arguments = {**arguments, **openai_config.extra_params}

So, if we write our config file with the following structure, specifically for openai (but it works for generic too):

# fastagent.secrets.yaml
# OpenAI provider configuration with parameter injection inside the request calls
openai:
  api_key: "<azure-api-key>"
  base_url: "https://<endpoint-base-url>.cognitiveservices.azure.com/openai/deployments/<deployment-name>"
  extra_params:
    extra_query:
      api-version: "<api-version>"

That passes, the complete object inside extra_params key, to the variable called arguments, and that injects arguments in the openai_client.chat.completions.create method. So, now I can call Azure Endpoints with specific parameters.

Generics

I haven't experimented with generic models such as grok or ollama models, but I know that you need certain control over the parameters passed to the requests, and this little change actually solves that problem. I haven't run any test or considered other options, that is why this is a discussion and not a PR haha I do not know if this change breaks something else.

Just wanted to share in case this is useful for someone, or if I just wasted my afternoon because there was a more direct and actually supported way of doing this... that's valid too.

[evalstate]

Hi @BubuDavid - this area has changed a bit in the branch, the most relevant lines are here:

fast-agent/src/mcp_agent/llm/providers/augmented_llm_openai.py

Lines 104 to 112 in 886f0c7

	def _openai_client(self) -> OpenAI:
	try:
	return OpenAI(api_key=self._api_key(), base_url=self._base_url())
	except AuthenticationError as e:
	raise ProviderKeyError(
	"Invalid OpenAI API key",
	"The configured OpenAI API key was rejected.\n"
	"Please check that your API key is valid and not expired.",
	) from e

and here:

fast-agent/src/mcp_agent/llm/providers/augmented_llm_openai.py

Lines 320 to 348 in 886f0c7

	def _prepare_api_request(
	self, messages, tools, request_params: RequestParams
	) -> dict[str, str]:
	# Create base arguments dictionary
	# overriding model via request params not supported (intentional)
	base_args = {
	"model": self.default_request_params.model,
	"messages": messages,
	"tools": tools,
	}
	if self._reasoning:
	base_args.update(
	{
	"max_completion_tokens": request_params.maxTokens,
	"reasoning_effort": self._reasoning_effort,
	}
	)
	else:
	base_args["max_tokens"] = request_params.maxTokens
	if tools:
	base_args["parallel_tool_calls"] = request_params.parallel_tool_calls
	arguments: Dict[str, str] = self.prepare_provider_arguments(
	base_args, request_params, self.OPENAI_EXCLUDE_FIELDS.union(self.BASE_EXCLUDE_FIELDS)
	)
	return arguments

This gives us a couple of options. I think it would probably be best to have an AzureOpenAi Provider (e.g. Provider.AZURE_OPEN_AI) which has that as an extra configuration option in fastagent.config.yaml. With the recent changes this should only need to be very few lines of code (e.g. https://github.com/evalstate/fast-agent/blob/fix/test-structured/src/mcp_agent/llm/providers/augmented_llm_deepseek.py).

I think you're on the Discord, so if you want to coordinate updating/testing that'd be appreciated 👍.

[aubhadia]

@evalstate Any working example with Azure OpenAI?