Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions docs/edge/ar/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1012,6 +1012,39 @@ mode: "wide"
uv add 'crewai[litellm]'
```
</Accordion>

<Accordion title="DeepInfra">
يتصل CrewAI بـ DeepInfra عبر عميل OpenAI الأصلي، لذا لا حاجة لتثبيت LiteLLM.

عيّن متغيرات البيئة التالية في ملف `.env`:
```toml Code
# Required
DEEPINFRA_API_KEY=<your-api-key>

# Optional, point at a proxy or a private deployment endpoint
DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai
```

مثال الاستخدام في مشروع CrewAI:
```python Code
llm = LLM(
model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
temperature=0.7
)
```

<Info>
ميزات DeepInfra:
- نماذج مفتوحة الأوزان (Llama وQwen وDeepSeek وGLM وMistral وغيرها) خلف API واحد متوافق مع OpenAI
- استدعاء الأدوات والبث والمخرجات المنظمة بمخطط JSON
- تسعير لكل token وطول السياق منشوران في كتالوج النماذج
</Info>

معرّفات النماذج بصيغة `org/model` أصلًا، لذا يتكوّن المرجع الكامل من ثلاثة أجزاء،
مثل `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. راجع
[كتالوج النماذج المنسّق لـ CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)،
وهو عام ولا يتطلب مصادقة، للحصول على المعرّفات الحالية وأطوال السياق والأسعار.
</Accordion>
</AccordionGroup>

## بث الاستجابات
Expand Down
33 changes: 33 additions & 0 deletions docs/edge/en/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1155,6 +1155,39 @@ In this section, you'll find detailed examples that help you select, configure,
uv add 'crewai[litellm]'
```
</Accordion>

<Accordion title="DeepInfra">
CrewAI talks to DeepInfra through its native OpenAI client, so no LiteLLM install is needed.

Set the following environment variables in your `.env` file:
```toml Code
# Required
DEEPINFRA_API_KEY=<your-api-key>

# Optional, point at a proxy or a private deployment endpoint
DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai
```

Example usage in your CrewAI project:
```python Code
llm = LLM(
model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
temperature=0.7
)
```

<Info>
DeepInfra features:
- Open-weight models (Llama, Qwen, DeepSeek, GLM, Mistral and more) behind one OpenAI-compatible API
- Tool calling, streaming and JSON schema structured outputs
- Per-token pricing and context length published in the model catalog
</Info>

Model ids are already in `org/model` form, so a full reference carries three segments,
for example `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. See the
[model catalog curated for CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),
which is public and needs no authentication, for current ids, context lengths and pricing.
</Accordion>
</AccordionGroup>

## Streaming Responses
Expand Down
33 changes: 33 additions & 0 deletions docs/edge/ko/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -755,6 +755,39 @@ CrewAI는 고유한 기능, 인증 방법, 모델 역량을 제공하는 다양
uv add 'crewai[litellm]'
```
</Accordion>

<Accordion title="DeepInfra">
CrewAI는 네이티브 OpenAI 클라이언트로 DeepInfra에 연결하므로 LiteLLM 설치가 필요하지 않습니다.

`.env` 파일에 다음 환경 변수를 설정하십시오:
```toml Code
# Required
DEEPINFRA_API_KEY=<your-api-key>

# Optional, point at a proxy or a private deployment endpoint
DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai
```

CrewAI 프로젝트에서의 예시 사용법:
```python Code
llm = LLM(
model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
temperature=0.7
)
```

<Info>
DeepInfra 특징:
- 하나의 OpenAI 호환 API로 제공되는 오픈 웨이트 모델 (Llama, Qwen, DeepSeek, GLM, Mistral 등)
- 도구 호출, 스트리밍, JSON 스키마 구조화 출력 지원
- 모델 카탈로그에 공개된 토큰 단위 가격과 컨텍스트 길이
</Info>

모델 ID는 이미 `org/model` 형식이므로 전체 참조는 세 개의 세그먼트로 구성됩니다.
예: `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. 현재 ID, 컨텍스트 길이, 가격은
인증 없이 공개된 [CrewAI용으로 선별된 모델 카탈로그](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)를
참조하세요.
</Accordion>
</AccordionGroup>

## 스트리밍 응답
Expand Down
33 changes: 33 additions & 0 deletions docs/edge/pt-BR/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -728,6 +728,39 @@ Nesta seção, você encontrará exemplos detalhados que ajudam a selecionar, co
uv add 'crewai[litellm]'
```
</Accordion>

<Accordion title="DeepInfra">
O CrewAI se comunica com o DeepInfra pelo cliente OpenAI nativo, então não é necessário instalar o LiteLLM.

Defina as seguintes variáveis de ambiente no seu arquivo `.env`:
```toml Code
# Required
DEEPINFRA_API_KEY=<your-api-key>

# Optional, point at a proxy or a private deployment endpoint
DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai
```

Exemplo de uso em seu projeto CrewAI:
```python Code
llm = LLM(
model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
temperature=0.7
)
```

<Info>
Recursos do DeepInfra:
- Modelos de pesos abertos (Llama, Qwen, DeepSeek, GLM, Mistral e outros) por trás de uma única API compatível com OpenAI
- Chamada de ferramentas, streaming e saídas estruturadas com JSON schema
- Preço por token e tamanho de contexto publicados no catálogo de modelos
</Info>

Os IDs dos modelos já estão no formato `org/model`, então uma referência completa tem três segmentos,
por exemplo `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. Consulte o
[catálogo de modelos selecionado para o CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),
que é público e não exige autenticação, para obter os IDs atuais, tamanhos de contexto e preços.
</Accordion>
</AccordionGroup>

## Respostas em streaming
Expand Down
16 changes: 16 additions & 0 deletions lib/crewai/src/crewai/llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,7 @@ def _ensure_litellm() -> bool:
"cerebras",
"dashscope",
"snowflake",
"deepinfra",
]


Expand Down Expand Up @@ -431,6 +432,13 @@ def __new__(cls, model: str, is_litellm: bool = False, **kwargs: Any) -> LLM:
provider = explicit_provider
use_native = True
model_string = model
if provider == "deepinfra" and not cls._matches_provider_pattern(
model, provider
):
raise ValueError(
"DeepInfra model ids are org/model, for example "
f"'deepseek-ai/DeepSeek-V4-Flash-0731'; got '{model}'"
)
elif "/" in model:
prefix, _, model_part = model.partition("/")

Expand All @@ -452,6 +460,7 @@ def __new__(cls, model: str, is_litellm: bool = False, **kwargs: Any) -> LLM:
"cerebras": "cerebras",
"dashscope": "dashscope",
"snowflake": "snowflake",
"deepinfra": "deepinfra",
}

canonical_provider = provider_mapping.get(prefix.lower())
Expand Down Expand Up @@ -582,6 +591,12 @@ def _matches_provider_pattern(cls, model: str, provider: str) -> bool:
# OpenRouter uses org/model format but accepts anything
return True

if provider == "deepinfra":
# DeepInfra ids are exactly org/model, so a full reference is
# deepinfra/<org>/<model>: one slash with both parts non-empty.
parts = model_lower.split("/")
return len(parts) == 2 and all(parts)
Comment thread
coderabbitai[bot] marked this conversation as resolved.

if provider == "snowflake":
return True

Expand Down Expand Up @@ -723,6 +738,7 @@ def _get_native_provider(cls, provider: str) -> type | None:
"hosted_vllm",
"cerebras",
"dashscope",
"deepinfra",
}
if provider in openai_compatible_providers:
from crewai.llms.providers.openai_compatible.completion import (
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,12 @@ class ProviderConfig:
base_url_env="DASHSCOPE_BASE_URL",
api_key_required=True,
),
"deepinfra": ProviderConfig(
base_url="https://api.deepinfra.com/v1/openai",
api_key_env="DEEPINFRA_API_KEY",
base_url_env="DEEPINFRA_BASE_URL",
api_key_required=True,
),
}

_OLLAMA_DEFAULT_PORT = 11434
Expand Down Expand Up @@ -142,6 +148,7 @@ class OpenAICompatibleCompletion(OpenAICompletion):
- hosted_vllm: vLLM server (https://github.com/vllm-project/vllm)
- cerebras: Cerebras (https://cerebras.ai)
- dashscope: Alibaba Dashscope/Qwen (https://dashscope.aliyun.com)
- deepinfra: DeepInfra (https://deepinfra.com)

Example:
# Using provider prefix
Expand Down
82 changes: 82 additions & 0 deletions lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,14 @@ def test_dashscope_config(self):
assert config.api_key_env == "DASHSCOPE_API_KEY"
assert config.api_key_required is True

def test_deepinfra_config(self):
"""Test DeepInfra provider configuration."""
config = OPENAI_COMPATIBLE_PROVIDERS["deepinfra"]
assert config.base_url == "https://api.deepinfra.com/v1/openai"
assert config.api_key_env == "DEEPINFRA_API_KEY"
assert config.base_url_env == "DEEPINFRA_BASE_URL"
assert config.api_key_required is True


class TestNormalizeOllamaBaseUrl:
"""Tests for _normalize_ollama_base_url helper."""
Expand Down Expand Up @@ -162,6 +170,15 @@ def test_missing_required_api_key_raises_error(self):
if original is not None:
os.environ[env_key] = original

def test_deepinfra_missing_api_key_names_env_var(self):
"""DeepInfra requires a key; the error tells the user which env var to set."""
with patch.dict(os.environ, {}, clear=True), pytest.raises(
ValueError, match="DEEPINFRA_API_KEY"
):
OpenAICompatibleCompletion(
model="deepseek-ai/DeepSeek-V4-Flash-0731", provider="deepinfra"
)

def test_api_key_from_env(self):
"""Test API key is read from environment variable."""
with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key-from-env"}):
Expand Down Expand Up @@ -312,6 +329,71 @@ def test_llm_creates_openai_compatible_for_dashscope_non_qwen(self):
assert llm.provider == "dashscope"
assert llm.base_url == "https://my-dashscope.example.com/v1"

def test_llm_creates_openai_compatible_for_deepinfra(self):
"""Test LLM factory creates OpenAICompatibleCompletion for DeepInfra."""
with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}):
llm = LLM(model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731")
assert isinstance(llm, OpenAICompatibleCompletion)
assert llm.provider == "deepinfra"
# DeepInfra ids are org/model, so the part after the provider
# prefix is itself two segments and must reach the API intact.
assert llm.model == "deepseek-ai/DeepSeek-V4-Flash-0731"
assert llm.base_url == "https://api.deepinfra.com/v1/openai"

def test_deepinfra_base_url_env_override(self):
"""DEEPINFRA_BASE_URL redirects DeepInfra to a proxy or private endpoint."""
with patch.dict(
os.environ,
{
"DEEPINFRA_API_KEY": "test-key",
"DEEPINFRA_BASE_URL": "https://proxy.example.com/v1/openai",
},
):
llm = LLM(model="deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct")
assert isinstance(llm, OpenAICompatibleCompletion)
assert llm.base_url == "https://proxy.example.com/v1/openai"

def test_llm_creates_openai_compatible_for_deepinfra_explicit_provider(self):
"""Explicit provider="deepinfra" routes natively without the model prefix."""
with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}):
llm = LLM(model="deepseek-ai/DeepSeek-V4-Flash-0731", provider="deepinfra")
assert isinstance(llm, OpenAICompatibleCompletion)
assert llm.provider == "deepinfra"
assert llm.model == "deepseek-ai/DeepSeek-V4-Flash-0731"

@pytest.mark.parametrize(
"model",
[
"deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
"DeepSeek-V4-Flash-0731",
"deepseek-ai/DeepSeek-V4-Flash-0731/",
],
)
def test_llm_explicit_deepinfra_provider_rejects_malformed_model(self, model):
"""provider="deepinfra" applies the org/model check before building a client."""
with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}), pytest.raises(
ValueError, match="org/model"
):
LLM(model=model, provider="deepinfra")

@pytest.mark.parametrize(
("model", "expected"),
[
("deepseek-ai/DeepSeek-V4-Flash-0731", True),
("Qwen/Qwen3.5-27B", True),
("meta-llama/Llama-4-Scout-17B-16E-Instruct", True),
("", False),
("qwen", False),
("Qwen/", False),
("/Qwen3.5-27B", False),
("deepseek-ai/DeepSeek-V4-Flash-0731/extra", False),
("deepseek-ai/DeepSeek-V4-Flash-0731/", False),
],
)
def test_deepinfra_requires_an_org_and_a_model(self, model, expected):
"""DeepInfra references are exactly org/model: one slash, both parts present."""
assert LLM._matches_provider_pattern(model, "deepinfra") is expected

def test_llm_with_explicit_provider(self):
"""Test LLM with explicit provider parameter."""
with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key"}):
Expand Down