RAG Embedding Pooling Type Error Resolution
Problem
Local RAG system fails with: Pooling type 'none' is not OAI compatible. Please use a different pooling type.
Error persists even after Docker image changes and RAG disabling — suggests configuration cache or hardcoded initialization.
Root Cause
Protocol mismatch between application and local inference server (LocalAI/vLLM/Ollama). Application sends embedding request with pooling: none (requesting raw hidden states), but server expects OpenAI-compatible behavior where pooling is handled automatically.
Solutions
1. Server-Side: Adjust Inference Config
Locate embedding model YAML config (e.g., text-embeddings-ada-002.yaml). Ensure embeddings: true and pooling is set to a compatible type (not none).
2. Client-Side: Override Pooling Parameter
Set environment variable EMBEDDING_POOLING_TYPE=mean or patch code to omit the pooling field entirely (let server handle it per OAI spec).
3. Docker: Clean Restart
docker-compose down
# Check for stale .env files overriding manual fixes
# Verify BASE_URL points to current inference server
docker-compose up -dKey Concept: Pooling Types
| Type | Behavior |
|---|---|
none | Returns raw token-level vectors (not OAI-compatible) |
mean | Averages all token embeddings into one sentence vector |
cls | Uses the [CLS] token embedding as sentence representation |
| OAI default | Server handles pooling internally; client just gets final vector |
Related
- jp-code-academy — Project encountering this error
- solopreneur-content-stack — Local inference server context