Local RAG Acceleration

Local Embedding Transformer

Orcha Agent OS includes an optional self-hosted Python FastAPI Sentence Transformer microservice (orcha-embedding-transformer) to generate dense schema vector embeddings locally with zero cloud API costs.

Overview

Large enterprise databases often contain thousands of columns and tables. To prune schemas and recall past natural-language-to-SQL pairs efficiently, Orcha embeds schema descriptions using the lightweight all-MiniLM-L6-v2 transformer model.

Why Local Embeddings?

Zero External Cost

Generate millions of tokens of schema embeddings without paying per-token API fees.

Air-Gapped Privacy

Database table names, column descriptions, and business glossary data never leave your network.

Sub-10ms Inference

Local CPU/GPU inference provides immediate response times for real-time schema vector search.

Docker Setup

Start the local embedding container (mapped to port 5001):

bash

docker-compose up --build -d orcha-embeddings

API Specification

Endpoint: POST http://localhost:5001/embed

bash

curl -X POST http://localhost:5001/embed \
  -H "Content-Type: application/json" \
  -d '{"texts": ["customer billing revenue quarterly total"]}'

Returns a 384-dimensional float array vector ready for indexing in Convex Vector Search.

Next.js Environment Configuration

Add the local service URL to your .env.local file:

env

EMBEDDING_SERVICE_URL="http://localhost:5001"
EMBEDDING_PROVIDER="local"