Embeddings & Similarity

Generate vector embeddings from text and compute similarity between them. On startup, (llm/auto-configure) registers available providers and selects the first in this order: Jina, Voyage, Cohere, Nomic, OpenAI, Mistral, Fireworks, Together, Gemini, OpenRouter, DeepInfra, Azure, Cloudflare, Alibaba, TEI, then local Ollama. Incomplete account configurations are skipped unless explicitly selected. Set SEMA_EMBEDDING_PROVIDER / SEMA_EMBEDDING_MODEL or --embedding-provider / --embedding-model to override that selection. Reranking has independent SEMA_RERANK_PROVIDER / SEMA_RERANK_MODEL and --rerank-provider / --rerank-model overrides.

Configuration

llm/configure-embeddings

Configure a dedicated embedding provider separately from the chat provider — so you can use one provider for chat and another for embeddings. Pass :default-model to pick the model (otherwise each provider uses its default: jina-embeddings-v3, voyage-3, or text-embedding-3-small):

(llm/configure-embeddings :voyage
  {:api-key (env "VOYAGE_API_KEY") :default-model "voyage-3-large"})

(llm/configure-embeddings :ollama
  {:model "embeddinggemma" :host "http://localhost:11434"})

(llm/configure-embeddings :together
  {:api-key (env "TOGETHER_API_KEY")})

(llm/configure-embeddings :fireworks
  {:api-key (env "FIREWORKS_API_KEY")})

;; OpenAI-compatible embedding provider, with a model and optional base URL
(llm/configure-embeddings :openai
  {:api-key (env "OPENAI_API_KEY") :default-model "text-embedding-3-large"})

Configuration does not replace the chat provider, even when both use the same name.

Model precedence is per-call :model, configured :default-model, configured :model, <PROVIDER>_EMBEDDING_MODEL, then the provider default. Azure and TEI require an explicit deployment/model name or their model environment variable. :base-url overrides <PROVIDER>_BASE_URL; Ollama also accepts :host and reads OLLAMA_HOST. Custom OpenAI-compatible providers require an explicit :base-url. API keys default to the provider's environment variable; Gemini also accepts GOOGLE_API_KEY.

These options can be set at configuration time and overridden by each llm/embed call:

Option	Meaning
:dimensions	Positive output dimension, where supported by the model
:input-type	Cohere / Voyage input type, such as "search_query" / "query"
:task	Jina task, such as "retrieval.query", or Nomic task, such as "search_query"
:long-text-mode	Nomic: "truncate" or "mean"
:max-tokens-per-text	Nomic: positive token limit per text
:truncate	Boolean; Ollama truncation, Voyage truncation, or Cohere END/NONE
:keep-alive	Ollama model lifetime, e.g. "5m" or 0
:timeout	Positive request timeout in milliseconds

Use options supported by the selected provider/model. Configure :rerank-model and :top-k separately for reranking; calls can override them with :model, :top-k, :provider, and :timeout.

Known protocols reject unsupported options before making an HTTP request. Mistral and Voyage translate :dimensions to output_dimension; OpenAI-compatible dimension options use dimensions. Cloudflare's BGE preset and the TEI preset use the served model's full vector width and do not accept :dimensions.

Ollama uses its native embedding API, POST /api/embed. Install the default model first with ollama pull embeddinggemma. Omitting :truncate and :keep-alive preserves the server defaults. Ollama has no native reranking endpoint.

Nomic hosted embeddings

The :nomic preset uses Nomic's native text embedding API, POST https://api-atlas.nomic.ai/v1/embedding/text. It reads NOMIC_API_KEY, defaults to nomic-embed-text-v1.5, and accepts NOMIC_BASE_URL or :base-url as an API root ending in /v1. It supports embeddings only; no hosted Nomic reranking endpoint is configured.

(llm/configure-embeddings :nomic
  {:api-key (env "NOMIC_API_KEY") :task "search_document"})
(define docs (llm/embed ["Cats are mammals." "Rocks are minerals."]))
(define query (llm/embed "Which animals are mammals?" {:task "search_query"}))

Use :task with "search_document", "search_query", "classification", or "clustering". The default is "search_document". :dimensions maps to dimensionality (64–768 for v1.5); :long-text-mode maps to long_text_mode, and :max-tokens-per-text maps to the token limit. The default long-text mode is "truncate", matching Nomic's Python SDK. :truncate #t is accepted as an alias for :long-text-mode "truncate"; #f does not mean "mean" and is rejected. Use :task, not :input-type. :keep-alive applies to Ollama, not Nomic.

The adapter sends at most ten texts per request, preserves input order, and sums input token usage across requests. :timeout applies to each request. A failed request returns an error, not a partial result. Embedding calls do not retry automatically; a repeated call can charge again for earlier successful requests.

The adapter has deterministic tests against the documented schema and passed live API verification with nomic-embed-text-v1.5 on 2026-10-21.

OpenRouter and DeepInfra

Create keys in OpenRouter or DeepInfra, then set OPENROUTER_API_KEY or DEEPINFRA_API_KEY. The defaults are openai/text-embedding-3-small and Qwen/Qwen3-Embedding-0.6B, respectively. These presets configure embeddings independently of chat and do not configure reranking.

(llm/configure-embeddings :openrouter {})
(llm/embed "A short document.")

(llm/configure-embeddings :deepinfra {})
(llm/embed ["A short document." "Et kort dokument."])

Their API roots are https://openrouter.ai/api/v1 and https://api.deepinfra.com/v1/openai. Override them with :base-url, OPENROUTER_BASE_URL, or DEEPINFRA_BASE_URL. Model-specific task instructions remain part of the input text; a hosted Qwen model does not automatically use Jina or Cohere task parameters.

Azure OpenAI v1

Create an Azure account and an embedding model deployment. Set the API key, resource endpoint, and deployment name:

export AZURE_OPENAI_API_KEY="your-resource-key"
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com"
export AZURE_OPENAI_EMBEDDING_DEPLOYMENT="your-embedding-deployment"

(llm/configure-embeddings :azure {})
(llm/embed "A short document.")

The request's model is the deployment name. Sema appends /openai/v1 to the resource endpoint and uses bearer API-key authentication. AZURE_BASE_URL, AZURE_OPENAI_BASE_URL, or :base-url can instead supply the complete API root. AZURE_API_KEY is an API-key alias. This preset uses the current v1 API; it does not construct older dated deployment routes or refresh Microsoft Entra tokens.

Cloudflare Workers AI

Create an account, then follow the Workers AI token setup. The token needs Workers AI Read and Edit permissions. Set CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID. CLOUDFLARE_API_KEY is accepted as a token alias.

(llm/configure-embeddings :cloudflare {})
(llm/embed "A short document.")

The default model is @cf/baai/bge-m3. The API root is https://api.cloudflare.com/client/v4/accounts/ACCOUNT_ID/ai/v1. :account-id overrides the account environment variable. :base-url or CLOUDFLARE_BASE_URL supplies a complete root instead, without requiring an account ID. This is the Workers AI compatible route, not the native /ai/run API.

Alibaba Model Studio

Create a pay-as-you-go Model Studio API key. Use ALIBABA_API_KEY or DASHSCOPE_API_KEY. Set ALIBABA_BASE_URL or DASHSCOPE_BASE_URL to the compatible API root for the key's region and workspace. For example, Singapore uses https://WORKSPACE_ID.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1. Use the embedding API reference to select the matching regional root. Sema does not guess a region or workspace.

(llm/configure-embeddings :alibaba {:dimensions 256})
(llm/embed ["A short document." "Et kort dokument."])

The default is text-embedding-v4. Sema splits v3/v4 inputs into requests of at most ten texts, and qwen3.7-text-embedding inputs into requests of at most twenty. The per-call model selects the limit. Results preserve input order and sum token usage across requests. A failed chunk returns an error; no partial result is returned. Earlier successful chunks can still incur charges. This preset returns dense float vectors; it does not expose the native DashScope sparse-vector API.

Self-hosted TEI

The :tei preset connects to Hugging Face Text Embeddings Inference. The default API root is http://127.0.0.1:8080/v1; override it with TEI_BASE_URL or :base-url. Set TEI_MODEL or :model to the server's model ID or --served-model-name alias. The request does not load or switch models on the server. Authentication is optional; set TEI_API_KEY when the server requires it.

On Apple Silicon, use the native Metal installation:

brew install text-embeddings-inference
text-embeddings-router --model-id BAAI/bge-small-en-v1.5 \
  --hostname 127.0.0.1 --port 8080 --max-client-batch-size 32

In another terminal:

export TEI_MODEL="BAAI/bge-small-en-v1.5"
sema --embedding-provider tei -e '(map embedding/length (llm/embed ["hello" "hei"]))'

This model produces 384-dimensional vectors. For NVIDIA GPUs, the official container setup supports models such as Qwen/Qwen3-Embedding-0.6B. Select the container for your hardware from the supported hardware list. The cpu-1.9 image is published for x86-64; use the native Metal setup on Apple Silicon.

(llm/configure-embeddings :tei
  {:model "BAAI/bge-small-en-v1.5" :base-url "http://127.0.0.1:8080/v1"})
(llm/embed ["hello" "hei"])

Sema uses batches of at most 32 texts. Keep the server's --max-client-batch-size at least 32. This preset exposes the compatible dense-text endpoint, without native sparse output, reranking, or per-request pooling controls. Model-specific query/document prefixes and normalization must match the model and index setup.

Embedding cassette keys include the resolved provider, endpoint, model, and options. Re-record older embedding tapes after upgrading.

Generating Embeddings

llm/embed

Generate an embedding for a string or a list of strings. Returns a bytevector containing densely-packed f64 values in little-endian format. This format avoids per-element unboxing overhead for similarity computations compared to a list of floats.

;; Single embedding (returns a bytevector)
(define v1 (llm/embed "hello world"))

;; Pick the model per call with an options map
(llm/embed "hello world" {:provider :openai :model "text-embedding-3-small"
                         :dimensions 256 :timeout 30000})

;; Batch embeddings
(llm/embed ["cat" "dog" "fish"])       ; => list of bytevectors

Embedding Accessors

embedding/length

Returns the number of dimensions (f64 elements) in an embedding bytevector.

(define v (llm/embed "hello"))
(embedding/length v)                   ; => 1024 (depends on provider)

embedding/ref

Access a specific dimension by index.

(define v (llm/embed "hello"))
(embedding/ref v 0)                    ; => 0.0123 (first dimension)

embedding/->list

Convert an embedding bytevector to a list of floats (useful for interop).

(define v (llm/embed "hello"))
(embedding/->list v)                   ; => (0.0123 -0.0456 ...)

embedding/list->embedding

Convert a list of numbers to an embedding bytevector.

(define v (embedding/list->embedding '(0.1 0.2 0.3)))
(embedding/length v)                   ; => 3

Computing Similarity

llm/similarity

Compute cosine similarity between two embedding vectors. Returns a value between -1.0 and 1.0. Accepts both bytevectors (fast path) and lists of floats (backward compatible).

(define v1 (llm/embed "hello world"))
(define v2 (llm/embed "hi there"))
(llm/similarity v1 v2)                 ; => 0.87 (cosine similarity)

;; Also works with plain lists
(llm/similarity '(0.1 0.2 0.3) '(0.4 0.5 0.6))

Reranking

llm/rerank

Reorder a list of candidate documents by their relevance to a query using a hosted cross-encoder reranker (Cohere, Jina, Voyage, Together AI, or Fireworks AI — the same API key you already use for embeddings; see Supported Embedding Providers below for setup). Where llm/similarity / vector-store/search embed the query and documents independently (a bi-encoder), a reranker reads the query and each document together, so it's far more precise. The standard pattern is to retrieve a generous shortlist by vector search, then rerank it to the best few.

(llm/rerank "how do I read a file?"
            (list "vectors are cool" "use file/read to read a file" "unrelated trivia")
            {:top-k 2})
;; => ({:index 1 :score 0.91 :document "use file/read to read a file"} ...)

Returns {:index :score :document} maps, highest relevance first; :index points back into the input list. Options: :top-k, :model, :timeout, and :provider (:cohere / :jina / :voyage / :together / :fireworks). See the RAG guide for the full retrieve → rerank → answer pipeline.

Token Counting

llm/token-count

Estimate the number of tokens in a string or list of strings. Uses a heuristic (chars/4) — no tokenizer dependency required.

(llm/token-count "hello world")           ; => 3
(llm/token-count '("hello" "world"))      ; => sum of individual counts

llm/token-estimate

Returns a detailed estimate map with the token count and the estimation method used.

(llm/token-estimate "hello world")
; => {:method "chars/4" :tokens 3}

Supported Embedding Providers

Provider	Env Variable	Reranking	Default model
Jina	JINA_API_KEY	✅	jina-embeddings-v3
Voyage	VOYAGE_API_KEY	✅	voyage-3
Cohere	COHERE_API_KEY	✅	embed-english-v3.0
Nomic	NOMIC_API_KEY	—	nomic-embed-text-v1.5
Together AI	TOGETHER_API_KEY	✅	BAAI/bge-base-en-v1.5
Fireworks AI	FIREWORKS_API_KEY	✅	fireworks/qwen3-embedding-8b
OpenAI	OPENAI_API_KEY	—	text-embedding-3-small
Ollama	OLLAMA_HOST (optional)	—	embeddinggemma
Mistral	MISTRAL_API_KEY	—	mistral-embed
Gemini	GEMINI_API_KEY / GOOGLE_API_KEY	—	gemini-embedding-001
OpenRouter	OPENROUTER_API_KEY	—	openai/text-embedding-3-small
DeepInfra	DEEPINFRA_API_KEY	—	Qwen/Qwen3-Embedding-0.6B
Azure	AZURE_OPENAI_API_KEY / AZURE_API_KEY	—	Required deployment name
Cloudflare	CLOUDFLARE_API_TOKEN / CLOUDFLARE_API_KEY	—	@cf/baai/bge-m3
Alibaba	ALIBABA_API_KEY / DASHSCOPE_API_KEY	—	text-embedding-v4
TEI	TEI_MODEL, optional TEI_API_KEY	—	Required served model name

See Provider Management for the full provider capability table.