BGE-M3 (embeddings)
BAAI · China · MIT
Commercial use: Yes — free for commercial use
MIT. The reranker sibling (bge-reranker-v2-m3) is Apache 2.0. Both unrestricted.
The model that does the searching in an AI-search or document-question system. Not the one that writes the answer — the one that finds the right pages first.
Why it should matter to you
Most "our AI does not know our business" problems are retrieval problems, and retrieval quality is set here rather than by which chatbot you bought. It is also the cheapest part of the stack to self-host, which means your documents can stay inside your network even if the answering model does not.
How it connects to our work
Every retrieval system we build runs an embedding model plus a reranker, and the reranker is the step teams most often skip. It costs very little and it is usually the difference between a search that feels useful and one that does not.
From our field notesThe subscription bought a reader, not a library. This is the part that builds the library.
Advantages
- The default open embedding model for retrieval — 38M downloads and rising
- Handles 100+ languages, dense and sparse retrieval in one model
- Tiny and cheap to self-host, so your documents never leave your network to be embedded
- Pairs with bge-reranker-v2-m3, which reliably improves retrieval quality for very little compute
Disadvantages
- 8K context per chunk, so long documents must be split sensibly
- Generic embeddings place domain jargon poorly — your industry's vocabulary is not where the model expects
- Being a Chinese-origin model, it draws the same procurement questions as the LLMs, despite never seeing a prompt
Reach for it when
The retrieval half of any RAG system. This is the component doing the actual searching, and it is almost always the one worth self-hosting first.
Where it falls down
Domain-specific matching out of the box. If your sector reuses common words with uncommon meanings, expect to add a reranker and a domain vocabulary rather than swapping in a bigger embedding model.
Running it
CPU is fine. Any GPU is instant. See the hardware sizing tables for how that maps to specific chips and cards, and the quantisation guide for what you give up at each bit width.
Which cloud instance actually runs this, and what it costs
At Q4_K_M the weights are about 0.5 GB, so 20 catalogued instances fit with room for context, of which the 8 best value are shown. Cheapest per token is Hetzner GEX45 (~$0.06 per million output tokens at an estimated 1344 tok/s). Fastest on a single card is Azure NCads H100 v5 at roughly 6700 tok/s for $0.29 per million tokens, which is the speed-versus-cost trade in one line.
| Instance | GPU | VRAM | Bandwidth | Est. tok/s | $/hr | $/M tokens |
|---|---|---|---|---|---|---|
| Hetzner GEX45 | RTX PRO 4000 Blackwell | 24 GB | 672 GB/s | ~1344 | $0.290 | $0.06 |
| RunPod A100 PCIe | A100 80GB | 80 GB | 1935 GB/s | ~3870 | $1.190 | $0.09 |
| Hetzner GEX131 | RTX PRO 6000 Blackwell Max-Q | 96 GB | 1792 GB/s | ~3584 | $1.420 | $0.11 |
| RunPod L40S | L40S | 48 GB | 864 GB/s | ~1728 | $0.790 | $0.13 |
| RunPod H100 PCIe | H100 PCIe | 80 GB | 2000 GB/s | ~4000 | $1.990 | $0.14 |
| Lambda A100 40GB | A100 40GB | 40 GB | 1555 GB/s | ~3110 | $1.990 | $0.18 |
| Lambda H100 PCIe | H100 PCIe | 80 GB | 2000 GB/s | ~4000 | $3.290 | $0.23 |
| AWS g5.xlarge | A10G | 24 GB | 600 GB/s | ~1200 | $1.006 | $0.23 |
Token generation on a single request is memory-bandwidth-bound, not compute-bound: the GPU must read every active weight from memory for each token it emits. So the ceiling is roughly GPU memory bandwidth divided by the size of the active weights. A 16 GB model on a 300 GB/s L4 tops out near 19 tokens/sec; the same model on a 2039 GB/s A100 tops out near 127. Figures on model pages are that arithmetic, computed from the bandwidth and weight-size columns here. They are a ceiling for one request at batch size 1, before serving overhead, so treat them as an upper bound for comparing instances rather than a benchmark. Batching raises total throughput well above this and does not raise per-request speed.
For identical silicon, the hyperscalers charge roughly 2 to 5 times what the specialist GPU clouds charge. An A100 80GB is about $3.43/GPU-hr on AWS, $3.67 on Azure and $5.03 on Google Cloud, against $1.19-1.39 on RunPod and $1.99 on Lambda. That spread is the single largest cost lever in a self-hosted inference budget, and it is larger than any saving from picking a smaller model.
Three costs that are absent from every headline rate: egress, storage for the weights, and idle time. A 70B model at Q4 is tens of gigabytes that must be pulled to the instance on every cold start, and an endpoint billed hourly is billed while it sits idle waiting for a request. Rates read 2026-09-06; check each provider's live pricing on the directory hub.
Free tiers carrying this model
| Provider | Type | The catch | Live limits |
|---|---|---|---|
| Hugging Face Inference Providers | Aggregator | The free credit is genuinely small — this is for trying a model, not running one. The Hub's real value here is free unlimited weight downloads. | check → |
Jurisdiction: China
This is the crucial split most write-ups miss: using a Chinese lab's HOSTED API sends your data to Chinese infrastructure and is a real procurement question. Downloading their OPEN WEIGHTS and running them on your own hardware, or on a Western host, sends nothing anywhere. DeepSeek and Z.ai publish under MIT; Alibaba publishes most of Qwen3 under Apache 2.0. Those are among the most permissive licences on this page.
Watch for: Several US states and a number of government bodies restrict Chinese-hosted AI services on official devices. That restriction is about the hosted service, not about the weights running in your own VPC — but expect to have to explain the difference to a security reviewer.
Frequently asked
Can I use BGE-M3 (embeddings) commercially?
MIT. The reranker sibling (bge-reranker-v2-m3) is Apache 2.0. Both unrestricted.
What hardware do I need to run BGE-M3 (embeddings)?
CPU is fine. Any GPU is instant. Weights alone are roughly 0.5 GB at Q4_K_M and 0.8 GB at Q8_0. Add KV cache on top of that, which grows with your context length.
What licence is BGE-M3 (embeddings) released under?
MIT. Full commercial use. Shortest and least restrictive of the common licences; no explicit patent grant.
Where can I use BGE-M3 (embeddings) for free?
Free tiers carrying it include Hugging Face Inference Providers. Limits differ per provider and change often, so check each provider's own limits page. You can also self-host the weights, which has no rate limit at all.
Similar models
Needle 3
Any CPU. A phone, a Raspberry Pi or a microcontroller-class board is enough.
DeepSeek-V4-Flash
Hosted, or 256 GB+ unified memory at Q4. Not a laptop model.
GLM-5.3-Flash
80-96 GB at Q4_K_M. A workstation or Ultra-class Mac.
Qwen3.8 27B
24 GB GPU at Q4_K_M, or a 32 GB Mac at Q6_K.
Wiring BGE-M3 (embeddings) into something real?
We build the evaluation harness, the failover and the cost ceilings around a model like this, so it survives contact with production.
Maps to AI strategy and consulting and AI automation development. Or see it working: our case studies.