Hallucination Guard – CertainLogic Verifier

Overview

CertainLogic Verifier is an open‑source, self‑hosted middleware layer that sits between your LLM calls and your application. It validates every AI response against a verified facts database, flags hallucinations, caches verified answers (bypassing the LLM), and provides cryptographic audit logs.

Key capabilities:

• 99%+ hallucination block rate – rule‑based checks + TF‑IDF memory search against your facts_db
• 85‑98% token savings – semantic cache hits skip the LLM entirely
• Self‑hosted & air‑gapped – nothing leaves your infrastructure; ready for HIPAA/GDPR/SOC2/FedRAMP
• MIT licensed – no proprietary lock‑in; inspect every validation rule
• Deterministic grounding – same query → same verified answer, every time
• Cryptographic audit logs – SHA‑256 chained JSONL for compliance

Quick Start (2‑Minute Install)

# Clone the repository
git clone https://github.com/CertainLogicAI/hallucination-guard
cd hallucination-guard

# Set up Python environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

# Start the service
uvicorn main:app --host 0.0.0.0 --port 8000

Verify it's working:

curl -X POST http://localhost:8000/validate \
  -d '{"query": "What is the price of GPT‑5?", "response": "\$200/month"}'

Installation Options

1. Docker (Recommended for Production)

docker build -t hallucination-guard .
docker run -p 8000:8000 hallucination-guard

2. Kubernetes/Helm

See deploy/helm/ in the repository for production‑ready Helm charts.

3. Systemd Service

A sample systemd unit file is included at deploy/systemd/hallucination-guard.service.

Configuration

Facts Database

The verifier checks responses against facts_db.json. Populate it with your domain‑specific verified facts.

Example entry:

{
  "fact": "Python was created in 1991 by Guido van Rossum",
  "category": "programming",
  "source": "official Python history",
  "verified_at": "2026‑04‑20"
}

Adding facts:

• Manually edit facts_db.json
• Use the /facts/add endpoint (POST with JSON)
• Bulk‑load from documents via the /warming/extract endpoint

Environment Variables

Set these in .env or as environment variables:

PRODUCT_MODE=coder           # coder|agent (determines rate limits)
OPENROUTER_API_KEY=your_key  # Required for cache‑miss fallback
LOG_LEVEL=INFO              # DEBUG|INFO|WARNING|ERROR
CACHE_DIR=./cache           # Persistent cache storage

Usage

Validating a Single Response

import requests

response = requests.post(
    "http://localhost:8000/validate",
    json={
        "query": "What year was Python created?",
        "response": "Python was created in 1991."
    }
)
print(response.json())

Integrating with AI Agent Pipelines

Place the verifier between your LLM call and your application logic:

def get_ai_response(query):
    # 1. Check cache first
    cache_check = requests.post("http://localhost:8000/cache/check", 
                                json={"query": query})
    if cache_check.json().get("cached"):
        return cache_check.json()["response"]
    
    # 2. Call LLM
    llm_response = call_llm(query)
    
    # 3. Validate
    validation = requests.post("http://localhost:8000/validate",
                               json={"query": query, "response": llm_response})
    
    if validation.json().get("valid"):
        return llm_response
    else:
        # Handle hallucination
        raise ValueError(f"Hallucination detected: {validation.json()}")

Cache Management

• View cache stats: GET /cache/stats
• Clear cache: POST /cache/clear
• Warm cache: POST /warming/run (requires OpenRouter API key)

Advanced Features

Deterministic Memory Search

The verifier uses TF‑IDF similarity to match queries against known facts, even with paraphrasing.

Uncertainty Detection

Responses containing "I think", "might be", "not sure" are penalized and flagged for review.

Numeric‑Unit Matching

Checks that numeric values match known facts with correct units (e.g., "5 km" vs "5 miles").

Audit Logs

All validations are logged to audit_log.jsonl with SHA‑256 chaining for tamper evidence.

Resources

scripts/

• install.sh – One‑line installer for Linux/macOS
• docker-compose.yml – Multi‑service setup with PostgreSQL for audit logs

references/

• api-reference.md – Complete API documentation
• facts-schema.md – Facts database schema and validation rules
• integration-guide.md – Step‑by‑step integration with popular AI frameworks

assets/

• sample-facts.json – Example facts database with 50+ verified entries
• docker-compose.prod.yml – Production‑ready Docker Compose configuration

Support & Community

• GitHub Issues: https://github.com/CertainLogicAI/hallucination-guard/issues
• Discord: https://discord.gg/clawd (OpenClaw community)
• X/Twitter: @CertainLogicAI

License

MIT – see LICENSE in the repository.