Skip to content

Health Checks

Purpose: Endpoints and commands to verify service health Audience: DevOps, Support Engineers Source of Truth: TRUTH_MAP.md

Last Updated: February 2, 2026

Quick Health Check

# Run all checks
./scripts/verify-services.sh

Individual Service Checks

Backend API

# Health endpoint
curl http://localhost:8080/health

# Expected response
{"status":"healthy","version":"1.0.0","services":{"postgres":"ok","redis":"ok","nats":"ok"}}

PostgreSQL

# Connection check
pg_isready -h localhost -p 5432

# Query check
psql "$POSTGRES_CONN_STR" -c "SELECT 1;"

# RLS verification
psql "$POSTGRES_CONN_STR" -c "SELECT tablename, rowsecurity FROM pg_tables WHERE schemaname = 'public' AND rowsecurity = true;"

Redis

# Ping
redis-cli -h localhost -p 6379 ping
# Expected: PONG

# Memory check
redis-cli -h localhost -p 6379 info memory | grep used_memory_human

NATS JetStream

# HTTP health
curl http://localhost:8222/healthz
# Expected: ok

# Stream status
nats stream info FULCRUM_EVENTS --server nats://localhost:4222

Ollama

# List models
curl http://localhost:11434/api/tags
# Expected: {"models":[{"name":"llama3.2",...}]}

# Generate test (Semantic Judge)
curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"test","stream":false}'

Dashboard

# Next.js health
curl http://localhost:3000/api/health
# Expected: {"status":"ok"}

Production Health Checks

API Backend (Railway)

# Canonical endpoint (per TRUTH_MAP)
curl https://api.fulcrumlayer.io/health
# Expected: {"service":"fulcrum-server","status":"healthy"}

# MCP health
curl https://api.fulcrumlayer.io/mcp/health
# Expected: {"service":"fulcrum-mcp","status":"healthy"}

Dashboard (Vercel)

curl https://fulcrumlayer.io/api/health
# Expected: {"status":"ok"}

Automated Monitoring

Prometheus Metrics

Metric Endpoint Purpose
fulcrum_policy_evaluations_total :8080/metrics Policy engine throughput
fulcrum_policy_latency_seconds :8080/metrics Evaluation latency
fulcrum_budget_usage_dollars :8080/metrics Cost tracking

Alert Conditions

Alert Condition Severity
PostgreSQL Down Health check fails 3x Critical
Redis Down Health check fails 3x Warning
NATS Consumer Lag Lag > 1000 messages Warning
API Latency High P99 > 500ms Warning
Document Purpose
TRUTH_MAP.md Infrastructure source of truth
MONITORING.md Detailed monitoring setup
disaster-recovery.md Emergency procedures

Back to Documentation Evidence sources: TRUTH_MAP.md, cmd/fulcrum-server/main.go