import "github.com/kinorai/omnifeed/internal/observability"
Package observability wires structured logging, Prometheus metrics, and Kubernetes-style health endpoints.
func Explain(err error) string
Explain renders a failed crawl/search error as a short, caller-safe explanation: the classified reason, the upstream HTTP status when the error carries one, and the root cause with internal endpoints redacted. Transports prefix their own context (“fetch_url failed: “ + Explain(err)) so an MCP or HTTP client sees what metrics already know instead of an opaque failure. Returns “” only when err is nil.
func NewLogger(level, format string) *slog.Logger
| NewLogger returns a slog.Logger configured per the given level and format. Level: debug | info | warn | error (default info). Format: json | text (default json). |
func Reason(err error) string
Reason maps a crawl error to the bounded `reason` label recorded on omnifeed_requests_total, so metrics and alerts can tell a 403 from a CAPTCHA from a timeout. It reads the classified cause from a domain.FetchError that the engines attach as data — no error-string parsing. Errors that aren’t a FetchError fall back to canceled (caller abort), timeout (context deadline), or “error”.
func RegisterPprof(mux *http.ServeMux)
RegisterPprof attaches /debug/pprof/* to mux. Opt-in via OMNIFEED_ENABLE_PPROF.
func StatusOf(err error) string
StatusOf reports the coarse request status recorded alongside Reason on the crawl/search metrics: “ok” when err is nil, “error” otherwise.
Health exposes Kubernetes-style health endpoints.
/livez — process is up; always 200 unless we're shutting down
/readyz — runs all ReadyCheck functions; 200 only if all pass
/healthz — alias of /readyz for backwards compat
type Health struct {
// contains filtered or unexported fields
}
func NewHealth(cacheTTL time.Duration, checks ...ReadyCheck) *Health
NewHealth returns a Health with the given readiness checks. Results are cached for `cacheTTL` to avoid hammering upstreams on every probe.
func (h *Health) MarkShuttingDown()
MarkShuttingDown causes /livez to return 503 so the LB stops sending traffic.
func (h *Health) Register(mux *http.ServeMux)
Register attaches the endpoints to mux.
Metrics holds Prometheus collectors emitted by the proxy.
type Metrics struct {
RequestsTotal *prometheus.CounterVec // engine, tenant, status, reason
RequestAttempts *prometheus.CounterVec // upstream, attempt (first|retry)
RequestSecs *prometheus.HistogramVec // engine, status, reason
UpstreamSecs *prometheus.HistogramVec // upstream, op, status
LimiterWaitSecs *prometheus.HistogramVec // engine
RatelimitErrors *prometheus.CounterVec // op
RatelimitPenalties *prometheus.CounterVec // upstream
RatelimitDegraded *prometheus.GaugeVec // scope
ResponseChars *prometheus.HistogramVec // engine
EngineFallbacks *prometheus.CounterVec // from_engine, reason
SearxngUnresponsive *prometheus.CounterVec // engine, error
SearxngEngineHits *prometheus.CounterVec // engine
SearxngEmpty *prometheus.CounterVec // scoped
SearxngQueries *prometheus.CounterVec // scoped
SearxngEngineZero *prometheus.CounterVec // engine
RedditRounds prometheus.Histogram
SearchesTotal *prometheus.CounterVec // searcher, status, reason, scoped
SearchSecs *prometheus.HistogramVec // searcher, status
SearchEnginePos *prometheus.HistogramVec // engine
SearchEngineUnique *prometheus.CounterVec // engine
// contains filtered or unexported fields
}
func NewMetrics() *Metrics
NewMetrics builds and registers all collectors.
func (m *Metrics) Observe(engine, tenant, status, reason string, duration time.Duration)
Observe records a single crawl result. reason is a bounded classification of WHY a request failed (see Reason); it is “ok” on success.
func (m *Metrics) ObserveAttempt(upstream string, retry bool)
ObserveAttempt records one HTTP attempt from the retrying client against the named upstream. retry is false for the first try and true for each retry, so attempt=”retry” counts the re-drives that #2’s RetryableStatus veto removes for non-transient blocks.
func (m *Metrics) ObserveEmptySearch(scoped bool)
ObserveEmptySearch counts a search that came back with no results and no failure report — the shape a silently blocked pool produces, and the shape an honest zero-hit query produces. scoped says whether a `site:` filter was applied, because the site-scoped path is where silent blocks concentrate.
func (m *Metrics) ObserveEngineRank(engine string, rank int, unique bool)
ObserveEngineRank records the rank an engine gave one of its own results, and whether that result was unique to it. Both are aggregates only — the URL and the query stay in the audit log, never in a label.
func (m *Metrics) ObserveEngineResults(engine string, rows int)
ObserveEngineResults counts the rows one SearXNG engine contributed to a search response. Engines that block silently — HTTP 200, no error, empty results — are invisible in every other metric, and this is what makes them visible: their series stops advancing while the rest of the pool continues.
func (m *Metrics) ObserveEngineZeroResults(engine string)
ObserveEngineZeroResults counts one search an engine contributed nothing to while the search itself returned rows. One increment per search per engine, never per row: it measures how OFTEN an engine sits out, not how much it missed.
func (m *Metrics) ObserveFallback(fromEngine, reason string)
ObserveFallback counts one engine→generic-fallback handoff: fromEngine is the dedicated engine that failed, reason its classified failure (see Reason).
func (m *Metrics) ObserveLimiterWait(engine, outcome string, duration time.Duration)
ObserveLimiterWait records the time an engine spent blocked acquiring the per-domain limiter. Wired as the DomainLimiter’s OnWait hook. outcome is “acquired”, “canceled” (the wait died in the queue) or “budget_exceeded” (the wait was longer than the caller’s budget, so the limiter refused to queue).
func (m *Metrics) ObserveRatelimitBackendError(op string)
ObserveRatelimitBackendError counts one failed Redis operation in the distributed limiter. op is “acquire” or “release”. Wired as the redis limiter’s OnError hook.
func (m *Metrics) ObserveRatelimitPenalty(upstream string)
ObserveRatelimitPenalty counts one upstream Retry-After fed back into the limiters as a hold on that host. upstream is the client’s own upstream label, never the host — hosts are unbounded.
func (m *Metrics) ObserveResponseChars(engine string, chars int)
ObserveResponseChars records the character count of a successful crawl’s extracted content, PRE-truncation (the engine’s output, before any transport max_chars clipping) — the quality guard for scrape-option changes. Recorded at the registry choke point so the label names the engine that actually produced the document, fallbacks included.
func (m *Metrics) ObserveSearch(searcher, status, reason string, scoped bool, duration time.Duration)
ObserveSearch records a single search query result. reason classifies WHY a search failed (see Reason); it is “ok” on success. scoped says whether the caller asked for a `site:` filter — site-scoped search runs on a different engine pool and fails for different reasons, and every existing sum() over this counter collapses the label anyway. SearchSecs stays keyed on searcher+status only — adding reason or scoped would just inflate histogram cardinality.
func (m *Metrics) ObserveSearxngQuery(scoped bool)
ObserveSearxngQuery counts one query actually sent to SearXNG — past the limiter, before the HTTP call. This is the rate the engines behind SearXNG see, which is what their own quotas are measured against; the caller-side counter (ObserveSearch) includes queries pacing refused.
func (m *Metrics) ObserveUnresponsiveEngine(engine, errType string)
ObserveUnresponsiveEngine counts one engine SearXNG reported unresponsive on a search response, by engine name and error type.
func (m *Metrics) ObserveUpstream(upstream, op, status string, duration time.Duration)
ObserveUpstream records one upstream HTTP round-trip attempt (request start until the response body is fully read, or transport error). status is “ok” for 2xx and “error” otherwise. Wired as the httpx client’s OnUpstream hook.
func (m *Metrics) RegisterMetrics(mux *http.ServeMux)
RegisterMetrics attaches /metrics to mux.
func (m *Metrics) SetRatelimitDegraded(scope string, down bool)
SetRatelimitDegraded records whether pacing is currently degraded to per-pod limits because the Redis backend is unreachable. Wired as the FallbackLimiter’s OnDegraded hook, which fires on transitions only.
Keyed by scope (“domain” or “searxng”): each scope has its own FallbackLimiter and degrades on its own, so one recovering must not zero the other’s signal.
ReadyCheck is a function that reports whether the binary is ready to serve. Return nil when ready, an error describing why not otherwise.
type ReadyCheck func(context.Context) error
Generated by gomarkdoc