import "github.com/kinorai/omnifeed/internal/search/searxng"
Package searxng implements domain.Searcher against a SearXNG instance’s JSON API (GET /search?format=json). The instance must list `json` under `search.formats` in its settings.yml or every query returns 403.
Config configures the Searcher.
type Config struct {
Endpoint string // base URL of the SearXNG instance, e.g. http://searxng:8080
Client *httpx.Client
Logger *slog.Logger
// Metrics, when non-nil, receives per-search unresponsive-engine counts
// (omnifeed_searxng_unresponsive_engines_total).
Metrics *observability.Metrics
// SiteEngines restricts which SearXNG engines run when the caller passes a
// Site filter. SearXNG forwards `site:` to the engines rather than applying
// it itself, and engines differ wildly in what they do with it: some honour
// it, some ignore the operator and answer with unrelated pages, and some
// return nothing at all for any `site:` query. On a pool with the last kind
// in it, a site-scoped search is answered by whichever engines are left,
// with the ignorers' noise ranking high because nothing corroborates it.
// Listing the engines that implement the operator keeps the filter
// meaningful. Empty (the default) queries the whole pool, as before.
SiteEngines []string
// CriticalEngines are the engines whose absence makes a search Degraded
// (see domain.SearchReport): when one of them was asked and either failed
// openly (unresponsive_engines) or contributed no rows, the search still
// succeeds, but the report says it ran without that engine so a caller
// with a backup can top it up. On this deployment that is `google cse`:
// the rest of the pool answers when it is rate limited, with visibly
// weaker results. "Asked" accounts for SiteEngines only: an engine SearXNG
// skips for a filter it does not support (time_range, language) reads as
// missing too. Empty (the default) never reports a search degraded.
CriticalEngines []string
// Limiter, when non-nil, paces queries to the SearXNG instance. It exists
// for the ENGINES behind SearXNG, not for SearXNG itself: SearXNG fans one
// query out to every enabled engine, so each engine sees exactly this
// searcher's query rate, and an engine that decides the rate is bot-like
// suspends itself (or gets CAPTCHA'd) for the whole pool's benefit. Nil
// keeps the previous unpaced behaviour.
Limiter httpx.Limiter
// MaxWait caps how long a query may sit in the Limiter before it gives up.
// Ignored when Limiter is nil. Defaults to defaultMaxWait.
MaxWait time.Duration
// Audit selects the per-search audit log: "off" (default), "summary" or
// "full". See config.Config.SearchAudit for why this is its own setting
// rather than a log level.
Audit string
}
Searcher queries a SearXNG instance and reshapes results into the canonical domain.SearchResult.
type Searcher struct {
// contains filtered or unexported fields
}
func New(cfg Config) *Searcher
New returns a Searcher wired with the given config.
func (s *Searcher) InitEngineMetrics(ctx context.Context)
InitEngineMetrics fetches the instance’s enabled engines from GET /config and mints their per-engine series at zero (see initEngineMetrics). It is meant to run in its own goroutine after startup: it never touches readiness, retries with backoff while SearXNG is not answering yet, and returns after success, after the retry budget, or when ctx is done. A failure is logged once at WARN; nothing else depends on it — engines still get their series lazily, the first time a search response names them.
func (*Searcher) Name() string
Name returns the searcher identifier (“searxng”).
func (s *Searcher) Search(ctx context.Context, query string, opts domain.SearchOptions) (domain.SearchReport, error)
Search runs the query against SearXNG and returns up to opts.Limit results.
Generated by gomarkdoc