import "github.com/kinorai/omnifeed/internal/engine/stackexchange"
Package stackexchange implements the Stack Exchange engine. It reads the public Stack Exchange API (api.stackexchange.com, v2.3) and renders a question URL on Stack Overflow, Super User, Server Fault, Ask Ubuntu, MathOverflow or any {name}.stackexchange.com site as the question header + body and its answers by votes (TOON), headed by a “# title” line.
Like the Hacker News, GitHub and Discourse engines (and unlike Reddit and the generic engine), this engine fetches its upstream DIRECTLY over HTTP rather than through crawl4ai: the question pages sit behind Cloudflare, which answers crawl4ai with a 403 (“Blocked by anti-bot protection”), while the API serves the same content keyless. It does require omnifeed to have outbound access to api.stackexchange.com.
Keyless callers share a quota of 300 requests a day per IP; a key (OMNIFEED_STACKEXCHANGE_KEY) raises it to 10,000. A question costs 2 requests (question + answers), a short /a/ link 3.
Answer is one answer, in vote order. Accepted marks the answer the asker accepted.
type Answer struct {
ID int64 `json:"id" toon:"id"`
Score int `json:"score" toon:"score"`
Accepted bool `json:"accepted" toon:"accepted"`
Author string `json:"author" toon:"author"`
Created string `json:"created" toon:"created"`
Body string `json:"body" toon:"body"`
}
Config configures a Stack Exchange Engine.
type Config struct {
Client *httpx.Client
Limiter httpx.Limiter
APIBase string // defaults to the public v2.3 API; overridden in tests
Key string // optional app key; empty = keyless (300 requests/day/IP)
Timeout time.Duration // wall-clock budget per crawl; defaults to defaultTimeout
Logger *slog.Logger
}
Engine implements domain.Engine for Stack Exchange question URLs via the public Stack Exchange API.
type Engine struct {
// contains filtered or unexported fields
}
func New(cfg Config) *Engine
New returns a Stack Exchange Engine configured per cfg.
func (e *Engine) Crawl(ctx context.Context, rawURL string, _ domain.EngineOptions) (domain.Document, error)
Crawl fetches the question behind rawURL and its answers from the Stack Exchange API and returns them as TOON headed by a “# title” line.
func (*Engine) Matches(rawURL string) bool
Matches claims question URLs on the Stack Exchange network only (see questionRE and siteFor). Every other page on those hosts falls through to the generic crawl4ai fallback.
func (*Engine) Name() string
Name returns the engine identifier (“stackexchange”).
Question is the header of a question, stripped to LLM-relevant fields. AcceptedAnswer is the id of the accepted answer (0 = none), the same id the accepted entry in Answers carries.
type Question struct {
Title string `json:"title" toon:"title"`
Site string `json:"site" toon:"site"`
Score int `json:"score" toon:"score"`
Tags []string `json:"tags,omitempty" toon:"tags,omitempty"`
Author string `json:"author" toon:"author"`
Asked string `json:"asked" toon:"asked"`
Answers int `json:"answers" toon:"answers"`
AcceptedAnswer int64 `json:"accepted_answer,omitempty" toon:"accepted_answer,omitempty"`
Closed string `json:"closed,omitempty" toon:"closed,omitempty"`
Body string `json:"body" toon:"body"`
}
Thread groups a question with its answers. Note, when set, tells the reader (the LLM) that the answer list was truncated — metadata keys land in MCP _meta, which models never see, so the signal has to live in the content itself.
type Thread struct {
Question Question `json:"question" toon:"question"`
Note string `json:"note,omitempty" toon:"note,omitempty"`
Answers []Answer `json:"answers" toon:"answers"`
}
Generated by gomarkdoc