import "github.com/kinorai/omnifeed/internal/engine/github"
Package github implements the GitHub engine. It reads the public GitHub REST API (api.github.com) and renders an issue URL as issue + comments and a pull request URL as PR + conversation comments + reviews + inline review comments + changed files with patches (TOON) — mirroring the Hacker News engine’s shape.
Like the Hacker News engine (and unlike Reddit and the generic engine), this engine fetches its upstream DIRECTLY over HTTP rather than through crawl4ai: the REST API is a public JSON API with no bot wall, and a browser render of a GitHub issue is both slower and lossy (comments are lazily paginated in the DOM). It does require omnifeed to have outbound access to api.github.com.
Comment is one conversation-tab comment (issue or PR), pruned to the three fields an agent actually reads.
type Comment struct {
Login string `json:"login" toon:"login"`
Created string `json:"created" toon:"created"`
Body string `json:"body" toon:"body"`
}
Config configures a GitHub Engine.
type Config struct {
Client *httpx.Client
Limiter httpx.Limiter
APIBase string // defaults to the public REST API; overridden in tests
Token string // optional PAT; empty = anonymous (60 req/h/IP)
Timeout time.Duration // wall-clock budget per crawl; defaults to defaultTimeout
Logger *slog.Logger
}
Engine implements domain.Engine for GitHub issue and pull-request URLs via the REST API.
type Engine struct {
// contains filtered or unexported fields
}
func New(cfg Config) *Engine
New returns a GitHub Engine configured per cfg.
func (e *Engine) Crawl(ctx context.Context, rawURL string, _ domain.EngineOptions) (domain.Document, error)
Crawl fetches the issue or pull request behind rawURL from the GitHub REST API and returns it encoded as TOON.
func (*Engine) Matches(rawURL string) bool
Matches claims only the github.com URLs this engine renders: issue and pull request pages. Everything else on the host (blob/tree/actions/releases/ discussions, repo roots, …) falls through to the generic fallback, which renders the page through crawl4ai.
func (*Engine) Name() string
Name returns the engine identifier (“github”).
File is one changed file. Patch is empty once the diff budget is spent — the filename and stats are still listed so the change set stays complete.
type File struct {
Name string `json:"name" toon:"name"`
Status string `json:"status" toon:"status"`
Additions int `json:"additions" toon:"additions"`
Deletions int `json:"deletions" toon:"deletions"`
Patch string `json:"patch,omitempty" toon:"patch,omitempty"`
}
InlineComment is a review comment anchored to a line of the diff. ReplyTo keeps review threads reconstructable without nesting (same flat+parent shape the Reddit and Hacker News engines use).
type InlineComment struct {
Path string `json:"path" toon:"path"`
Line int `json:"line" toon:"line"`
ReplyTo int64 `json:"reply_to,omitempty" toon:"reply_to,omitempty"`
Login string `json:"login" toon:"login"`
Created string `json:"created" toon:"created"`
Body string `json:"body" toon:"body"`
}
Issue is the header of a GitHub issue, stripped to LLM-relevant fields.
type Issue struct {
Title string `json:"title" toon:"title"`
Author string `json:"author" toon:"author"`
State string `json:"state" toon:"state"`
Created string `json:"created" toon:"created"`
Labels []string `json:"labels,omitempty" toon:"labels,omitempty"`
Comments int `json:"comments" toon:"comments"`
Body string `json:"body,omitempty" toon:"body,omitempty"`
}
IssueThread groups an issue with its comments. Note, when set, tells the reader (the LLM) that a list was truncated — metadata keys land in MCP _meta, which models never see, so the signal has to live in the content itself.
type IssueThread struct {
Issue Issue `json:"issue" toon:"issue"`
Note string `json:"note,omitempty" toon:"note,omitempty"`
Comments []Comment `json:"comments" toon:"comments"`
}
PullRequest is the header of a pull request. It carries the issue fields (a PR *is* an issue) plus the diff stats that only the pulls endpoint reports.
type PullRequest struct {
Title string `json:"title" toon:"title"`
Author string `json:"author" toon:"author"`
State string `json:"state" toon:"state"`
Draft bool `json:"draft" toon:"draft"`
Merged bool `json:"merged" toon:"merged"`
Created string `json:"created" toon:"created"`
Labels []string `json:"labels,omitempty" toon:"labels,omitempty"`
Comments int `json:"comments" toon:"comments"`
Additions int `json:"additions" toon:"additions"`
Deletions int `json:"deletions" toon:"deletions"`
ChangedFiles int `json:"changed_files" toon:"changed_files"`
Body string `json:"body,omitempty" toon:"body,omitempty"`
}
PullThread is everything the four PR endpoints contribute, in one document. Note carries truncation warnings, as on IssueThread.
type PullThread struct {
PR PullRequest `json:"pr" toon:"pr"`
Note string `json:"note,omitempty" toon:"note,omitempty"`
Comments []Comment `json:"comments" toon:"comments"`
Reviews []Review `json:"reviews" toon:"reviews"`
InlineComments []InlineComment `json:"inline_comments" toon:"inline_comments"`
Files []File `json:"files" toon:"files"`
}
Review is one submitted PR review (APPROVED / CHANGES_REQUESTED / COMMENTED).
type Review struct {
Login string `json:"login" toon:"login"`
State string `json:"state" toon:"state"`
Submitted string `json:"submitted" toon:"submitted"`
Body string `json:"body,omitempty" toon:"body,omitempty"`
}
Generated by gomarkdoc