import "github.com/kinorai/omnifeed/internal/browser/crawl4ai"
Package crawl4ai implements the browser.Browser port over crawl4ai’s /execute_js endpoint: it drives a real headless Chromium to a page and runs caller-supplied JavaScript from inside it. This is the Reddit engine’s browser backend.
crawl4ai has no session reuse on /execute_js, so a Session here does not hold a live page: Navigate records the target URL and every Eval re-navigates to it before running its script. Semantically each Eval is one fresh browser navigation — the same behaviour omnifeed’s Reddit engine has always had.
Why /execute_js (not /crawl): crawl4ai 0.9.x rejects caller-supplied js_code on /crawl “from an untrusted request”. /execute_js is the sanctioned endpoint that runs caller JS — it builds the crawler config server-side. It must be enabled on the crawl4ai side (CRAWL4AI_EXECUTE_JS_ENABLED=true) and, once crawl4ai has a token, requires it (sent as `Authorization: Bearer <token>`).
Browser drives crawl4ai’s /execute_js endpoint.
type Browser struct {
// contains filtered or unexported fields
}
func New(client *httpx.Client, crawl4aiURL, token string) *Browser
New constructs a Browser. crawl4aiURL is OMNIFEED_CRAWL4AI_URL (the /crawl URL); the sibling /execute_js URL is derived from it. token, when set, is sent as `Authorization: Bearer <token>` (crawl4ai’s CRAWL4AI_API_TOKEN).
func (*Browser) Name() string
Name identifies the backend.
func (b *Browser) Open(context.Context) (browser.Session, error)
Open returns a stateless session: it holds only the URL recorded by Navigate, since /execute_js re-navigates on every call.
Generated by gomarkdoc