Real-Time Crawl Process Management, Dispatching & Job Moderation
02:00 UTC daily (candidate seeds & harvest Hacker News/GitHub).03:00 UTC every Monday (re-crawl entire 940+ company database).| Status | Target Website | Source | Jobs Found | Duration / Error | Created At | Actions |
|---|---|---|---|---|---|---|
| Loading crawl processes... | ||||||
Automatically harvest unlisted micro-companies, boutique tech studios & indie makers from open sources (Hacker News 'Who is Hiring' Algolia API & GitHub Open Repositories 880+ companies), then dispatch Agents to crawl and parse jobs directly from company career sites.
Parse the latest Hacker News hiring thread. Extract company name, website, tech stack, hiring email, and direct application links.
Ingest from curated directories of 880+ remote & boutique studios (remoteintech & established-remote). Enqueue companies and dispatch Agents to crawl.
Combine both Hacker News & GitHub repositories. Ingest companies in parallel and dispatch background crawler agents.
Automatically crawl 41 curated US SMBs (2,800+ jobs) across 9 sectors: Health/Biotech, Cleantech, Robotics/Hardware, Fintech, Logistics, E-commerce, FoodTech, EdTech, HRTech.
Instantly dispatch crawls across targeted micro-companies, boutique engineering studios, and regional tech ecosystems.
For the first dataset ingestion, you can trigger Initial Bulk Blast to scan all curated small & medium companies across the US. Afterward, the system automatically sweeps weekly (every Monday at 03:00 UTC) to refresh jobs without manual intervention.
Sweep all active companies in the database concurrently. Automatically shards work into parallel batches on Cloudflare Queues for maximum speed.
Push all 90+ curated companies (Midwest, Texas, East Coast, West Coast) into the background queue for asynchronous crawling.
Run directly from your local terminal or VPS with courteous rate-limit protection:
Re-crawl all companies and direct ATS boards weekly (Cloudflare Cron 0 3 * * 1 + 1/7 rolling partition). Automatically detects new and closed roles.
| Job Title | Discipline | Seniority | Remote | Status | First Seen | Actions |
|---|---|---|---|---|---|---|
| Loading jobs for moderation... | ||||||
| Status | Agent | Target URL | Duration | Message | Time |
|---|---|---|---|---|---|
| Loading telemetry logs... | |||||
You can configure Bot Token, Bot Username, and Admin Permissions directly on this page. Configuration is persisted securely in Cloudflare D1.
@BotFather on Telegram, send /newbot, and copy your HTTP API Token.
/start or /admin with your bot to trigger crawlers, search jobs, and receive alerts.
123456...abcd once saved.
?token=...) and bot command /admin auth <token>.
Send a direct test alert via Telegram Bot to verify API Token and Chat ID deliverability.
| Role | Name / First Name | Telegram Username | Telegram ID | Personal MCP Token | Joined Date | Actions |
|---|---|---|---|---|---|---|
| Loading users... | ||||||
| Time & User | Type & Severity | Comment & Context | Linked Entity | Suggested Action | Status & Actions |
|---|---|---|---|---|---|
| Loading feedback records... | |||||