Pipeline Architecture
GNI_Autonomous runs 4 autonomous pipelines via GitHub Actions cron jobs. No human intervention required. Every day it collects global news, analyzes it with AI, produces sentiment reports with 95% confidence intervals, runs a 4-agent multi-perspective debate, and publishes everything to this live web app.
gni_pipeline
02:00 + 10:00 UTC~6,175/run
RSS collection → intelligence funnel → AI analysis → CI → pillar reports → save → verify
gni_mad
02:30 + 10:30 UTC~12,393/run
4-agent debate (Bull/Bear/Black Swan/Ostrich) → 3 rounds → Arbitrator → verdict → predictions
gni_heartbeat
Every 30 min0 (zero)
Monitor escalation delta → trigger adaptive → NYSE alerts → divergence detection
gni_adaptive
On trigger0–12,393
Fresh analysis when escalation spikes — CRITICAL=0 Groq, HIGH=4 calls, LOW=19 calls
Pipeline Stages — Step by Step
Step 0
Quota Guard Pre-flight
Reads groq_daily_usage → sums today's tokens → checks against 85K safe ceiling. If insufficient headroom: EXIT with Telegram alert. Sacred runs (02:00+10:00 UTC) always permitted.
Stage 1a
RSS Collection
Fetches up to 20 articles per RSS source from 25+ global news feeds. Target: 400+ raw articles per run. All saved to pipeline_articles table.
Stage 1b
Injection Detection
66 prompt injection patterns scanned across 7 layers: Unicode norm, source credibility, context boundary, NER, Groq hardened JSON, output sanitization, audit trail. Flagged articles written to audit_trail table.
Stage 2
Deduplication
MD5 hash of first 6 title words. Articles with same hash within 6 hours are deduplicated. Prevents duplicate reporting on same event.
Stage 3
Intelligence Funnel
Each article scored on geopolitical significance (0-20). Top N articles selected with source diversity enforced (max 3 per source). Pillar routing: geo/tech/fin tags assigned.
Stage 4a
Primary Analysis + CI
Groq llama-3.3-70b-versatile analyzes top articles. 3 independent runs at temperatures 0.1, 0.3, 0.7 generate 95% confidence intervals using t-distribution (t=4.303 for n=3).
Stage 4b
Three Pillar Reports
Three separate AI analyses for Geopolitical, Technology, Financial domains. Same top articles, domain-specific prompts. Runs after 120s rate limit reset.
Stage 5
Save + Telegram
Report saved to Supabase reports table with SHA-256 audit chain. Telegram notification sent with escalation level, sentiment, and MAD verdict.
Stage 6
GPVS Verify
After verify_date passes, actual SPY market movement compared to prediction direction. Accuracy logged. Source weights updated via EMA: correct=weight×1.1, wrong=weight×0.9.
Quadratic MAD Protocol — Novel Contribution #1
The MAD (Multi-Agent Debate) protocol applies the Johari Window framework to geopolitical intelligence. Four agents represent four quadrants: Bull (Known Positives), Bear (Known Negatives), Black Swan (Unknown Negatives), Ostrich (Ignored Realities). After 3 rounds of debate with Arbitrator coaching, a BULLISH/BEARISH/NEUTRAL verdict is reached with confidence score.
🐂 Bull
Known + Proactive
Identifies opportunity costs of inaction. What positive outcomes are being missed?
🐻 Bear
Known + Ignored
Finds systemic failures and fragile systems. What known risks are being dismissed?
🦢 Black Swan
Unknown + Negative
Uncovers hidden dangers from low-scoring articles. What could catch everyone off guard?
🦦 Ostrich
Unknown + Ignored
Exposes what institutions are in denial about. What threat is everyone pretending does not exist?
Technology Stack
Python 3.10+
Pipeline scripts
main.py, mad_runner.py, monitoring_pipeline.py, adaptive_pipeline.py
Next.js 14 + TypeScript
Web app + API routes
33 pages live on Vercel
Groq API
LLM inference
llama-3.3-70b-versatile — 100K tokens/day free
Supabase
PostgreSQL database
15 tables — reports, predictions, audit_trail, groq_daily_usage...
GitHub Actions
Cron + CI/CD
4 workflows — gni_pipeline, gni_mad, gni_heartbeat, gni_adaptive
Telegram Bot API
Alert notifications
Sacred completion, CRITICAL escalation, NYSE alerts, divergence
Vercel
Deployment + CDN
Auto-deploy on git push to main