Vultisignal Data Flow Diagram

Complete System Architecture

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│                 │    │                  │    │                 │
│  GitHub API     │    │  Local Storage   │    │  GitHub Actions │
│  (43+ repos)    │    │  Processing      │    │  Automation     │
│                 │    │                  │    │                 │
└─────────┬───────┘    └─────────┬────────┘    └─────────┬───────┘
          │                      │                       │
          │ collect --github     │ process               │ schedule/trigger
          │                      │ --html-dashboard      │
          ▼                      │                       ▼
┌─────────────────┐              │              ┌─────────────────┐
│                 │              │              │                 │
│ vultisignal.py  │◄─────────────┤              │  Daily Runner   │
│ CLI Interface   │              │              │  (.github/      │
│                 │              │              │   workflows/)   │
└─────────┬───────┘              │              └─────────────────┘
          │                      │
          │ call                 │
          ▼                      │
┌─────────────────┐              │
│                 │              │
│ collectors/     │              │
│ github.py       │              │
│                 │              │
└─────────┬───────┘              │
          │                      │
          │ save                 │
          ▼                      │
┌─────────────────┐              │
│                 │              │
│ exports/github/ │              │
│ YYYY-MM-DD.json │              │
│                 │              │
└─────────┬───────┘              │
          │                      │
          │ read                 │
          ▼                      │
┌─────────────────┐              │
│                 │              │
│ scripts/        │◄─────────────┘
│ html_generator  │
│ .py             │
│                 │
└─────────┬───────┘
          │
          │ render
          ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│                 │    │                 │    │                 │
│ templates/      │    │ assets/         │    │ Jinja2          │
│ ├─base.html     │    │ custom.css      │    │ Template        │
│ ├─dashboard.html│    │                 │    │ Engine          │
│ └─simple_...html│    │                 │    │                 │
└─────────┬───────┘    └─────────┬───────┘    └─────────┬───────┘
          │                      │                       │
          │ template             │ styling               │ process
          │                      │                       │
          └──────────────────────┼───────────────────────┘
                                 │
                                 ▼
                    ┌─────────────────┐
                    │                 │
                    │ docs/           │
                    │ ├─index.html    │
                    │ └─assets/       │
                    │   └─custom.css  │
                    └─────────┬───────┘
                              │
                              │ deploy
                              ▼
                    ┌─────────────────┐
                    │                 │
                    │ GitHub Pages    │
                    │ (gh-pages       │
                    │  branch)        │
                    │                 │
                    └─────────────────┘

Key Data Transformations

1. Collection Phase

• Input: GitHub API endpoints (issues, PRs, commits, releases)
• Process: collectors/github.py fetches and normalizes data
• Output: exports/github/YYYY-MM-DD.json (timestamped snapshot)

2. Processing Phase

• Input: Daily JSON snapshot + user command
• Process: scripts/html_generator.py analyzes and transforms data
• Analytics: Cross-platform detection, stale item analysis, priority scoring
• Output: Structured data context for templates

3. Template Rendering

• Input: Processed data context + Jinja2 templates
• Templates: base.html (shared) + dashboard.html (enhanced) or simple_dashboard.html
• Styling: assets/custom.css copied to docs/assets/
• Output: Complete HTML dashboard with embedded CSS references

4. Deployment

• Local: docs/index.html for immediate viewing
• Production: GitHub Actions publishes to gh-pages branch
• Result: Live dashboard at https://username.github.io/vultisignal

Command Flow Examples

Local Development

python3 vultisignal.py collect --github
     ↓
exports/github/2025-07-28.json
     ↓
python3 vultisignal.py process --html-dashboard --local
     ↓
docs/index.html + docs/assets/custom.css

Production Automation

GitHub Actions trigger (schedule/push)
     ↓
python3 vultisignal.py collect --github
     ↓
python3 vultisignal.py process --html-dashboard --current
     ↓
GitHub Pages deployment (gh-pages branch)

Data Dependencies

• Collection: Requires GITHUB_TOKEN environment variable
• Processing: Requires existing JSON data file for target date
• Templates: Requires templates/ directory with Jinja2 files
• Styling: Uses assets/custom.css (copied with --local flag)
• Deployment: GitHub Actions handles asset copying and deployment

Error Handling

Each phase includes comprehensive error handling:

• Collection: API failures, rate limiting, missing repositories
• Processing: Missing data files, malformed JSON, template errors
• Rendering: Missing templates, invalid context data
• Deployment: Asset copying failures, GitHub Pages issues

The system is designed to be resilient and provide clear error messages at each stage.