Responsibilities
- Generate presigned upload keys and accept proxy uploads.
- Consume queue events from R2 uploads and deletions.
- Parse WhatsApp exports, deduplicate messages, and compute daily digests.
- Generate weekly summaries and update D1.
- Manage quotas, regeneration, and replay workflows.
Sources: beacon-platform/docs/architecture.md, beacon-platform/docs/operations.md
Core endpoints (admin)
| Path | Method | Purpose |
|---|---|---|
| /presign | POST | Generate R2 upload key |
| /upload | POST | Proxy upload |
| /files | GET/DELETE | List or delete exports |
| /exports/progress | GET | Export processing status |
| /clear | POST | Cleanup digests/summaries/hashes |
| /regenerate/daily | POST | Recompute daily digests |
| /regenerate/weekly | POST | Recompute weekly summaries |
| /regenerate/weekly/all | POST | Recompute weekly summaries for all |
| /replay/export | POST | Re-enqueue a single export |
| /replay/all | POST | Re-enqueue all exports |
| /quota/status | GET | AI quota status |
| /quota/set | POST | Update AI quota settings |
| /quota/bypass | GET/POST | Enable or disable bypass |
| /pipeline/daily-config | GET/POST | Daily digest processing mode |
Sources: beacon-platform/docs/architecture.md, beacon-platform/docs/operations.md
Processing flow
flowchart TD Upload -->|R2 ObjectCreated| Queue Queue --> Ingest Ingest --> Parse Parse --> Dedupe Dedupe --> DailyAI DailyAI --> DailyStore DailyStore --> WeeklyCheck WeeklyCheck -->|threshold met| WeeklyAI WeeklyAI --> WeeklyStore WeeklyCheck -->|below threshold| SkipWeekly
Sources: beacon-platform/docs/architecture.md
Deduplication
- Content is normalized (case, whitespace, unicode normalization) before hashing.
- Dedup key: (community_id, source_id, day_date, content_hash).
- Sender hash enables unique sender counts without storing PII.
Sources: beacon-platform/docs/architecture.md, beacon-platform/docs/database-structure.md
AI and privacy
- Messages are redacted for phones, emails, and URLs before AI.
- Outputs must pass JSON schema and privacy regex checks.
- One retry with stricter privacy instruction; fallback to neutral summary if still invalid.
Sources: beacon-platform/docs/privacy.md, beacon-platform/docs/architecture.md
Deletion and cleanup
- /clear removes digests and summaries by date range (optionally hashes).
- /communities/{id} deletes D1 records and R2 files for a community.
- R2 ObjectDeleted events also trigger cleanup.
Sources: beacon-platform/AUDIT_REPORT.md, beacon-platform/infra/R2_EVENTS.md