Email and Happenings ingest

The inbound Email Worker handles four related sources:

• the weekly I-House Happenings newsletter;
• one-off event and announcement email;
• resident contact-form relays;
• direct or relayed casework email.

Newsletter and announcement content goes to Notion for editorial review. Resident casework goes to the private Casework database and uses the same routing and consolidation code as WhatsApp.

Flow

flowchart LR
  FWD["Editor forwards<br/>the newsletter"] --> ER["Email Routing<br/>happenings@in.ihnyc-rc.org"]
  ER --> W["ihnyc-rc-happenings-ingest<br/>(email Worker)"]
  W -->|"archive raw .eml"| R2["R2: happenings-archive<br/>&lt;week-of&gt;/&lt;hash&gt;.eml"]
  W -->|"sender + subject gate"| AI["Workers AI<br/>(Llama 4 Scout, JSON schema)"]
  AI --> UP["upsert by Key (dedupe)"]
  UP --> EV["Notion: Events DB"]
  UP --> HU["Notion: House Updates DB"]
  EV -.->|"editor enables 'Include in External Feed'"| ICS["programs.ics (15 min cron)"]
  HU -.->|"enrich (15 min, on change)"| SITE["/api/sync-updates → KV → site Updates page"]

One-off announcement email uses the same Events and House Updates review surfaces. Replies and forwards cannot create a public House Update automatically; they are held for review or treated as casework.

New rows leave Include in External Feed disabled. An editor reviews, corrects, and publishes them in Notion. Two independent downstream readers consume Notion:

• Events → calendar: regenerate-programs-ics (every 15 min) builds programs.ics from Events rows where Include in External Feed = true. See programs-ics-generation.
• House Updates → website: /api/sync-updates (every 15 min, cheap no-op when unchanged) enriches and renders the Updates page.

The Updates page is the visible end of that second path. Each card below started as a line in a forwarded newsletter, was parsed into Notion, and rendered here after curation:

House Updates grouped by source · June 2026

Inbound email (Cloudflare Email Routing)

Address	Routed on	Destination
happenings@in.ihnyc-rc.org	the in. subdomain	→ this Worker
dmarc@ihnyc-rc.org + catch-all	the apex	→ forward to a verified team inbox

The in. subdomain is used so the root domain stays reserved for a future real mailbox. The old Postal/LLC mailbox was decommissioned 2026-06-17. Sending (worker@ / pub@ via Resend, DKIM-authenticated) is unaffected — Email Routing governs inbound MX only.

Gates and dedupe

• Gate: acts only when the envelope sender contains the configured ALLOWED_SENDER (the editor’s forwarding address, set as a Worker var) and the subject contains SUBJECT_MATCH (I-House Happenings Week of). A Fwd: prefix still matches.
• Dedupe: each row carries a stable Key — event|<name>|<startISO>|<location> or house_update|<title> — so the same occurrence re-listed across weekly newsletters maps to one Notion page. is_update newsletters update in place.
• Cross-source event dedupe: a matching event from email and WhatsApp is kept as one event with combined source provenance.

Casework email

Casework email is archived first, then filtered for outbound mail, auto-replies, receipts, spam, and other non-cases. Accepted mail is normalized into the shared Casework shape.

• Provider conversation IDs and standard reply headers keep one email thread together.
• Related open cases can be consolidated after a model-assisted match.
• A reply to a resolved case can reopen it when the thread proves continuity.
• Contact-form submissions keep Contact Form as their source; direct and relayed mail use Email.
• Department and urgency values are normalized to the canonical Casework options.
• Current role owners come from RC Console D1, not the retired Notion roster databases.
• A Console switch can pause alert email without losing or un-routing the case.

Attachments are uploaded to private Notion file blocks. Do not copy them into public issue threads.

Parsing + observability

One Workers AI call per newsletter (@cf/meta/llama-4-scout-17b-16e-instruct, JSON-schema structured output) extracts house updates, upcoming events (with multi-date expansion), and recurring events. wrangler tail shows step logs: [recv] → [parse] (model + token usage) → [parsed] (counts) → per-row [write n/N] → [done] (total ms, AI calls, Notion calls, tally). A real run is ~40s (the AI parse) + the Notion upserts.

Raw-email archive + corpus

Every inbound email is archived as a raw .eml to R2 (ARCHIVE_BUCKET = ihnyc-rc-happenings-archive) before parsing, keyed <week-of>/<sha256[:16]>.eml. It’s best-effort — a failed archive PUT never blocks ingest. So the live weekly email self-archives, and the corpus grows on its own.

• Backfill (the Jan-2024 → now back catalogue that predates the worker) is uploaded directly to R2 via scripts/archive-emls.mjs <dir> (same key scheme; content hash makes re-runs idempotent). No replay through the worker — so no AI/Notion load for historical mail.
• Why a corpus: (1) drift analysis — see which weeks moved recurring events to Canva or changed structure; (2) eval — re-run the parser over real history when tuning the prompt/model (--remote, since it needs the AI binding); (3) a raw safety net to re-process after a fix. The model parse is not a deterministic CI test — the corpus is for eval, not green/red fixtures. Planned follow-ups (TODOs): few-shot prompting from curated examples, and a small hand-labeled eval set.

Deploy + config

• Auto-deploys on push to main via .github/workflows/deploy-happenings-worker.yml (runs npm ci for postal-mime, then wrangler deploy).
• Bindings: AI (Workers AI), ARCHIVE_BUCKET (R2 ihnyc-rc-happenings-archive).
• Vars: NOTION_EVENTS_DB_ID, NOTION_HOUSE_UPDATES_DB_ID, ALLOWED_SENDER, SUBJECT_MATCH, WAI_MODEL, RESEND_FROM, FAILURE_NOTIFY_TO.
• Secrets: NOTION_TOKEN (write on both Events + House Updates DBs), RESEND_API_KEY (failure-notice email), INGEST_TEST_TOKEN (optional, gates the /test fetch endpoint), and the service token used to read casework owners from RC Console.

Why not D1?

An earlier plan (PRs 74) would have made house updates D1-native and dropped Notion. That was closed — Notion is kept deliberately as the editorial staging area before anything reaches the site or calendar.

Related

• whatsapp-ingest — the group-chat counterpart (events posted only in WhatsApp)
• rc-admin — current council roster and casework owners
• programs-ics-generation — Events → ICS
• n8n — the pipelines this and sync-updates replaced
• shortio-migration — sibling n8n→Pages migration