IHNYC Resident Council Docs

Email and Happenings ingest

Active6 min read

Table of Contents

The Happenings newsletter path moved to Calendar

happenings@in.ihnyc-rc.org now routes to a dedicated source Worker that sanitizes the message and delivers candidates to the Calendar service, which owns both the Programs event queue and the House Updates bulletin. Notion is no longer in that path. The Worker described on this page keeps casework and archive-only recipients; its House Updates email, manual, and backfill writers are gated off by HAPPENINGS_BASE_ROUTING_MODE = "calendar_v2".

For the current newsletter path, see What happens when you forward to happenings@. The rest of this page describes the casework and archive lanes, which are unchanged, and retains the newsletter history for context.

The inbound Email Worker handles four related sources:

  • the weekly I-House Happenings newsletter, now handed off to Calendar;
  • one-off event and announcement email;
  • resident contact-form relays;
  • direct or relayed casework email.

Resident casework goes to the private Casework database.

Flow

flowchart LR
  FWD["Editor forwards<br/>the newsletter"] --> ER["Email Routing<br/>happenings@in.ihnyc-rc.org"]
  ER --> SRC["ihnyc-happenings-source<br/>(Calendar-owned Worker)"]
  SRC -->|"sanitized receipt + candidates"| CAL["Calendar review queues"]
  CASE["Casework and<br/>archive-only recipients"] --> W["ihnyc-rc-happenings-ingest<br/>(this Worker)"]
  W -->|"archive raw .eml"| R2["R2 archive buckets"]
  W --> CW["Notion: Casework DB"]

Historically this Worker parsed the newsletter into Notion Events and House Updates databases, and two independent readers consumed those rows: regenerate-programs-ics built programs.ics from Events marked Include in External Feed, and /api/sync-updates rendered the site Updates page. Both destinations are retired. Programs events are canonical in Calendar D1, and House Updates are published from Calendar.

Replies and forwards still cannot create a public notice automatically. They are held for review or treated as casework.

The Updates page below is the historical visible end of the Notion path, kept as a record of what that pipeline produced:

The ihnyc-rc.org House Updates page: cards grouped by source (dining hall, facilities, programs) with categories and timestamps, rendered from the Notion House Updates DB House Updates grouped by source · June 2026

Inbound email (Cloudflare Email Routing)

AddressRouted onDestination
happenings@in.ihnyc-rc.orgthe in. subdomain→ this Worker
dmarc@ihnyc-rc.org + catch-allthe apex→ forward to a verified team inbox

The in. subdomain is used so the root domain stays reserved for a future real mailbox. The old Postal/LLC mailbox was decommissioned 2026-06-17. Sending (worker@ / pub@ via Resend, DKIM-authenticated) is unaffected — Email Routing governs inbound MX only.

Gates and dedupe (historical newsletter behavior)

  • Gate: acted only when the envelope sender contained the configured ALLOWED_SENDER (the editor’s forwarding address, set as a Worker var) and the subject contained SUBJECT_MATCH (I-House Happenings Week of). A Fwd: prefix still matched.
  • Dedupe: each row carried a stable Key — event|<name>|<startISO>|<location> or house_update|<title> — so the same occurrence re-listed across weekly newsletters mapped to one Notion page. is_update newsletters updated in place.
  • Cross-source event dedupe: a matching event from email and WhatsApp was kept as one event with combined source provenance.

The Calendar-owned path replaced sender-and-subject gating with envelope-recipient admission plus content recognition, and replaced key-based upsert with a per-message content binding and an explicit operator review queue. See the current path.

Casework email

Casework email is archived first, then filtered for outbound mail, auto-replies, receipts, spam, and other non-cases. Accepted mail is normalized into the shared Casework shape.

  • Provider conversation IDs and standard reply headers keep one email thread together.
  • Related open cases can be consolidated after a model-assisted match.
  • A reply to a resolved case can reopen it when the thread proves continuity.
  • Contact-form submissions keep Contact Form as their source; direct and relayed mail use Email.
  • Department and urgency values are normalized to the canonical Casework options.
  • Current role owners come from RC Console D1, not the retired Notion roster databases.
  • A Console switch can pause alert email without losing or un-routing the case.

Attachments are uploaded to private Notion file blocks. Do not copy them into public issue threads.

Parsing + observability

One Workers AI call per newsletter (@cf/meta/llama-4-scout-17b-16e-instruct, JSON-schema structured output) extracts house updates, upcoming events (with multi-date expansion), and recurring events. wrangler tail shows step logs: [recv] → [parse] (model + token usage) → [parsed] (counts) → per-row [write n/N] → [done] (total ms, AI calls, Notion calls, tally). A real run is ~40s (the AI parse) + the Notion upserts.

Raw-email archive + corpus

Every inbound email is archived as a raw .eml to R2 (ARCHIVE_BUCKET = ihnyc-rc-happenings-archive) before parsing, keyed <week-of>/<sha256[:16]>.eml. It’s best-effort — a failed archive PUT never blocks ingest. So the live weekly email self-archives, and the corpus grows on its own.

  • Backfill (the Jan-2024 → now back catalogue that predates the worker) is uploaded directly to R2 via scripts/archive-emls.mjs <dir> (same key scheme; content hash makes re-runs idempotent). No replay through the worker — so no AI/Notion load for historical mail.
  • Why a corpus: (1) drift analysis — see which weeks moved recurring events to Canva or changed structure; (2) eval — re-run the parser over real history when tuning the prompt/model (--remote, since it needs the AI binding); (3) a raw safety net to re-process after a fix. The model parse is not a deterministic CI test — the corpus is for eval, not green/red fixtures. Planned follow-ups (TODOs): few-shot prompting from curated examples, and a small hand-labeled eval set.

Deploy + config

  • Auto-deploys on push to main via .github/workflows/deploy-happenings-worker.yml (runs npm ci for postal-mime, then wrangler deploy).
  • Bindings: AI (Workers AI), ARCHIVE_BUCKET (R2 ihnyc-rc-happenings-archive).
  • Vars: HAPPENINGS_BASE_ROUTING_MODE (set to calendar_v2, which disables the House Updates email, manual, and backfill writers), ALLOWED_SENDER, SUBJECT_MATCH, WAI_MODEL, RESEND_FROM, FAILURE_NOTIFY_TO, and NOTION_HOUSE_UPDATES_DB_ID, retained only for the reviewed rollback path.
  • Secrets: NOTION_TOKEN, RESEND_API_KEY (failure-notice email), INGEST_TEST_TOKEN (optional, gates the /test fetch endpoint), and the service token used to read casework owners from RC Console.

The D1 question was settled the other way

An earlier plan (PRs 74) to make House Updates D1-native was closed, and Notion was kept as the editorial staging area. That decision was later reversed: House Updates and Programs events are now Calendar-owned in D1, with review queues replacing Notion staging. Casework remains in Notion.