Civi serves uploaded files at /civicrm/file?id=X&eid=Y&fcs=<JWT>; the fcs
is a JWT signed with the site key. Without it, Civi's file handler crashes
on a null JWT decode (Firebase\JWT\JWT::decode argument null). We don't
have the site key on the Next.js side, so let Civi mint the URLs for us.
Backend (/api/staff/report):
- Add file_name selects for org-side file fields (Certificate of
Incorporation and friends) so org files have names alongside URLs.
- Collect every file id referenced by activity and org custom fields.
- Call APIv4 Attachment.get with select: ["id", "url"] to fetch signed
URLs in one round trip. Build a urlByFileId map.
- Org-side file values are now wrapped in { id, file_name, url } shape
matching the activity-side files (previously bare file ids that the
frontend couldn't render).
- Activity-side file values gain a url property from the map.
- If Attachment.get doesn't expose url on this Civi version, the call is
caught and we fall through to bare URLs without fcs (no regression).
Frontend (FieldValue):
- Prefer v.url when present, normalizing absolute and relative shapes
against CIVI_BASE_URL.
- Fall back to /civicrm/file?reset=1&id=X if url wasn't provided.
UX iteration after first live look:
- Sticky anchor strip below the header with a chip per section (incl.
Submissions) so staff can jump around a long page.
- Compact one-line rows that show only the latest value; multi-history
fields get a muted 'N earlier entries' toggle that reveals the rest
inline. Same affordance for file fields.
- Empty fields collapse under a single 'N empty fields' toggle per
section instead of taking a row each.
- Stage 5: Y1_Q<n>_<metric> fields render as a read-only matrix table
(rows: metrics; columns: Q1..Q4) matching the form's matrix layout.
File proxy (/api/staff/file) deleted. APIv4 Attachment isn't exposed
on this Civi instance (per the June upload spike), which is why the
previous proxy returned broken images. Staff are already authenticated
to Civi when they arrive here, so file fields now render as outbound
links to CIVI_BASE_URL/civicrm/file?reset=1&id=<id> and the browser
uses the staff session. No more proxy auth, no more SSRF surface to
harden, no broken images.
CIVI_BASE_URL flows from the staff page (server component) into the
client as a prop. No secret material crosses the boundary.
- Allowlist inline MIME types (png/jpeg/gif/webp/pdf only); everything
else, including SVG and HTML, served as application/octet-stream
with content-disposition: attachment.
- X-Content-Type-Options: nosniff and a restrictive CSP on every response.
- Validate the upstream URL Civi returns: must match CIVI_BASE_URL origin
before we attach basic-auth creds and follow it. redirect: manual to
prevent off-host hops.
- Drop SVG from the client's inline-image list (server forces download).
Closes the file-upload gap. Files now actually land in CiviCRM (verified
empirically against the live Civi instance via spike scripts).
Spike findings (see scripts/spike-file-upload.mjs):
- APIv4 Attachment is NOT exposed on this Civi
- APIv4 File + EntityFile ARE exposed; File.create accepts inline
base64 `content` and returns a usable file id
- Custom file fields store the file id directly in the custom column,
so EntityFile linkage is unnecessary for this use case
- Round-trip via Contact.update + Contact.get .file_name join verified
on a real org contact
Pipeline:
Renderer (FileField) picks up onChange →
POST /api/upload (multipart) with file + cid + cs + fieldRef →
verifyChecksum, MIME allowlist + magic-byte sniff, 5 MB cap →
civi.File.create({ file_name, mime_type, content: base64 }) →
returns { id, file_name } →
renderer stores in RHF state via setValue
Form submit →
POST /api/submit (JSON) with the {id, file_name} value →
submit detects the file shape and writes the id as the value of
the activity/contact custom field
File changes:
app/api/upload/route.ts
Replaced the 501 stub with the real File.create call. Comment
documents that EntityFile linkage is intentionally skipped and that
orphan cleanup is owned by a CiviCRM scheduled job.
app/api/submit/route.ts
For type:"file" values shaped as {id, file_name}, write the id as
the custom field value (activity or contact, depending on the
civiField / civiContactField the field declares).
components/fields/FieldRenderer.tsx
Replaced the bare <input type=file> register() with FileField, an
upload-on-pick subcomponent. The native input is NOT register()'d:
its FileList value was the original bug. FileField owns its
uploading + error state and writes {id, file_name} via setValue on
success. Submit is blocked upstream while uploads are in flight.
components/StageSection.tsx, components/EngagementForm.tsx
Thread setValue, cid, cs, and an onUploadStateChange callback
through to FieldRenderer. EngagementForm tracks uploads-in-flight
count; onSubmit refuses to submit while the count is > 0.
config/form.ts
Promotes Certificate_of_Incorporation from readonly to a real
file field now that the pipeline works.
app/api/data/route.ts
Drops the readonly carveout that was only needed while the
certificate was readonly.
scripts/list-civi-entities.mjs (new)
APIv4 entity probe + APIv3 attachment-API probe. Used to determine
that File (not Attachment) was the right entity on this Civi.
scripts/spike-file-upload.mjs (new)
The actual end-to-end test that proved out the pipeline before
wiring. Safe to re-run on any Civi instance during future audits.
Not in this change:
- Orphan attachment cleanup (CiviCRM scheduled job, Civi admin scope)
- Per-field MIME allowlists (single global list for v1)
- S3 / presigned-URL path for >5 MB files (deferred; capped at 5 MB
today to stay under Amplify Lambda's 6 MB sync payload limit)
Lays groundwork for closing the file-upload gap discovered while wiring
the org-contact custom fields. Currently no file fields in the form
actually persist to CiviCRM -- the renderer FileList drops at the
onSubmit JSON.stringify, and there is no /api/upload route or
Attachment.create call anywhere.
This commit adds:
1. scripts/spike-attachment-upload.mjs
One-off spike to answer the open question that gates the rest of the
work: does APIv4 Attachment.create accept an unbound upload, or must
we attach to an entity at create time? If unbound works we can use
the planned two-step pattern (upload returns a file id; submit
references it). If not, activity-bound file fields need a different
flow because the activity does not exist yet at upload time.
The spike also exercises the Contact.update + .file_name read-back
path against the Certificate_of_Incorporation field on a real org
contact, then cleans up after itself.
Usage:
node --env-file=.env.local scripts/spike-attachment-upload.mjs \
--org-id=<id> [--keep]
2. app/api/upload/route.ts
Structural pieces that do not depend on the spike outcome:
- multipart parsing via Request.formData()
- 5 MB hard cap (under Amplify Lambda 6 MB sync payload limit)
- MIME allowlist (PDF, DOC/DOCX, XLS/XLSX, JPEG/PNG/GIF/WEBP)
- magic-byte sniff to cross-check the client-reported MIME
- filename sanitization (path traversal scrub, length cap)
- checksum verification, rate limiting, field-ref allowlist
- STUB-mode short-circuit for local dev without live Civi
- explicit 501 where the Civi Attachment.create wiring goes,
with a comment pointing at the spike that resolves it
Result: endpoint compiles, registers as a Next route, returns 501
with a clear message; build passes; nothing wired into the frontend
yet so the existing form is unaffected.
Phase 2 (renderer upload-on-pick), Phase 3 (submit reshape), Phase 4
(promote Certificate_of_Incorporation to editable) follow once the
spike output picks the Attachment.create variant.
Adds four fields from the Organization Contact's Food_Co_op_Organizing
custom group to the Stage 0 (always-visible) section:
Date Incorporated (date, editable)
Name on Incorporation Certificate (text, editable)
Certificate of Incorporation (readonly; see note)
Equity share (currency, editable)
These live on the Organization Contact record, not on the Check-in
activity, so they read/write through a different code path:
- FieldConfig gains civiContactField, mutually exclusive with civiField
- /api/data extends the org Contact.get select to include them and
merges values into the prefill payload keyed by form-side name
- /api/submit splits incoming values: contact-bound fields go through
Contact.update (run first), activity-bound fields stay in the
Activity.create call (run second)
- FieldRenderer readonly branch now detects file-shaped values
({id, file_name}) and displays the filename rather than [object Object]
Certificate_of_Incorporation is wired readonly only: the form's
file-upload pipeline is not actually wired end-to-end (FileList drops
at JSON.stringify in onSubmit; no /api/upload endpoint exists). A
follow-up will close that gap.
Also adds scripts/inspect-org-custom-fields.mjs, a one-off introspection
script for dumping CustomField metadata when wiring a new group.
Updates the tool product name across the app UI (header, page title,
section labels, form buttons, success/error states, report stat labels),
README, deployment docs, and the CiviCRM email template guidance.
Custom domain references move from check-in.fci.coop to survey.fci.coop
(DNS update still required).
The underlying CiviCRM "Check-in (organizing)" activity type, custom
group machine names (Check_in_data__organizing_), health-check ids,
and the internal org_engagement_check_in form id are unchanged --
those are CiviCRM contract surfaces, not product copy.
Sweeps every active label / help / intro / visible paragraph in the
app to use 'member-owner' terminology consistently. Stub option
labels for the Capital Stack (Member equity / Member loans) updated
on the label side; option `value:` strings stay as the CRM-side
stored values. Sentinel comparisons in ReportView's chart legend
updated to track the new field labels (otherwise the `(custom
label)` parenthetical would print spuriously even at default).
Untouched on purpose:
- Civi machine names (`Members__current_`, `Member_*`,
`*_Member*`) — wire-level identifiers, must match Civi.
- Option-group `value:` strings — CRM-stored values, must match.
- NCG_Member / INFRA_Member option labels — these refer to a
co-op's membership in distributor networks, not member-owners.
- Commented-out fields and technical comments referencing Civi
field names.
- Remap globals.css tokens to FCI brand palette (Eggplant #801d7f,
Spring Pea #96bc33, Seed Grant #679038, Squash #c9ad2d, FCI gray
#4b5657). Existing leaf-* / clay-* class names preserved.
- Switch body font to Open Sans (FCI's free fallback for Museo Sans).
Headings keep Fraunces.
- Add contact identity (first name, last name, email) as readonly
fields at the top of Stage 0. /api/data fetches via APIv4
Contact.get with email_primary.email join; values flow through
FormDataPayload.contact and into the form's evalState so the
readonly renderer displays them. Draft restore re-applies them so
a stale local draft can't override.
- amplify.yml: fetch Amplify Secrets from SSM Parameter Store when
they don't arrive as build-shell env vars (the common failure mode
behind "Refusing to run in production without CIVI_*"). Adds a
length-only diagnostic echo and a hard-fail guard so a missing
required var stops the build with a clear message instead of
bundling empty strings and crashing the SSR Lambda at runtime.
Two visual additions to the read-only activity report.
Sparkline: when the user expands earlier-entries on a numeric field
(number/currency/percent) with two or more numeric points, the
expansion now leads with a 240x56 inline SVG trend chart — chronological
polyline, faint area fill, small dots at every measurement, a slightly
larger emphasized dot on the most recent point. Min and max captions
sit beneath in tabular-nums, formatted in the field's native style
(currency uses Intl, percent appends %, etc.). Non-numeric fields are
unchanged.
DateTimeline: a new card between the context header and the section
accordions. Walks every date-type field in stage sections 1-5 (Stage 0
omitted as it isn't a stage in the journey sense), pulls each field's
most-recent entered date, and lays the events out in five horizontal
swim lanes — one per stage rank, labeled at the left. Time axis
spans from the earliest event to max(latest event, Date_Opened).
Stage 5's Date_Opened is rendered as a larger clay-700 dot with a
heavier ring so it reads as the journey's anchor at the right end.
A faint clay-300 dashed vertical line marks 'today' if it falls
within the range. Color scale across stages is leaf-300 / leaf-500
/ leaf-600 / leaf-700 / clay-700 — a sprout-to-fruit gradient that
matches the existing palette. Empty stage rows still draw their lane
line at half opacity so the structure stays readable. SR-only event
list provides screen-reader access to all plotted dates with their
labels.
Stub payload enriched with four cross-stage date entries so the
timeline has content in dev preview.
CiviCRM APIv4 file custom fields return a bare file id by default; an
extra '.file_name' join is required to get the human-readable filename.
Both the form prefill walk (lib/prefill.ts) and the report walk
(app/api/report/route.ts) now request '<civiField>.file_name' for every
file-type field alongside the primary value, and wrap the prefill into
a { id, file_name } object so downstream UI has both. Falls back to
file_name undefined when the join returns null (eg orphaned id).
The form's FilePriorIndicator already accepts the object shape, so it
now shows the filename inline. The report's FormattedValue gets a
matching case: renders the file_name string if present, falls back to
'Attachment #<id>' when only the id came through.
Mirrors the form's IA and Field Almanac aesthetic; same auth (cid/cs
checksum) so the org owner who can fill the form can also view its
history.
New routes:
- GET /api/report — verifies checksum, resolves the org via the
Primary Contact relationship, fires Contact.get + Activity.get +
OptionValue.get in parallel. For every form field with a civiField,
walks all the org's Check-in (organizing) activities and collects
every non-empty value into a sorted-DESC history list. Returns
ReportPayload (orgName, currentStage, activities, fieldHistory,
options). Has stub-mode payload for env-less local dev.
- /report — page entry; same layout shell (SiteHeader + SiteFooter,
3xl page width). Eyebrow "Activity report - Co-op organizing".
ReportView component:
- ReportContextHeader: large org name, progress dots + uppercase
"Current stage" eyebrow + the Civi option *label* on its own line
at display-font xl/2xl leaf-800 (matches the form's header). Below
it a 3-up stat band: total check-ins, fields tracked, date span.
- One accordion card per stage section, in stage-rank order. Only
sections that have at least one field-with-entries render — past,
current, or "future-with-data" all welcome; truly empty stages stay
hidden so the page is calm.
- Same journey rail (md+) and mobile stem (md-) with past =
check-filled-leaf, current = filled-leaf-with-ring, future = dashed
hollow ring; solid leaf line vs dashed muted between markers.
- Within each card: divide-y rows. Field label and help on the left,
most-recent value on the right in display-font lg leaf-800, dated
beneath with an "{N} earlier entries" disclosure that expands a
small vertical timeline (date on left, value on right).
- FormattedValue handles currency (Intl), percent, number (tabular
nums), date (long, timezone-safe for YYYY-MM-DD), boolean (Yes/No),
select/readonly (resolved via option group), multiselect (handles
array or delimited string), file (filename), text-like (as-is).
- Loading / empty / error states match the form's treatments.
types/form.ts: new FieldHistoryEntry, ActivitySummary, ReportPayload.
The fieldHistory map keys by FieldConfig.name and only includes fields
that have at least one non-empty entry.
Stage authority moves from Organization.Food_Co_op_Organizing.Stage to the
most recent Check-in (organizing) activity whose Stage custom field is set.
Staff create these activities manually to mark transitions; org-owner form
submissions no longer write the Stage field at all, so they cannot override
a staff-set transition.
- /api/data: removed the Contact.get for org-side Stage; added an
Activity.get filtered to ACTIVITY_TYPE_NAME + ACTIVITY_STAGE_FIELD IS
NOT EMPTY, ordered by activity_date_time DESC, id DESC, limit 1.
Fallback when no such activity exists: Inquiry (rank 0). Org-name
lookup, stage activity, prefill, and option-group fetch all run in
parallel via Promise.all.
- /api/submit: removed the stageAtSubmission read + the
[ACTIVITY_STAGE_FIELD] write on the activity record. The form's
activities are stage-null by design now.
- config/form.ts: dropped the stage_at_submission readonly field (no
longer being set or displayed). Kept ACTIVITY_STAGE_FIELD export — it's
now used by /api/data to find stage-bearing activities. Updated the
current_stage field comment to reflect the new source.
- components/EngagementForm.tsx: dropped stage_at_submission from
evalState (no longer referenced by any visibility rule or readonly
display).
Org.Food_Co_op_Organizing.Stage remains in CiviCRM for staff list views;
the middleware no longer reads or writes it. No backfill required —
orgs without a stage-bearing activity simply read as Inquiry.
- Initialize Next.js project with Tailwind CSS
- Create CiviCRM APIv4 integration layer
- Implement stage-based form visibility (stages 0-5)
- Add field mapping configuration for CRM-to-form linking
- Create API routes for data retrieval and submission
- Record form submissions as CiviCRM activities
- Support dynamic contactId and orgId via URL parameters
- Ensure robust form state management with react-hook-form
Co-authored-by: joelbrock <52835+joelbrock@users.noreply.github.com>
Joel Brock