Closes the file-upload gap. Files now actually land in CiviCRM (verified
empirically against the live Civi instance via spike scripts).
Spike findings (see scripts/spike-file-upload.mjs):
- APIv4 Attachment is NOT exposed on this Civi
- APIv4 File + EntityFile ARE exposed; File.create accepts inline
base64 `content` and returns a usable file id
- Custom file fields store the file id directly in the custom column,
so EntityFile linkage is unnecessary for this use case
- Round-trip via Contact.update + Contact.get .file_name join verified
on a real org contact
Pipeline:
Renderer (FileField) picks up onChange →
POST /api/upload (multipart) with file + cid + cs + fieldRef →
verifyChecksum, MIME allowlist + magic-byte sniff, 5 MB cap →
civi.File.create({ file_name, mime_type, content: base64 }) →
returns { id, file_name } →
renderer stores in RHF state via setValue
Form submit →
POST /api/submit (JSON) with the {id, file_name} value →
submit detects the file shape and writes the id as the value of
the activity/contact custom field
File changes:
app/api/upload/route.ts
Replaced the 501 stub with the real File.create call. Comment
documents that EntityFile linkage is intentionally skipped and that
orphan cleanup is owned by a CiviCRM scheduled job.
app/api/submit/route.ts
For type:"file" values shaped as {id, file_name}, write the id as
the custom field value (activity or contact, depending on the
civiField / civiContactField the field declares).
components/fields/FieldRenderer.tsx
Replaced the bare <input type=file> register() with FileField, an
upload-on-pick subcomponent. The native input is NOT register()'d:
its FileList value was the original bug. FileField owns its
uploading + error state and writes {id, file_name} via setValue on
success. Submit is blocked upstream while uploads are in flight.
components/StageSection.tsx, components/EngagementForm.tsx
Thread setValue, cid, cs, and an onUploadStateChange callback
through to FieldRenderer. EngagementForm tracks uploads-in-flight
count; onSubmit refuses to submit while the count is > 0.
config/form.ts
Promotes Certificate_of_Incorporation from readonly to a real
file field now that the pipeline works.
app/api/data/route.ts
Drops the readonly carveout that was only needed while the
certificate was readonly.
scripts/list-civi-entities.mjs (new)
APIv4 entity probe + APIv3 attachment-API probe. Used to determine
that File (not Attachment) was the right entity on this Civi.
scripts/spike-file-upload.mjs (new)
The actual end-to-end test that proved out the pipeline before
wiring. Safe to re-run on any Civi instance during future audits.
Not in this change:
- Orphan attachment cleanup (CiviCRM scheduled job, Civi admin scope)
- Per-field MIME allowlists (single global list for v1)
- S3 / presigned-URL path for >5 MB files (deferred; capped at 5 MB
today to stay under Amplify Lambda's 6 MB sync payload limit)
Lays groundwork for closing the file-upload gap discovered while wiring
the org-contact custom fields. Currently no file fields in the form
actually persist to CiviCRM -- the renderer FileList drops at the
onSubmit JSON.stringify, and there is no /api/upload route or
Attachment.create call anywhere.
This commit adds:
1. scripts/spike-attachment-upload.mjs
One-off spike to answer the open question that gates the rest of the
work: does APIv4 Attachment.create accept an unbound upload, or must
we attach to an entity at create time? If unbound works we can use
the planned two-step pattern (upload returns a file id; submit
references it). If not, activity-bound file fields need a different
flow because the activity does not exist yet at upload time.
The spike also exercises the Contact.update + .file_name read-back
path against the Certificate_of_Incorporation field on a real org
contact, then cleans up after itself.
Usage:
node --env-file=.env.local scripts/spike-attachment-upload.mjs \
--org-id=<id> [--keep]
2. app/api/upload/route.ts
Structural pieces that do not depend on the spike outcome:
- multipart parsing via Request.formData()
- 5 MB hard cap (under Amplify Lambda 6 MB sync payload limit)
- MIME allowlist (PDF, DOC/DOCX, XLS/XLSX, JPEG/PNG/GIF/WEBP)
- magic-byte sniff to cross-check the client-reported MIME
- filename sanitization (path traversal scrub, length cap)
- checksum verification, rate limiting, field-ref allowlist
- STUB-mode short-circuit for local dev without live Civi
- explicit 501 where the Civi Attachment.create wiring goes,
with a comment pointing at the spike that resolves it
Result: endpoint compiles, registers as a Next route, returns 501
with a clear message; build passes; nothing wired into the frontend
yet so the existing form is unaffected.
Phase 2 (renderer upload-on-pick), Phase 3 (submit reshape), Phase 4
(promote Certificate_of_Incorporation to editable) follow once the
spike output picks the Attachment.create variant.
Adds four fields from the Organization Contact's Food_Co_op_Organizing
custom group to the Stage 0 (always-visible) section:
Date Incorporated (date, editable)
Name on Incorporation Certificate (text, editable)
Certificate of Incorporation (readonly; see note)
Equity share (currency, editable)
These live on the Organization Contact record, not on the Check-in
activity, so they read/write through a different code path:
- FieldConfig gains civiContactField, mutually exclusive with civiField
- /api/data extends the org Contact.get select to include them and
merges values into the prefill payload keyed by form-side name
- /api/submit splits incoming values: contact-bound fields go through
Contact.update (run first), activity-bound fields stay in the
Activity.create call (run second)
- FieldRenderer readonly branch now detects file-shaped values
({id, file_name}) and displays the filename rather than [object Object]
Certificate_of_Incorporation is wired readonly only: the form's
file-upload pipeline is not actually wired end-to-end (FileList drops
at JSON.stringify in onSubmit; no /api/upload endpoint exists). A
follow-up will close that gap.
Also adds scripts/inspect-org-custom-fields.mjs, a one-off introspection
script for dumping CustomField metadata when wiring a new group.
Multi-line field whose last property had no trailing comma got
corrupted into invalid JS:
civiField: `${G0}.X`
help: "...",
(JS requires the comma between properties.) The insertion path now
checks the last non-whitespace, non-comma char of the property line
just above the closing brace; if it isn't already a comma, one is
inserted before the new help line is spliced in.
Two additions, both touching the form-config story:
1. scripts/sync-help-from-civi.mjs
Diffs per-field help text in config/form.ts against CustomField rows
in CiviCRM and (with --write) updates the file in place. Reads env
from .env.local via Node's --env-file flag. Run as `npm run sync-help`
or `npm run sync-help -- --write`. A --debug mode prints the parser's
field list without calling Civi.
Rationale: this form is low-traffic and help text doesn't change
often once in production. A manual one-off sync is leaner than
coupling every page load (or every build) to a Civi API call.
2. fieldGroups: visual clustering of related fields within a section
New optional FieldGroupConfig overlay on StageSectionConfig — pure
presentation, names existing fields by name so submit/visibility
logic walks them unchanged. StageSection.tsx pulls grouped fields
out of the standalone per-field grid and renders each group as its
own bordered card with an optional heading. Stage 2 now clusters
Market Study, Pro Forma, Business Plan, and Board Self Assessment
(each a date + upload pair) into their own cards.
Joel Brock