APIv4 Attachment isn't exposed on this Civi install (confirmed in the
June 2026 upload-spike notes), so the Attachment.get call we shipped at
63e73e7 silently returned nothing and we fell through to the bare
/civicrm/file URL — which crashes Civi on a null fcs JWT decode.
APIv3 Attachment.get IS exposed and returns the signed URL with fcs
baked in (verified against id=150 in the user's API Explorer):
"url": "https://.../civicrm/file?reset=1&id=150&fcs=<JWT>"
Changes:
- lib/civicrm.ts: add a civi3() helper that calls /civicrm/ajax/rest with
AuthX headers, normalizing v3's array-or-keyed-object values shape into
a plain array.
- app/api/staff/report/route.ts: replace the dead v4 Attachment.get with
civi3("Attachment", "get", { id: {IN: [...]}, return: ["id","url"] }).
Each file's url goes into the value payload as before, so the frontend
needs no change.
Fallback chain remains intact: if Attachment.get fails (auth, endpoint
unavailable, etc.) the frontend still uses the /civicrm/webform-mw/file
extension route from b65bc6d/41467bd.
Security-review follow-up to b65bc6d. The file-redirect route signs an
fcs JWT that Civi's /civicrm/file handler accepts as proof of access.
Any user with the base `access CiviCRM` permission could iterate file
IDs and have us laundering tokens past the entity-level ACLs that would
normally apply (e.g. a staff user without view permission on a given
contact could still pull files attached to that contact).
Tighten it:
- Resolve the file's linked entity_table + entity_id (was: entity_id only).
- Run the entity-type's native permission check before signing the JWT:
civicrm_activity -> CRM_Activity_BAO_Activity::checkPermission
civicrm_contact -> CRM_Contact_BAO_Contact_Permission::allow
Unknown entity types deny by default — adding a new type requires an
explicit edit here, so we don't accidentally widen the surface.
- Drop JWT lifetime from a week to 10 minutes. The token is minted at
click time (the user hits this route fresh on each file click), so
the long lifetime served no purpose and made each URL a longer-lived
bearer credential.
Files missing from civicrm_entity_file or pointing at unsupported entity
types now 403 via CRM_Utils_System::permissionDenied() instead of
producing a download URL.
Path 1 (APIv4 Attachment.get + select url) shipped but didn't fix the
Firebase\JWT decode crash — the deployed Civi version either omits `url`
from Attachment.get or returns it without the fcs param. Falling back to
the bare /civicrm/file?id=X URL hits the same JWT null crash.
Path 2: route file clicks through a tiny redirect endpoint in the Civi
extension instead. The extension runs PHP on Civi, has access to the
crypto.jwt service, and mints the same shape of token Civi's own file
URL builder uses ({exp, "civi.file": <id>}) before 302-redirecting to
the canonical /civicrm/file URL.
Civi extension changes:
- New CRM/WebformMw/Page/File.php — resolves eid from civicrm_entity_file
if not supplied, signs a 7-day JWT via Civi::service('crypto.jwt'),
redirects.
- xml/Menu/webform_mw.xml — registers civicrm/webform-mw/file. Requires
`access CiviCRM` (the user is already authenticated in the parent Civi
tab when they click the link).
Frontend (StaffReportView.tsx, FieldValue):
- When Attachment.get's url is missing, fall back to the new extension
route instead of bare /civicrm/file. Attachment.get's url remains the
fast path when present.
Deploy: admin needs to push the updated extension files to the Civi
server, then Disable/Enable webform-mw (or cv flush) so the new menu
route registers in civicrm_menu.
The Y1 Monthly Sales Target fields don't match the Y1_M<n>_<metric> regex
the monthly matrix collector uses to discover rows:
M1 -> Y1_Monthly_Sales_Targets (no _M1 suffix; trailing 's')
M2 -> Y1_Monthly_Sales_Targets_M2 (plural with _M2)
M3 -> Y1_Monthly_Sales_Target_M2 (Civi name says _M2 but the value
represents M3; pre-existing
schema error)
M4..M12 -> Y1_Monthly_Sales_Target_M<n>
Hardcode a period->civi-field-name map (Y1_MONTHLY_SALES_TARGET_FIELDS)
so the monthly matrix can pick these up alongside the regex-matched
Y1_M<n>_Actual_Sales / Y1_M<n>_Transactions rows. The M3->_M2
irregularity is called out inline so a future reader doesn't "fix" it
into a regression.
Civi serves uploaded files at /civicrm/file?id=X&eid=Y&fcs=<JWT>; the fcs
is a JWT signed with the site key. Without it, Civi's file handler crashes
on a null JWT decode (Firebase\JWT\JWT::decode argument null). We don't
have the site key on the Next.js side, so let Civi mint the URLs for us.
Backend (/api/staff/report):
- Add file_name selects for org-side file fields (Certificate of
Incorporation and friends) so org files have names alongside URLs.
- Collect every file id referenced by activity and org custom fields.
- Call APIv4 Attachment.get with select: ["id", "url"] to fetch signed
URLs in one round trip. Build a urlByFileId map.
- Org-side file values are now wrapped in { id, file_name, url } shape
matching the activity-side files (previously bare file ids that the
frontend couldn't render).
- Activity-side file values gain a url property from the map.
- If Attachment.get doesn't expose url on this Civi version, the call is
caught and we fall through to bare URLs without fcs (no regression).
Frontend (FieldValue):
- Prefer v.url when present, normalizing absolute and relative shapes
against CIVI_BASE_URL.
- Fall back to /civicrm/file?reset=1&id=X if url wasn't provided.
Five related refinements to the staff report:
1. Surface latest submitter. Pull the most recent non-empty
Survey_completed_by / Survey_completed_by_email values from the
Check_in_data__organizing_ history and render them just below the org
stats in the report header. Email is a mailto: link. Hidden when both
values are empty.
2. Y1 monthly matrix. Generalize the Y1 matrix collector to detect either
Y1_Q<n>_<metric> or Y1_M<n>_<metric> field-name patterns. Stage 5 now
renders the quarterly table (when present) followed by the monthly
table (when present); each table auto-labels its columns Q1..Qn or
M1..Mn from the data, and the caption reflects the cadence. Adding a
new Y1_M<n>_<metric> field in Civi extends the columns automatically.
3. Larger field value. The latest value in each CompactFieldRow is now
font-display text-xl text-leaf-800 (previously text-[13px] text-ink-soft).
Makes the current number the dominant element in each row.
4. Smaller right-aligned earlier-entries toggle. The "N earlier entries"
button moves out of the inline date line onto its own row beneath the
"as of <date>" caption, right-aligned, in a 10px link style.
5. Right-aligned expanded entries. When earlier entries are unhidden,
each row now shows date on the left and the value on the right, mirroring
the active value's right alignment. Values render in font-display text-base
text-ink-soft so they visually echo the latest value while being clearly
demoted in size and color. The list is constrained to max-w-[24rem] with
ml-auto so it sits under the active value column rather than spanning the
full row.
Captures the form-filler's name and email on every check-in. Both fields
are required; values write back to Check_in_data__organizing_.Survey_completed_by
and Survey_completed_by_email on the activity, giving us a per-submission
record of who filled out which check-in.
Implementation:
- config/form.ts: new submitterInfo section (rank -1) at the head of the
sections array. rank -1 keeps it out of the past/current/future stage
pathway computation.
- components/EngagementForm.tsx: filter the submitter section out of
sectionsToRender and render it directly with FieldRenderer inside a
bordered card above the stage list. The fields still flow through RHF
registration, onInvalid scroll-to-error, and the onSubmit visibility
filter the same as any other field.
The staff report auto-discovers these fields via CustomField.get since
they live in Check_in_data__organizing_, so the field history shows up
in the report with no extra wiring.
The iframe in the Engagement Report tab is sized to fit content, so it
has no internal scroll context. Clicking an anchor link inside it
changes the URL hash but the iframe content doesn't move and the user
has to scroll the outer CiviCRM page manually. Rather than coordinate
cross-frame scroll with the parent, just skip rendering the anchor
strip when framed. Standalone view is unchanged.
The framed report posts its content height to the parent so the Civi tab
can resize the iframe to fit. Two pieces interacted badly:
- The root layout sets html.h-full and body.min-h-full, so documentElement
and body heights track the iframe's viewport height.
- The parent template sets iframe.height = postedHeight + 24 every time
a height message arrives.
The combination produced an unbounded feedback loop: parent grows the
iframe by 24px, viewport grows, document height grows, ResizeObserver
fires, we post the new height, parent grows by another 24px. The outer
CiviCRM page scrollbar visibly shrank each cycle.
Fix on the report side (no extension change needed): when framed, override
html height to auto and body min-height to 0 so the document decouples
from the viewport. Observe body (the actual content), measure
body.scrollHeight, and skip posting when the value is unchanged. Original
styles are restored on unmount so route changes back to the standalone
view still work.
The Engagement Report tab page renders templates/CRM/WebformMw/Page/Tab.tpl
via Smarty. Without an explicit template-dir registration, Smarty cannot
locate the file and the tab fails to render. Use CRM_Core_Smarty's
prependTemplateDir() (cleaner than splicing template_dir by hand).
Mirrors the same fix the CiviCRM admin applied on the server so the repo
source no longer drifts from the working deployed state.
Adds a native window.confirm() inside onSubmit, after the uploads-in-flight
guard and before the submitting-state flip. Runs only after react-hook-form
validation passes, so users who hit Submit on an incomplete form still see
the existing field-level errors via onInvalid rather than a confusing
"are you sure?" prompt. Cancelling leaves the form state untouched.
The Engagement Report tab on Organization contact pages was loading the
contact-view summary recursively inside its own tab pane. Root cause:
Civi's menu router was not picking up the extension's xml/Menu file, so
`civicrm/contact/view/engagement-report` fell back to the parent
`civicrm/contact/view` route. Adding an explicit hook_civicrm_xmlMenu
implementation forces the menu file to register, after which the route
resolves to CRM_WebformMw_Page_Tab and the iframe renders as intended.
Deploy: replace the extension files on the Civi server, then in
Administer → System Settings → Extensions Disable + re-Enable webform-mw
(or run `cv flush` on the server) so the menu cache is rebuilt.
No fields on the survey should block submit — removes required:true
from Preliminary_Market_Assessment so the form is fully optional end
to end.
Surfaces Stage 0 sources-and-uses currency fields (Total_cost_of_project,
Member_equity_raised, Member_loans_raised, Member_preferred_shares_raised,
Bank_debt_raised, Grants_Donations_Raised, Other_sources_raised) starting
at Business Feasibility instead of Stabilize so they can be filled in
earlier in the lifecycle.
Cross-references the Civi-side cutover checklist from the deploy doc
so anyone deploying knows the CiviCRM prerequisites are tracked in a
single place.
App side:
- Per-route CSP: /staff/report now sets frame-ancestors 'self'
<CIVI_BASE_URL origin> and drops X-Frame-Options so the CiviCRM
extension can iframe it. All other routes keep frame-ancestors
'none' + X-Frame-Options: DENY via a path-negation source.
- Staff page recognises ?frame=1 and renders without SiteHeader/
SiteFooter so it fills the iframe cleanly.
- StaffReportView posts its scrollHeight to the parent window via
postMessage when framed; the Civi tab listens and auto-resizes
the iframe (no nested scrollbar). Anchor strip drops its sticky
positioning in frame mode since there's no internal scroll.
CiviCRM extension (civi-extension/webform-mw/, key webform-mw):
- info.xml + main hook file (webform_mw.php) implementing
hook_civicrm_tabset to add an 'Engagement Report' tab to
Organization contact-view pages.
- CRM/WebformMw/Page/Tab.php + Smarty template render an iframe
pointing at <WEBFORM_MW_APP_URL>/staff/report?org=<cid>&key=&frame=1,
with a postMessage listener that validates event.origin against
the configured app URL before resizing.
- Config via PHP constants in civicrm.settings.php (WEBFORM_MW_APP_URL,
WEBFORM_MW_STAFF_KEY) or matching env vars. Help banner shown when
unconfigured.
- README documents install, config, behaviour, security caveats.
UX iteration after first live look:
- Sticky anchor strip below the header with a chip per section (incl.
Submissions) so staff can jump around a long page.
- Compact one-line rows that show only the latest value; multi-history
fields get a muted 'N earlier entries' toggle that reveals the rest
inline. Same affordance for file fields.
- Empty fields collapse under a single 'N empty fields' toggle per
section instead of taking a row each.
- Stage 5: Y1_Q<n>_<metric> fields render as a read-only matrix table
(rows: metrics; columns: Q1..Q4) matching the form's matrix layout.
File proxy (/api/staff/file) deleted. APIv4 Attachment isn't exposed
on this Civi instance (per the June upload spike), which is why the
previous proxy returned broken images. Staff are already authenticated
to Civi when they arrive here, so file fields now render as outbound
links to CIVI_BASE_URL/civicrm/file?reset=1&id=<id> and the browser
uses the staff session. No more proxy auth, no more SSRF surface to
harden, no broken images.
CIVI_BASE_URL flows from the staff page (server component) into the
client as a prop. No secret material crosses the boundary.
Without this, even after setting STAFF_REPORT_KEY in the Amplify Secrets
tab the value never reaches the SSR Lambda — the build script only writes
the listed env vars into .env.production, which is what Next bundles.
- Allowlist inline MIME types (png/jpeg/gif/webp/pdf only); everything
else, including SVG and HTML, served as application/octet-stream
with content-disposition: attachment.
- X-Content-Type-Options: nosniff and a restrictive CSP on every response.
- Validate the upstream URL Civi returns: must match CIVI_BASE_URL origin
before we attach basic-auth creds and follow it. redirect: manual to
prevent off-host hops.
- Drop SVG from the client's inline-image list (server forces download).
Closes the file-upload gap. Files now actually land in CiviCRM (verified
empirically against the live Civi instance via spike scripts).
Spike findings (see scripts/spike-file-upload.mjs):
- APIv4 Attachment is NOT exposed on this Civi
- APIv4 File + EntityFile ARE exposed; File.create accepts inline
base64 `content` and returns a usable file id
- Custom file fields store the file id directly in the custom column,
so EntityFile linkage is unnecessary for this use case
- Round-trip via Contact.update + Contact.get .file_name join verified
on a real org contact
Pipeline:
Renderer (FileField) picks up onChange →
POST /api/upload (multipart) with file + cid + cs + fieldRef →
verifyChecksum, MIME allowlist + magic-byte sniff, 5 MB cap →
civi.File.create({ file_name, mime_type, content: base64 }) →
returns { id, file_name } →
renderer stores in RHF state via setValue
Form submit →
POST /api/submit (JSON) with the {id, file_name} value →
submit detects the file shape and writes the id as the value of
the activity/contact custom field
File changes:
app/api/upload/route.ts
Replaced the 501 stub with the real File.create call. Comment
documents that EntityFile linkage is intentionally skipped and that
orphan cleanup is owned by a CiviCRM scheduled job.
app/api/submit/route.ts
For type:"file" values shaped as {id, file_name}, write the id as
the custom field value (activity or contact, depending on the
civiField / civiContactField the field declares).
components/fields/FieldRenderer.tsx
Replaced the bare <input type=file> register() with FileField, an
upload-on-pick subcomponent. The native input is NOT register()'d:
its FileList value was the original bug. FileField owns its
uploading + error state and writes {id, file_name} via setValue on
success. Submit is blocked upstream while uploads are in flight.
components/StageSection.tsx, components/EngagementForm.tsx
Thread setValue, cid, cs, and an onUploadStateChange callback
through to FieldRenderer. EngagementForm tracks uploads-in-flight
count; onSubmit refuses to submit while the count is > 0.
config/form.ts
Promotes Certificate_of_Incorporation from readonly to a real
file field now that the pipeline works.
app/api/data/route.ts
Drops the readonly carveout that was only needed while the
certificate was readonly.
scripts/list-civi-entities.mjs (new)
APIv4 entity probe + APIv3 attachment-API probe. Used to determine
that File (not Attachment) was the right entity on this Civi.
scripts/spike-file-upload.mjs (new)
The actual end-to-end test that proved out the pipeline before
wiring. Safe to re-run on any Civi instance during future audits.
Not in this change:
- Orphan attachment cleanup (CiviCRM scheduled job, Civi admin scope)
- Per-field MIME allowlists (single global list for v1)
- S3 / presigned-URL path for >5 MB files (deferred; capped at 5 MB
today to stay under Amplify Lambda's 6 MB sync payload limit)
Lays groundwork for closing the file-upload gap discovered while wiring
the org-contact custom fields. Currently no file fields in the form
actually persist to CiviCRM -- the renderer FileList drops at the
onSubmit JSON.stringify, and there is no /api/upload route or
Attachment.create call anywhere.
This commit adds:
1. scripts/spike-attachment-upload.mjs
One-off spike to answer the open question that gates the rest of the
work: does APIv4 Attachment.create accept an unbound upload, or must
we attach to an entity at create time? If unbound works we can use
the planned two-step pattern (upload returns a file id; submit
references it). If not, activity-bound file fields need a different
flow because the activity does not exist yet at upload time.
The spike also exercises the Contact.update + .file_name read-back
path against the Certificate_of_Incorporation field on a real org
contact, then cleans up after itself.
Usage:
node --env-file=.env.local scripts/spike-attachment-upload.mjs \
--org-id=<id> [--keep]
2. app/api/upload/route.ts
Structural pieces that do not depend on the spike outcome:
- multipart parsing via Request.formData()
- 5 MB hard cap (under Amplify Lambda 6 MB sync payload limit)
- MIME allowlist (PDF, DOC/DOCX, XLS/XLSX, JPEG/PNG/GIF/WEBP)
- magic-byte sniff to cross-check the client-reported MIME
- filename sanitization (path traversal scrub, length cap)
- checksum verification, rate limiting, field-ref allowlist
- STUB-mode short-circuit for local dev without live Civi
- explicit 501 where the Civi Attachment.create wiring goes,
with a comment pointing at the spike that resolves it
Result: endpoint compiles, registers as a Next route, returns 501
with a clear message; build passes; nothing wired into the frontend
yet so the existing form is unaffected.
Phase 2 (renderer upload-on-pick), Phase 3 (submit reshape), Phase 4
(promote Certificate_of_Incorporation to editable) follow once the
spike output picks the Attachment.create variant.
Adds four fields from the Organization Contact's Food_Co_op_Organizing
custom group to the Stage 0 (always-visible) section:
Date Incorporated (date, editable)
Name on Incorporation Certificate (text, editable)
Certificate of Incorporation (readonly; see note)
Equity share (currency, editable)
These live on the Organization Contact record, not on the Check-in
activity, so they read/write through a different code path:
- FieldConfig gains civiContactField, mutually exclusive with civiField
- /api/data extends the org Contact.get select to include them and
merges values into the prefill payload keyed by form-side name
- /api/submit splits incoming values: contact-bound fields go through
Contact.update (run first), activity-bound fields stay in the
Activity.create call (run second)
- FieldRenderer readonly branch now detects file-shaped values
({id, file_name}) and displays the filename rather than [object Object]
Certificate_of_Incorporation is wired readonly only: the form's
file-upload pipeline is not actually wired end-to-end (FileList drops
at JSON.stringify in onSubmit; no /api/upload endpoint exists). A
follow-up will close that gap.
Also adds scripts/inspect-org-custom-fields.mjs, a one-off introspection
script for dumping CustomField metadata when wiring a new group.
Updates the tool product name across the app UI (header, page title,
section labels, form buttons, success/error states, report stat labels),
README, deployment docs, and the CiviCRM email template guidance.
Custom domain references move from check-in.fci.coop to survey.fci.coop
(DNS update still required).
The underlying CiviCRM "Check-in (organizing)" activity type, custom
group machine names (Check_in_data__organizing_), health-check ids,
and the internal org_engagement_check_in form id are unchanged --
those are CiviCRM contract surfaces, not product copy.
Form: dropped the boxed card treatment for FieldGroupCard in favor
of a leaf-tinted left rule + small uppercase mini-label. Eats only
~14px of horizontal space (border + pl-3/sm:pl-4) instead of
~32-40px for the previous bg-tinted card with px-4/sm:px-5 on both
sides, so the inner 2-col grid keeps more breathing room for the
fields themselves.
Report: same field-group concept now applies to ReportSection.
Grouped FieldHistoryRows render together inside a leaf-tinted left
rule with a small label above. Walk preserves the declared field
order — a group is emitted at its first member's position; the
other members are skipped when the loop later reaches them. Mixes
cleanly with the existing MembershipChart inline insertion and the
divide-y rhythm of standalone rows.
Sweeps every active label / help / intro / visible paragraph in the
app to use 'member-owner' terminology consistently. Stub option
labels for the Capital Stack (Member equity / Member loans) updated
on the label side; option `value:` strings stay as the CRM-side
stored values. Sentinel comparisons in ReportView's chart legend
updated to track the new field labels (otherwise the `(custom
label)` parenthetical would print spuriously even at default).
Untouched on purpose:
- Civi machine names (`Members__current_`, `Member_*`,
`*_Member*`) — wire-level identifiers, must match Civi.
- Option-group `value:` strings — CRM-stored values, must match.
- NCG_Member / INFRA_Member option labels — these refer to a
co-op's membership in distributor networks, not member-owners.
- Commented-out fields and technical comments referencing Civi
field names.
Body-copy mailto links ("Chris @ FCI") and the chrome nav link
were rendering as plain text, hard to spot. Added an @layer base
rule that gives every `a[href]` a 1px underline at 2px offset and
the FCI Seed Grant green, with a subtle hover thicken. Tailwind
utilities still win on a per-element basis (the nav link keeps its
`text-ink-soft` color, the brand-image wrapper opts out via
`no-underline`).
Member_equity_raised and Member_loans_raised had their civiField
lines stripped of trailing commas when the script inserted help
text. Re-added the comma; build passes.
Multi-line field whose last property had no trailing comma got
corrupted into invalid JS:
civiField: `${G0}.X`
help: "...",
(JS requires the comma between properties.) The insertion path now
checks the last non-whitespace, non-comma char of the property line
just above the closing brace; if it isn't already a comma, one is
inserted before the new help line is spliced in.
More pair-clustering across stages: Stage 1 gets four (Preliminary
Market Assessment, Preliminary Sources & Uses, Vision, Business
Concept), Stage 3 gets one (Site Letter of Intent). Stage 2 groups
drop their labels — the pair structure reads on its own and the
extra heading was visual noise.
Bar's secondary line is now a horizontal cluster of the same six
pills used in the page header, followed by the viewed stage label.
Active pill expands and gains a slow halo (rail-pulse keyframe) when
in the bar; same component runs without the pulse in the static
header. Width transitions smoothly between ranks as the user scrolls,
so the indicator visibly tracks progress through the form.
StageProgress now takes an optional `pulse` prop and tightens its
transition timing for nicer scroll-driven animation.
Repurpose the previously-empty left side of the floating submit bar.
Top line is the org name; secondary line updates as the user scrolls
so the currently viewed stage is always visible even after the
top-of-form header has scrolled out of sight.
IntersectionObserver with a top-biased rootMargin tracks which
section is in view; topmost intersecting section wins ties.
Submit-state feedback (error / in-flight) still takes priority over
the viewing/draft text when active.
Two additions, both touching the form-config story:
1. scripts/sync-help-from-civi.mjs
Diffs per-field help text in config/form.ts against CustomField rows
in CiviCRM and (with --write) updates the file in place. Reads env
from .env.local via Node's --env-file flag. Run as `npm run sync-help`
or `npm run sync-help -- --write`. A --debug mode prints the parser's
field list without calling Civi.
Rationale: this form is low-traffic and help text doesn't change
often once in production. A manual one-off sync is leaner than
coupling every page load (or every build) to a Civi API call.
2. fieldGroups: visual clustering of related fields within a section
New optional FieldGroupConfig overlay on StageSectionConfig — pure
presentation, names existing fields by name so submit/visibility
logic walks them unchanged. StageSection.tsx pulls grouped fields
out of the standalone per-field grid and renders each group as its
own bordered card with an optional heading. Stage 2 now clusters
Market Study, Pro Forma, Business Plan, and Board Self Assessment
(each a date + upload pair) into their own cards.
Joel Brock