The staff-report iframe is expanded to full content height (no nested
scrollbar), so a modal <dialog> centered in its viewport landed in the
middle of the whole document and sat off-screen unless the parent page
was scrolled there. Cross-origin means the iframe can't read the
parent's scroll position itself.
Tab.tpl now broadcasts the iframe's visible slice (webform-mw-viewport,
rAF-throttled on scroll/resize/load and after each height change), and
AttachmentLightbox pins to that slice and fills its height. Falls back
to a fixed box when no viewport message arrives (older extension), and
standalone mode keeps native centering at a taller 85vh.
Extension bumped to 0.3.1 (template change -> re-copy + cv flush on
prod). PRODUCTION_CUTOVER.md change log updated.
Three quick fixes off first-deploy testing:
1. /api/staff/file 404'd for valid files. Refactor fileBelongsToOrg
to SELECT the org's and activities' file columns and JS-compare
instead of WHERE ... OR with custom field refs (APIv4 fragility
around nested OR + dotted custom fields). Same ownership probe,
same shape /api/staff/report itself uses to read file values.
2. Lightbox ballooned to full report height because the staff iframe
auto-grows to fit content (often 3000+ px). Cap to a fixed
640px x min(92vw, 900px) box so it stays a reasonable preview
regardless of iframe document size.
3. Production frame-ancestors blocked crm.fci.coop from iframing
survey.fci.coop -- the CSP only included the dev Civi origin
derived from CIVI_BASE_URL. Add CIVI_FRAME_ALLOWED_ORIGINS
(comma-separated) so one app deploy can be embedded by both
dev and prod Civi. Falls back to CIVI_BASE_URL for single-Civi
compatibility.
PRODUCTION_CUTOVER.md updated inline and in the change log.
Replaces the dropped entity_file→org check with a probe against the
actual ownership chain — the file_id stored in a custom-field column
on the org or on one of its activities.
For each request:
1. Discover file-typed CustomField refs in ACTIVITY_GROUP_NAMES and
ORG_GROUP_NAMES (one CustomField.get).
2. In parallel, probe:
- Contact.get(id=orgId) WHERE any org file field == fileId
- Activity.get(target=orgId) WHERE any activity file field == fileId
using APIv4 OR clauses.
3. Allow only if at least one probe returns a hit.
This is the same ownership the staff report itself uses to surface the
file — the proxy now refuses to broker bytes for any file id that
wouldn't appear in the org's own report. civicrm_entity_file remains
unused for auth (it's anchored to the submitter, not the org).
Two bugs surfaced on first dev-server test:
1. /api/staff/file 404s for valid file ids. The old per-org check
read civicrm_entity_file and required entity_id==orgId, but our
upload route anchors files to the submitter's contact id, not the
org's — the entity_file row is metadata-only on this install
(see comment in app/api/upload/route.ts). The custom-field column
is the real ownership signal, which /api/staff/report already uses,
and the staff key already gates org access. Drop the bogus check;
keep the entity_table whitelist as defence.
2. Same-origin PDF iframe blocked by frame-ancestors 'none'. The
strict global CSP excludes /staff/report; add /api/staff/file to
the same embed-friendly profile so the lightbox iframe can load.
Also move the sandbox/default-src 'none' CSP to the attachment path
only — a strict sandbox header breaks Chrome's PDF viewer on inline
responses (it needs to load fonts and plugin-mode rendering). On
inline we rely on the SAFE_INLINE_MIMES allowlist + X-Content-Type-
Options + the app's global CSP.
Defence in depth against XSS through a non-allowlisted upload path:
our /api/upload route validates mimes, but the underlying civicrm_file
row can be populated through other routes (Civi admin UI uploads,
imports). A row with mime_type=text/html or image/svg+xml would have
been served inline from this same-origin proxy.
- SAFE_INLINE_MIMES allowlist: png/jpeg/gif/webp/pdf only
- Anything outside it is rewritten to application/octet-stream plus
Content-Disposition: attachment so the browser downloads
- Adds Content-Security-Policy sandbox so even a mistaken inline serve
cannot run script or exfiltrate
Adds a /api/staff/file proxy that re-streams Civi attachments with
Content-Disposition: inline so a native <dialog> lightbox can preview
images and PDFs in place. Office docs keep their plain download link
and gain a "View in Google Docs" secondary link (uses the Civi-signed
URL so Google can fetch without our staff key).
Also threads mime through /api/staff/report (Attachment.get mime_type)
so the dispatcher picks the right affordance without relying solely on
filename inference.
Drops the trailing sentence about returning later for another check-in (now redundant with the post-submit CTA) and matches the rest of the UI's 'survey' terminology.
Each stage-transition activity now becomes a horizontal range on its lane: start = the activity's date, end = the next activity with a higher stage rank, or extending to today if still in effect. Milestone date-field dots overlay on top of the ranges. Activity dots at each range start carry a tooltip with the activity subject. Added computeStageRanges in lib/stageRank.ts and switched DateTimeline to take activities directly (deriving both the rank resolver and the ranges internally). Wide year-spanning timelines now show their full extent even when no milestone dates have been entered.
Each date-field event is now plotted on the lane corresponding to the Framework Stage the co-op was in on the field's stored date, resolved from the activity stream's stage transitions. Date_Opened is pinned to Stage 5 as the journey anchor. Events that pre-date any known transition fall back to the field's section rank so they still surface somewhere. Extracts the STAGE_RANK map (previously inline in ReportView) into lib/stageRank.ts alongside the new buildStageRankAtDate resolver factory.
Subtitle now tells respondents they can skip fields they do not know. Post-submit CTA says Submit an update since the same form is the ongoing update channel, not a one-off survey.
DateTimeline now skips section rank 0 (intake) and renders stage lanes
top-to-bottom 5→1, matching how the framework actually numbers stages.
MembershipChart's latest-Actual label was placed at the circle's
x + 6, which overflowed the SVG when the most-recent point landed near
the right padding. Anchor it to xOf(maxT) - 4 with textAnchor="end",
mirroring the Goal label so both endpoint labels stay inside the chart.
Every JSON-based upload path on this Civi stores the `content` field
verbatim on disk — confirmed against both APIv4 File.create AND APIv3
Attachment.create (both came back as base64 text in hex dumps). The
multipart `file` part to /civicrm/ajax/rest is also a dead end: APIv3
Attachment.create on this install doesn't see $_FILES (rejected with
"Mandatory key(s) missing: id or content or options.move-file").
The one path Civi honors is APIv3 Attachment.create + options.move-file
— pointing at a filesystem path the Civi server can read. So expose a
tiny multipart endpoint in the WebForm-mw Civi extension that copies
PHP's $_FILES['file']['tmp_name'] into the API call, then return the
new file id as JSON. PHP's $_FILES preserves binary natively.
Civi extension (requires admin deploy):
- CRM/WebformMw/Page/Upload.php : multipart POST handler. Validates
the upload, requires `access CiviCRM`, whitelists entity_table to
civicrm_contact|civicrm_activity, calls Attachment.create with
move-file pointing at the tmp upload, returns {id, name} JSON.
- xml/Menu/webform_mw.xml : registers civicrm/webform-mw/upload.
WebForm-mw side:
- lib/civicrm.ts : new civiMultipart() helper. POSTs multipart to an
arbitrary Civi path (not /civicrm/ajax/rest) with the same AuthX
headers. Returns the parsed JSON body.
- app/api/upload/route.ts : send the upload's bytes via civiMultipart
to civicrm/webform-mw/upload. Comment-block now records all four
upload paths we tried so a future reader doesn't repeat the cycle.
Deploy: admin syncs the updated civi-extension/webform-mw/ directory
and Disable/Re-enables the extension (or runs cv flush) so the new
menu route is registered.
The previous multipart attempt (f74fd07) was rejected by Civi with
"Mandatory key(s) missing: id or content or options.move-file" — the
/civicrm/ajax/rest endpoint on this install doesn't expose $_FILES to
v3 actions; only params['content'] is consulted. Our multipart `file`
part was ignored.
APIv3 Attachment.create has historically been the file-upload entry
point used by Civi's own form widgets and auto-decodes the `content`
field from base64, unlike v4 File.create which stores it verbatim.
Send the same shape we tried first (entity_table, entity_id, name,
mime_type, content=base64) but to v3 Attachment instead of v4 File.
The civi3Upload helper in lib/civicrm.ts is kept in place — it's not
useful for this endpoint but the multipart-POST shape may be needed
later for other Civi entities that do read $_FILES.
APIv4 File.create on this Civi install stores the `content` field
verbatim — no base64 decoding. The hex dump of a downloaded file
confirms it: bytes start with 69 56 42 4f ("iVBO...") which is the
base64 encoding of the PNG header (89 50 4e 47), not the header itself.
Every file uploaded via the form has been corrupt on disk since launch.
JSON can't carry binary safely (high bytes break UTF-8), so the fix is
to stop trying. APIv3 Attachment.create accepts a multipart `file` part
the standard way (read from $_FILES on the server side) which preserves
bytes exactly.
Changes:
- lib/civicrm.ts: new civi3Upload() helper. POSTs multipart/form-data
with `entity`, `action`, `json`, and `file` parts to /civicrm/ajax/rest
using the same AuthX headers as civi3(). Wraps the Uint8Array into an
ArrayBuffer slice so Blob's narrower BlobPart typing accepts it.
- app/api/upload/route.ts: replace the v4 File.create JSON call with
civi3Upload("Attachment", "create", ...). Attachment.create requires
an entity context, so anchor to the form-filler's contact id. Our
custom-field flow uses the returned file id directly (no entity_file
linkage needed for prefill/download), so the extra civicrm_entity_file
row is metadata-only.
Note: existing files in Civi (uploaded via the buggy path) are still
corrupt on disk. New uploads will be intact. To recover the old ones
the user would need to re-upload via the form, or run a one-off
backfill that reads the base64 text out of /civicrm.files/upload/ and
rewrites each file with its decoded bytes.
The {IN: [...]} operator object for `id` crashes Civi APIv3 on this
install — Civi's error renderer calls htmlentities() on the array
value and dies, producing the 500 with the html error page seen at
6c3e8dd's deploy.
Single-id calls succeed (confirmed in the user's API Explorer test
against id=150). So loop one call per file id and run them in
parallel via Promise.allSettled. Reports typically reference a
handful of files, so the round-trip overhead is small and individual
file failures don't poison the whole report.
Previous commit (8ace5f4) passed `return: ["id", "url"]` to APIv3
Attachment.get. APIv3 expects `return` as a comma-separated string
("id,url"); arrays are v4 syntax. Civi caught the type mismatch but
its error-rendering pathway then crashed on htmlentities() (which
was passed the offending array), producing a 500 with an HTML error
page rather than a clean JSON error.
Fix: pass return as "id,url" and add sequential:1 (canonical v3 client
shape — values come back as an array). The civi3 helper already
normalizes either response shape, but sequential matches what real
APIv3 clients send.
Logs to confirm after deploy:
- Success: no "[staff/report] Attachment.get (v3) failed" warning.
- File links resolve directly without bouncing off the
/civicrm/webform-mw/file extension route.
APIv4 Attachment isn't exposed on this Civi install (confirmed in the
June 2026 upload-spike notes), so the Attachment.get call we shipped at
63e73e7 silently returned nothing and we fell through to the bare
/civicrm/file URL — which crashes Civi on a null fcs JWT decode.
APIv3 Attachment.get IS exposed and returns the signed URL with fcs
baked in (verified against id=150 in the user's API Explorer):
"url": "https://.../civicrm/file?reset=1&id=150&fcs=<JWT>"
Changes:
- lib/civicrm.ts: add a civi3() helper that calls /civicrm/ajax/rest with
AuthX headers, normalizing v3's array-or-keyed-object values shape into
a plain array.
- app/api/staff/report/route.ts: replace the dead v4 Attachment.get with
civi3("Attachment", "get", { id: {IN: [...]}, return: ["id","url"] }).
Each file's url goes into the value payload as before, so the frontend
needs no change.
Fallback chain remains intact: if Attachment.get fails (auth, endpoint
unavailable, etc.) the frontend still uses the /civicrm/webform-mw/file
extension route from b65bc6d/41467bd.
Security-review follow-up to b65bc6d. The file-redirect route signs an
fcs JWT that Civi's /civicrm/file handler accepts as proof of access.
Any user with the base `access CiviCRM` permission could iterate file
IDs and have us laundering tokens past the entity-level ACLs that would
normally apply (e.g. a staff user without view permission on a given
contact could still pull files attached to that contact).
Tighten it:
- Resolve the file's linked entity_table + entity_id (was: entity_id only).
- Run the entity-type's native permission check before signing the JWT:
civicrm_activity -> CRM_Activity_BAO_Activity::checkPermission
civicrm_contact -> CRM_Contact_BAO_Contact_Permission::allow
Unknown entity types deny by default — adding a new type requires an
explicit edit here, so we don't accidentally widen the surface.
- Drop JWT lifetime from a week to 10 minutes. The token is minted at
click time (the user hits this route fresh on each file click), so
the long lifetime served no purpose and made each URL a longer-lived
bearer credential.
Files missing from civicrm_entity_file or pointing at unsupported entity
types now 403 via CRM_Utils_System::permissionDenied() instead of
producing a download URL.
Path 1 (APIv4 Attachment.get + select url) shipped but didn't fix the
Firebase\JWT decode crash — the deployed Civi version either omits `url`
from Attachment.get or returns it without the fcs param. Falling back to
the bare /civicrm/file?id=X URL hits the same JWT null crash.
Path 2: route file clicks through a tiny redirect endpoint in the Civi
extension instead. The extension runs PHP on Civi, has access to the
crypto.jwt service, and mints the same shape of token Civi's own file
URL builder uses ({exp, "civi.file": <id>}) before 302-redirecting to
the canonical /civicrm/file URL.
Civi extension changes:
- New CRM/WebformMw/Page/File.php — resolves eid from civicrm_entity_file
if not supplied, signs a 7-day JWT via Civi::service('crypto.jwt'),
redirects.
- xml/Menu/webform_mw.xml — registers civicrm/webform-mw/file. Requires
`access CiviCRM` (the user is already authenticated in the parent Civi
tab when they click the link).
Frontend (StaffReportView.tsx, FieldValue):
- When Attachment.get's url is missing, fall back to the new extension
route instead of bare /civicrm/file. Attachment.get's url remains the
fast path when present.
Deploy: admin needs to push the updated extension files to the Civi
server, then Disable/Enable webform-mw (or cv flush) so the new menu
route registers in civicrm_menu.
The Y1 Monthly Sales Target fields don't match the Y1_M<n>_<metric> regex
the monthly matrix collector uses to discover rows:
M1 -> Y1_Monthly_Sales_Targets (no _M1 suffix; trailing 's')
M2 -> Y1_Monthly_Sales_Targets_M2 (plural with _M2)
M3 -> Y1_Monthly_Sales_Target_M2 (Civi name says _M2 but the value
represents M3; pre-existing
schema error)
M4..M12 -> Y1_Monthly_Sales_Target_M<n>
Hardcode a period->civi-field-name map (Y1_MONTHLY_SALES_TARGET_FIELDS)
so the monthly matrix can pick these up alongside the regex-matched
Y1_M<n>_Actual_Sales / Y1_M<n>_Transactions rows. The M3->_M2
irregularity is called out inline so a future reader doesn't "fix" it
into a regression.
Civi serves uploaded files at /civicrm/file?id=X&eid=Y&fcs=<JWT>; the fcs
is a JWT signed with the site key. Without it, Civi's file handler crashes
on a null JWT decode (Firebase\JWT\JWT::decode argument null). We don't
have the site key on the Next.js side, so let Civi mint the URLs for us.
Backend (/api/staff/report):
- Add file_name selects for org-side file fields (Certificate of
Incorporation and friends) so org files have names alongside URLs.
- Collect every file id referenced by activity and org custom fields.
- Call APIv4 Attachment.get with select: ["id", "url"] to fetch signed
URLs in one round trip. Build a urlByFileId map.
- Org-side file values are now wrapped in { id, file_name, url } shape
matching the activity-side files (previously bare file ids that the
frontend couldn't render).
- Activity-side file values gain a url property from the map.
- If Attachment.get doesn't expose url on this Civi version, the call is
caught and we fall through to bare URLs without fcs (no regression).
Frontend (FieldValue):
- Prefer v.url when present, normalizing absolute and relative shapes
against CIVI_BASE_URL.
- Fall back to /civicrm/file?reset=1&id=X if url wasn't provided.
Five related refinements to the staff report:
1. Surface latest submitter. Pull the most recent non-empty
Survey_completed_by / Survey_completed_by_email values from the
Check_in_data__organizing_ history and render them just below the org
stats in the report header. Email is a mailto: link. Hidden when both
values are empty.
2. Y1 monthly matrix. Generalize the Y1 matrix collector to detect either
Y1_Q<n>_<metric> or Y1_M<n>_<metric> field-name patterns. Stage 5 now
renders the quarterly table (when present) followed by the monthly
table (when present); each table auto-labels its columns Q1..Qn or
M1..Mn from the data, and the caption reflects the cadence. Adding a
new Y1_M<n>_<metric> field in Civi extends the columns automatically.
3. Larger field value. The latest value in each CompactFieldRow is now
font-display text-xl text-leaf-800 (previously text-[13px] text-ink-soft).
Makes the current number the dominant element in each row.
4. Smaller right-aligned earlier-entries toggle. The "N earlier entries"
button moves out of the inline date line onto its own row beneath the
"as of <date>" caption, right-aligned, in a 10px link style.
5. Right-aligned expanded entries. When earlier entries are unhidden,
each row now shows date on the left and the value on the right, mirroring
the active value's right alignment. Values render in font-display text-base
text-ink-soft so they visually echo the latest value while being clearly
demoted in size and color. The list is constrained to max-w-[24rem] with
ml-auto so it sits under the active value column rather than spanning the
full row.
Captures the form-filler's name and email on every check-in. Both fields
are required; values write back to Check_in_data__organizing_.Survey_completed_by
and Survey_completed_by_email on the activity, giving us a per-submission
record of who filled out which check-in.
Implementation:
- config/form.ts: new submitterInfo section (rank -1) at the head of the
sections array. rank -1 keeps it out of the past/current/future stage
pathway computation.
- components/EngagementForm.tsx: filter the submitter section out of
sectionsToRender and render it directly with FieldRenderer inside a
bordered card above the stage list. The fields still flow through RHF
registration, onInvalid scroll-to-error, and the onSubmit visibility
filter the same as any other field.
The staff report auto-discovers these fields via CustomField.get since
they live in Check_in_data__organizing_, so the field history shows up
in the report with no extra wiring.
The iframe in the Engagement Report tab is sized to fit content, so it
has no internal scroll context. Clicking an anchor link inside it
changes the URL hash but the iframe content doesn't move and the user
has to scroll the outer CiviCRM page manually. Rather than coordinate
cross-frame scroll with the parent, just skip rendering the anchor
strip when framed. Standalone view is unchanged.
The framed report posts its content height to the parent so the Civi tab
can resize the iframe to fit. Two pieces interacted badly:
- The root layout sets html.h-full and body.min-h-full, so documentElement
and body heights track the iframe's viewport height.
- The parent template sets iframe.height = postedHeight + 24 every time
a height message arrives.
The combination produced an unbounded feedback loop: parent grows the
iframe by 24px, viewport grows, document height grows, ResizeObserver
fires, we post the new height, parent grows by another 24px. The outer
CiviCRM page scrollbar visibly shrank each cycle.
Fix on the report side (no extension change needed): when framed, override
html height to auto and body min-height to 0 so the document decouples
from the viewport. Observe body (the actual content), measure
body.scrollHeight, and skip posting when the value is unchanged. Original
styles are restored on unmount so route changes back to the standalone
view still work.
The Engagement Report tab page renders templates/CRM/WebformMw/Page/Tab.tpl
via Smarty. Without an explicit template-dir registration, Smarty cannot
locate the file and the tab fails to render. Use CRM_Core_Smarty's
prependTemplateDir() (cleaner than splicing template_dir by hand).
Mirrors the same fix the CiviCRM admin applied on the server so the repo
source no longer drifts from the working deployed state.
Adds a native window.confirm() inside onSubmit, after the uploads-in-flight
guard and before the submitting-state flip. Runs only after react-hook-form
validation passes, so users who hit Submit on an incomplete form still see
the existing field-level errors via onInvalid rather than a confusing
"are you sure?" prompt. Cancelling leaves the form state untouched.
The Engagement Report tab on Organization contact pages was loading the
contact-view summary recursively inside its own tab pane. Root cause:
Civi's menu router was not picking up the extension's xml/Menu file, so
`civicrm/contact/view/engagement-report` fell back to the parent
`civicrm/contact/view` route. Adding an explicit hook_civicrm_xmlMenu
implementation forces the menu file to register, after which the route
resolves to CRM_WebformMw_Page_Tab and the iframe renders as intended.
Deploy: replace the extension files on the Civi server, then in
Administer → System Settings → Extensions Disable + re-Enable webform-mw
(or run `cv flush` on the server) so the menu cache is rebuilt.
No fields on the survey should block submit — removes required:true
from Preliminary_Market_Assessment so the form is fully optional end
to end.
Surfaces Stage 0 sources-and-uses currency fields (Total_cost_of_project,
Member_equity_raised, Member_loans_raised, Member_preferred_shares_raised,
Bank_debt_raised, Grants_Donations_Raised, Other_sources_raised) starting
at Business Feasibility instead of Stabilize so they can be filled in
earlier in the lifecycle.
Cross-references the Civi-side cutover checklist from the deploy doc
so anyone deploying knows the CiviCRM prerequisites are tracked in a
single place.
App side:
- Per-route CSP: /staff/report now sets frame-ancestors 'self'
<CIVI_BASE_URL origin> and drops X-Frame-Options so the CiviCRM
extension can iframe it. All other routes keep frame-ancestors
'none' + X-Frame-Options: DENY via a path-negation source.
- Staff page recognises ?frame=1 and renders without SiteHeader/
SiteFooter so it fills the iframe cleanly.
- StaffReportView posts its scrollHeight to the parent window via
postMessage when framed; the Civi tab listens and auto-resizes
the iframe (no nested scrollbar). Anchor strip drops its sticky
positioning in frame mode since there's no internal scroll.
CiviCRM extension (civi-extension/webform-mw/, key webform-mw):
- info.xml + main hook file (webform_mw.php) implementing
hook_civicrm_tabset to add an 'Engagement Report' tab to
Organization contact-view pages.
- CRM/WebformMw/Page/Tab.php + Smarty template render an iframe
pointing at <WEBFORM_MW_APP_URL>/staff/report?org=<cid>&key=&frame=1,
with a postMessage listener that validates event.origin against
the configured app URL before resizing.
- Config via PHP constants in civicrm.settings.php (WEBFORM_MW_APP_URL,
WEBFORM_MW_STAFF_KEY) or matching env vars. Help banner shown when
unconfigured.
- README documents install, config, behaviour, security caveats.
UX iteration after first live look:
- Sticky anchor strip below the header with a chip per section (incl.
Submissions) so staff can jump around a long page.
- Compact one-line rows that show only the latest value; multi-history
fields get a muted 'N earlier entries' toggle that reveals the rest
inline. Same affordance for file fields.
- Empty fields collapse under a single 'N empty fields' toggle per
section instead of taking a row each.
- Stage 5: Y1_Q<n>_<metric> fields render as a read-only matrix table
(rows: metrics; columns: Q1..Q4) matching the form's matrix layout.
File proxy (/api/staff/file) deleted. APIv4 Attachment isn't exposed
on this Civi instance (per the June upload spike), which is why the
previous proxy returned broken images. Staff are already authenticated
to Civi when they arrive here, so file fields now render as outbound
links to CIVI_BASE_URL/civicrm/file?reset=1&id=<id> and the browser
uses the staff session. No more proxy auth, no more SSRF surface to
harden, no broken images.
CIVI_BASE_URL flows from the staff page (server component) into the
client as a prop. No secret material crosses the boundary.
Without this, even after setting STAFF_REPORT_KEY in the Amplify Secrets
tab the value never reaches the SSR Lambda — the build script only writes
the listed env vars into .env.production, which is what Next bundles.
- Allowlist inline MIME types (png/jpeg/gif/webp/pdf only); everything
else, including SVG and HTML, served as application/octet-stream
with content-disposition: attachment.
- X-Content-Type-Options: nosniff and a restrictive CSP on every response.
- Validate the upstream URL Civi returns: must match CIVI_BASE_URL origin
before we attach basic-auth creds and follow it. redirect: manual to
prevent off-host hops.
- Drop SVG from the client's inline-image list (server forces download).
Joel Brock