Files
WebForm-mw/scripts/spike-file-upload.mjs
Joel Brock 2400931a04 File upload pipeline: wire end-to-end via APIv4 File.create
Closes the file-upload gap. Files now actually land in CiviCRM (verified
empirically against the live Civi instance via spike scripts).

Spike findings (see scripts/spike-file-upload.mjs):
  - APIv4 Attachment is NOT exposed on this Civi
  - APIv4 File + EntityFile ARE exposed; File.create accepts inline
    base64 `content` and returns a usable file id
  - Custom file fields store the file id directly in the custom column,
    so EntityFile linkage is unnecessary for this use case
  - Round-trip via Contact.update + Contact.get .file_name join verified
    on a real org contact

Pipeline:

  Renderer (FileField) picks up onChange  →
    POST /api/upload (multipart) with file + cid + cs + fieldRef  →
      verifyChecksum, MIME allowlist + magic-byte sniff, 5 MB cap  →
        civi.File.create({ file_name, mime_type, content: base64 })  →
          returns { id, file_name }  →
            renderer stores in RHF state via setValue
  Form submit  →
    POST /api/submit (JSON) with the {id, file_name} value  →
      submit detects the file shape and writes the id as the value of
      the activity/contact custom field

File changes:

  app/api/upload/route.ts
    Replaced the 501 stub with the real File.create call. Comment
    documents that EntityFile linkage is intentionally skipped and that
    orphan cleanup is owned by a CiviCRM scheduled job.

  app/api/submit/route.ts
    For type:"file" values shaped as {id, file_name}, write the id as
    the custom field value (activity or contact, depending on the
    civiField / civiContactField the field declares).

  components/fields/FieldRenderer.tsx
    Replaced the bare <input type=file> register() with FileField, an
    upload-on-pick subcomponent. The native input is NOT register()'d:
    its FileList value was the original bug. FileField owns its
    uploading + error state and writes {id, file_name} via setValue on
    success. Submit is blocked upstream while uploads are in flight.

  components/StageSection.tsx, components/EngagementForm.tsx
    Thread setValue, cid, cs, and an onUploadStateChange callback
    through to FieldRenderer. EngagementForm tracks uploads-in-flight
    count; onSubmit refuses to submit while the count is > 0.

  config/form.ts
    Promotes Certificate_of_Incorporation from readonly to a real
    file field now that the pipeline works.

  app/api/data/route.ts
    Drops the readonly carveout that was only needed while the
    certificate was readonly.

  scripts/list-civi-entities.mjs (new)
    APIv4 entity probe + APIv3 attachment-API probe. Used to determine
    that File (not Attachment) was the right entity on this Civi.

  scripts/spike-file-upload.mjs (new)
    The actual end-to-end test that proved out the pipeline before
    wiring. Safe to re-run on any Civi instance during future audits.

Not in this change:
  - Orphan attachment cleanup (CiviCRM scheduled job, Civi admin scope)
  - Per-field MIME allowlists (single global list for v1)
  - S3 / presigned-URL path for >5 MB files (deferred; capped at 5 MB
    today to stay under Amplify Lambda's 6 MB sync payload limit)
2026-06-05 07:48:57 -07:00

185 lines
6.4 KiB
JavaScript

#!/usr/bin/env node
// scripts/spike-file-upload.mjs
//
// Supersedes spike-attachment-upload.mjs. The earlier spike showed APIv4
// Attachment.create is not exposed on this Civi instance, but APIv4
// `File` and `EntityFile` ARE present, and APIv3 Attachment is reachable
// as a fallback.
//
// CiviCRM custom file fields store the file id directly in the custom
// column on the entity's custom-value table -- the EntityFile linkage
// table is only needed for general attachments (e.g. on an Activity's
// "Attachments" tab). So for our form's custom-field-bound files we
// only need:
//
// File row in civicrm_file <-- File.create
// │
// │ (file id stored directly as the custom field value)
// ▼
// Custom field on the entity <-- Contact.update / Activity.create
//
// This spike confirms that pipeline end-to-end on a real org contact.
//
// USAGE
// node --env-file=.env.local scripts/spike-file-upload.mjs --org-id=<n> [--keep]
import { Buffer } from "node:buffer";
const args = process.argv.slice(2);
const orgIdArg = args.find((a) => a.startsWith("--org-id="));
const KEEP = args.includes("--keep");
const ORG_ID = orgIdArg ? Number(orgIdArg.slice("--org-id=".length)) : null;
if (!ORG_ID) {
console.error(
"Usage: node --env-file=.env.local scripts/spike-file-upload.mjs --org-id=<n> [--keep]",
);
process.exit(1);
}
const CUSTOM_FIELD = "Food_Co_op_Organizing.Certificate_of_Incorporation";
const TEST_FILENAME = `spike-${Date.now()}.txt`;
const TEST_MIME = "text/plain";
const TEST_BODY = "civi-webform file spike — safe to delete";
async function civi(entity, action, params) {
const {
CIVI_BASE_URL,
CIVI_API_KEY,
CIVI_SITE_KEY,
CIVI_HTTP_AUTH_USER,
CIVI_HTTP_AUTH_PASS,
} = process.env;
if (!CIVI_BASE_URL || !CIVI_API_KEY || !CIVI_SITE_KEY) {
throw new Error("Missing CIVI_BASE_URL / CIVI_API_KEY / CIVI_SITE_KEY.");
}
const url = `${CIVI_BASE_URL}/civicrm/ajax/api4/${entity}/${action}`;
const headers = {
"Content-Type": "application/x-www-form-urlencoded",
"X-Civi-Auth": `Bearer ${CIVI_API_KEY}`,
"X-Civi-Key": CIVI_SITE_KEY,
};
if (CIVI_HTTP_AUTH_USER && CIVI_HTTP_AUTH_PASS) {
headers["Authorization"] =
"Basic " +
Buffer.from(`${CIVI_HTTP_AUTH_USER}:${CIVI_HTTP_AUTH_PASS}`).toString("base64");
}
const res = await fetch(url, {
method: "POST",
headers,
body: new URLSearchParams({ params: JSON.stringify(params) }),
});
const text = await res.text();
let json;
try { json = JSON.parse(text); } catch {
throw new Error(`${entity}.${action} non-JSON (HTTP ${res.status}): ${text}`);
}
if (!res.ok || json.error_message) {
throw new Error(`${entity}.${action} HTTP ${res.status}: ${json.error_message ?? text}`);
}
return json;
}
function divider(label) {
console.log(`\n── ${label} ${"─".repeat(Math.max(0, 60 - label.length))}`);
}
// ── Q1: what fields does APIv4 File.create accept on this Civi?
divider("Q1: File.getFields");
const fields = await civi("File", "getFields", { action: "create" });
const fieldNames = (fields.values ?? []).map((f) => f.name);
console.log("File create-action fields:", fieldNames.join(", "));
const acceptsContent = fieldNames.includes("content");
console.log(`Accepts 'content' param: ${acceptsContent ? "yes ✓" : "NO -- must POST file differently"}`);
if (!acceptsContent) {
console.log(
"\nFile.create on this Civi doesn't accept inline content. The upload path",
"needs to use a different mechanism (likely the legacy APIv3 Attachment.create",
"or the civicrm/upload endpoint). Stopping spike to avoid guessing.",
);
process.exit(2);
}
// ── Q2: create a File row.
divider("Q2: File.create with base64 content");
const created = await civi("File", "create", {
values: {
file_name: TEST_FILENAME,
mime_type: TEST_MIME,
content: Buffer.from(TEST_BODY, "utf8").toString("base64"),
},
});
const fileId = created.values?.[0]?.id;
console.log("File.create result:", JSON.stringify(created, null, 2));
if (!fileId) {
console.log("RESULT: failed — no id returned");
process.exit(2);
}
console.log(`RESULT: file id = ${fileId} ✓`);
// ── Q3: read current value, write file id to custom field, read back.
divider(`Q3: Contact.update ${CUSTOM_FIELD} = ${fileId}`);
const before = await civi("Contact", "get", {
select: ["id", CUSTOM_FIELD, `${CUSTOM_FIELD}.file_name`],
where: [["id", "=", ORG_ID]],
});
const priorRow = before.values?.[0] ?? {};
const priorValue = priorRow[CUSTOM_FIELD] ?? null;
console.log("Prior value on contact:", JSON.stringify(priorRow, null, 2));
let writeOk = false;
try {
const upd = await civi("Contact", "update", {
where: [["id", "=", ORG_ID]],
values: { [CUSTOM_FIELD]: fileId },
});
console.log("Update result:", JSON.stringify(upd, null, 2));
writeOk = true;
console.log("RESULT: write OK ✓");
} catch (err) {
console.log(`Update failed: ${err.message}`);
}
// ── Q4: read back through the join /api/data uses.
if (writeOk) {
divider("Q4: read-back via Contact.get + .file_name join");
const after = await civi("Contact", "get", {
select: ["id", "display_name", CUSTOM_FIELD, `${CUSTOM_FIELD}.file_name`],
where: [["id", "=", ORG_ID]],
});
console.log(JSON.stringify(after, null, 2));
const row = after.values?.[0];
if (row && Number(row[CUSTOM_FIELD]) === Number(fileId) && row[`${CUSTOM_FIELD}.file_name`]) {
console.log("RESULT: round-trip OK ✓ — pipeline is viable");
} else {
console.log("RESULT: read-back incomplete or id mismatch — see payload");
}
}
// ── Cleanup: restore prior value, delete file row.
if (KEEP) {
console.log(`\n--keep set; leaving file id=${fileId} and contact pointing at it.`);
} else {
divider("Cleanup: restore prior contact value + File.delete");
try {
await civi("Contact", "update", {
where: [["id", "=", ORG_ID]],
values: { [CUSTOM_FIELD]: priorValue },
});
console.log(`Restored ${CUSTOM_FIELD} = ${priorValue}`);
} catch (err) {
console.log("Restore failed:", err.message);
}
try {
const del = await civi("File", "delete", { where: [["id", "=", fileId]] });
console.log(`File.delete id=${fileId}:`, JSON.stringify(del));
console.log("Cleanup OK ✓");
} catch (err) {
console.log("File.delete failed:", err.message);
console.log(`*** MANUAL CLEANUP NEEDED: File id=${fileId} is orphaned in CiviCRM ***`);
}
}
console.log("\nSpike complete.");