Upload: switch to APIv3 Attachment.create multipart (decode bytes properly)

APIv4 File.create on this Civi install stores the `content` field
verbatim — no base64 decoding. The hex dump of a downloaded file
confirms it: bytes start with 69 56 42 4f ("iVBO...") which is the
base64 encoding of the PNG header (89 50 4e 47), not the header itself.
Every file uploaded via the form has been corrupt on disk since launch.

JSON can't carry binary safely (high bytes break UTF-8), so the fix is
to stop trying. APIv3 Attachment.create accepts a multipart `file` part
the standard way (read from $_FILES on the server side) which preserves
bytes exactly.

Changes:
- lib/civicrm.ts: new civi3Upload() helper. POSTs multipart/form-data
  with `entity`, `action`, `json`, and `file` parts to /civicrm/ajax/rest
  using the same AuthX headers as civi3(). Wraps the Uint8Array into an
  ArrayBuffer slice so Blob's narrower BlobPart typing accepts it.
- app/api/upload/route.ts: replace the v4 File.create JSON call with
  civi3Upload("Attachment", "create", ...). Attachment.create requires
  an entity context, so anchor to the form-filler's contact id. Our
  custom-field flow uses the returned file id directly (no entity_file
  linkage needed for prefill/download), so the extra civicrm_entity_file
  row is metadata-only.

Note: existing files in Civi (uploaded via the buggy path) are still
corrupt on disk. New uploads will be intact. To recover the old ones
the user would need to re-upload via the form, or run a one-off
backfill that reads the base64 text out of /civicrm.files/upload/ and
rewrites each file with its decoded bytes.
This commit is contained in:
Joel Brock
2026-06-10 11:57:52 -07:00
parent e22c9226d8
commit f74fd07ba5
2 changed files with 127 additions and 29 deletions
+37 -29
View File
@@ -25,7 +25,7 @@
*/
import { NextResponse } from "next/server";
import { civi, verifyChecksum } from "@/lib/civicrm";
import { civi3Upload, verifyChecksum } from "@/lib/civicrm";
import { allFields } from "@/config/form";
import { rateLimit, clientIp } from "@/lib/rate-limit";
@@ -205,44 +205,52 @@ export async function POST(req: Request) {
);
}
// APIv4 File.create with inline base64 content. Spike (June 2026) on
// this Civi instance confirmed:
// - APIv4 Attachment is NOT exposed
// - APIv4 File + EntityFile ARE exposed
// - File.create with file_name + mime_type + content (base64) returns
// a usable file id
// - Custom file fields store the file id directly in the custom
// column, so EntityFile linkage is not needed for our use case
// - Round-trip via Contact.update + Contact.get .file_name join works
// APIv3 Attachment.create with a multipart `file` part. The earlier
// approach used APIv4 File.create with `content: base64String` and that
// Civi on this install stores the literal base64 *text* on disk — every
// file came back corrupted (hex 6956... = "iVBO..." = base64 PNG header).
// APIv3 Attachment.create reads $_FILES['file'] from the multipart body
// and writes the bytes through unchanged.
//
// We do not create EntityFile rows here. Civi's custom-field renderer
// joins through the custom column to civicrm_file directly, and the
// form-side prefill/read code in /api/data uses the same join.
// Attachment.create requires (entity_table, entity_id). Our custom-field
// flow stores the file id directly in the custom column (no
// civicrm_entity_file linkage needed for prefill/download), but the API
// still mandates an entity context for the upload call itself, so we
// anchor to the form-filler's contact id. The resulting civicrm_entity_file
// row is metadata-only — Civi's custom-field joins ignore it.
//
// The returned id is what the frontend stores in RHF state and
// ultimately sends as the field value on /api/submit. /api/submit then
// writes that id to the activity custom field (for stage-N file fields)
// or to the org contact custom field (for Food_Co_op_Organizing.*).
// The returned id is what the frontend stores in RHF state and ultimately
// sends as the field value on /api/submit. /api/submit then writes that
// id to the activity custom field (stage-N) or to the org contact custom
// field (Food_Co_op_Organizing.*).
//
// Orphan files: if the user uploads and then abandons the form, the
// File row persists with no entity referencing it. Cleanup is handled
// by a CiviCRM scheduled job (configured separately by the Civi admin)
// that deletes File rows with no inbound references older than ~24h.
// civicrm_file + civicrm_entity_file rows persist with no business
// reference. Cleanup is handled by a CiviCRM scheduled job (configured
// separately by the Civi admin) that prunes orphans older than ~24h.
let fileId: number;
try {
const res = await civi<{ id: number }>("File", "create", {
values: {
file_name: safeName,
const res = await civi3Upload<{ id: string | number }>(
"Attachment",
"create",
{
entity_table: "civicrm_contact",
entity_id: Number(cid),
name: safeName,
mime_type: clientMime,
content: Buffer.from(bytes).toString("base64"),
sequential: 1,
},
});
const id = res.values?.[0]?.id;
if (!id) throw new Error("File.create returned no id");
fileId = Number(id);
{ bytes, filename: safeName, mime: clientMime },
);
const idRaw = res.values?.[0]?.id;
const id = typeof idRaw === "string" ? Number(idRaw) : idRaw;
if (!id || !Number.isFinite(id)) {
throw new Error("Attachment.create returned no id");
}
fileId = id;
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
console.error("[upload] File.create failed:", msg);
console.error("[upload] Attachment.create failed:", msg);
return NextResponse.json(
{ error: "Could not save the upload. Please try again." },
{ status: 502 },