Upload: switch to APIv3 Attachment.create multipart (decode bytes properly)

APIv4 File.create on this Civi install stores the `content` field
verbatim — no base64 decoding. The hex dump of a downloaded file
confirms it: bytes start with 69 56 42 4f ("iVBO...") which is the
base64 encoding of the PNG header (89 50 4e 47), not the header itself.
Every file uploaded via the form has been corrupt on disk since launch.

JSON can't carry binary safely (high bytes break UTF-8), so the fix is
to stop trying. APIv3 Attachment.create accepts a multipart `file` part
the standard way (read from $_FILES on the server side) which preserves
bytes exactly.

Changes:
- lib/civicrm.ts: new civi3Upload() helper. POSTs multipart/form-data
  with `entity`, `action`, `json`, and `file` parts to /civicrm/ajax/rest
  using the same AuthX headers as civi3(). Wraps the Uint8Array into an
  ArrayBuffer slice so Blob's narrower BlobPart typing accepts it.
- app/api/upload/route.ts: replace the v4 File.create JSON call with
  civi3Upload("Attachment", "create", ...). Attachment.create requires
  an entity context, so anchor to the form-filler's contact id. Our
  custom-field flow uses the returned file id directly (no entity_file
  linkage needed for prefill/download), so the extra civicrm_entity_file
  row is metadata-only.

Note: existing files in Civi (uploaded via the buggy path) are still
corrupt on disk. New uploads will be intact. To recover the old ones
the user would need to re-upload via the form, or run a one-off
backfill that reads the base64 text out of /civicrm.files/upload/ and
rewrites each file with its decoded bytes.

app/api/upload/route.ts

@@ -25,7 +25,7 @@
  */
 
 import { NextResponse } from "next/server";
-import { civi, verifyChecksum } from "@/lib/civicrm";
+import { civi3Upload, verifyChecksum } from "@/lib/civicrm";
 import { allFields } from "@/config/form";
 import { rateLimit, clientIp } from "@/lib/rate-limit";
 
@@ -205,44 +205,52 @@ export async function POST(req: Request) {
     );
   }
 
-  // APIv4 File.create with inline base64 content. Spike (June 2026) on
+  // APIv3 Attachment.create with a multipart `file` part. The earlier
-  // this Civi instance confirmed:
+  // approach used APIv4 File.create with `content: base64String` and that
-  //   - APIv4 Attachment is NOT exposed
+  // Civi on this install stores the literal base64 *text* on disk — every
-  //   - APIv4 File + EntityFile ARE exposed
+  // file came back corrupted (hex 6956... = "iVBO..." = base64 PNG header).
-  //   - File.create with file_name + mime_type + content (base64) returns
+  // APIv3 Attachment.create reads $_FILES['file'] from the multipart body
-  //     a usable file id
+  // and writes the bytes through unchanged.
-  //   - Custom file fields store the file id directly in the custom
-  //     column, so EntityFile linkage is not needed for our use case
-  //   - Round-trip via Contact.update + Contact.get .file_name join works
   //
-  // We do not create EntityFile rows here. Civi's custom-field renderer
+  // Attachment.create requires (entity_table, entity_id). Our custom-field
-  // joins through the custom column to civicrm_file directly, and the
+  // flow stores the file id directly in the custom column (no
-  // form-side prefill/read code in /api/data uses the same join.
+  // civicrm_entity_file linkage needed for prefill/download), but the API
+  // still mandates an entity context for the upload call itself, so we
+  // anchor to the form-filler's contact id. The resulting civicrm_entity_file
+  // row is metadata-only — Civi's custom-field joins ignore it.
   //
-  // The returned id is what the frontend stores in RHF state and
+  // The returned id is what the frontend stores in RHF state and ultimately
-  // ultimately sends as the field value on /api/submit. /api/submit then
+  // sends as the field value on /api/submit. /api/submit then writes that
-  // writes that id to the activity custom field (for stage-N file fields)
+  // id to the activity custom field (stage-N) or to the org contact custom
-  // or to the org contact custom field (for Food_Co_op_Organizing.*).
+  // field (Food_Co_op_Organizing.*).
   //
   // Orphan files: if the user uploads and then abandons the form, the
-  // File row persists with no entity referencing it. Cleanup is handled
+  // civicrm_file + civicrm_entity_file rows persist with no business
-  // by a CiviCRM scheduled job (configured separately by the Civi admin)
+  // reference. Cleanup is handled by a CiviCRM scheduled job (configured
-  // that deletes File rows with no inbound references older than ~24h.
+  // separately by the Civi admin) that prunes orphans older than ~24h.
   let fileId: number;
   try {
-    const res = await civi<{ id: number }>("File", "create", {
+    const res = await civi3Upload<{ id: string | number }>(
-      values: {
+      "Attachment",
-        file_name: safeName,
+      "create",
+      {
+        entity_table: "civicrm_contact",
+        entity_id: Number(cid),
+        name: safeName,
         mime_type: clientMime,
-        content: Buffer.from(bytes).toString("base64"),
+        sequential: 1,
       },
-    });
+      { bytes, filename: safeName, mime: clientMime },
-    const id = res.values?.[0]?.id;
+    );
-    if (!id) throw new Error("File.create returned no id");
+    const idRaw = res.values?.[0]?.id;
-    fileId = Number(id);
+    const id = typeof idRaw === "string" ? Number(idRaw) : idRaw;
+    if (!id || !Number.isFinite(id)) {
+      throw new Error("Attachment.create returned no id");
+    }
+    fileId = id;
   } catch (err) {
     const msg = err instanceof Error ? err.message : String(err);
-    console.error("[upload] File.create failed:", msg);
+    console.error("[upload] Attachment.create failed:", msg);
     return NextResponse.json(
       { error: "Could not save the upload. Please try again." },
       { status: 502 },

lib/civicrm.ts

@@ -154,6 +154,96 @@ export async function civi3<T = unknown>(
   return { values, count: json.count };
 }
 
+/**
+ * Multipart APIv3 call. Used for binary uploads — APIv4 File.create stores
+ * the `content` field verbatim (no base64 decoding), so files come back
+ * corrupted. APIv3 Attachment.create accepts the file via the standard
+ * multipart `file` part (read from $_FILES on the server) which preserves
+ * the bytes exactly.
+ *
+ * Caller provides `params` (non-binary metadata) and `file` (the binary +
+ * filename + mime). Authentication is the same AuthX headers as civi3().
+ */
+export async function civi3Upload<T = unknown>(
+  entity: string,
+  action: string,
+  params: Record<string, unknown>,
+  file: { bytes: Uint8Array; filename: string; mime: string },
+  opts: CiviApiOptions = {},
+): Promise<CiviApiResponse<T>> {
+  if (isStubMode()) {
+    console.warn(
+      `${STUB_LOG_PREFIX} v3-multipart ${entity}.${action} — env not set, returning empty values`,
+    );
+    return { values: [] };
+  }
+
+  const base = opts.baseUrl ?? process.env.CIVI_BASE_URL!;
+  const url = `${base}/civicrm/ajax/rest`;
+
+  const form = new FormData();
+  form.append("entity", entity);
+  form.append("action", action);
+  form.append("json", JSON.stringify(params));
+  // Wrap the Uint8Array in a fresh ArrayBuffer slice so Blob's typing
+  // (which only accepts ArrayBuffer, not the wider ArrayBufferLike) is
+  // happy. The slice is a no-op on real Uint8Array inputs.
+  const fileBuf = file.bytes.buffer.slice(
+    file.bytes.byteOffset,
+    file.bytes.byteOffset + file.bytes.byteLength,
+  ) as ArrayBuffer;
+  form.append(
+    "file",
+    new Blob([fileBuf], { type: file.mime }),
+    file.filename,
+  );
+
+  // Do NOT set Content-Type — fetch sets multipart/form-data with the
+  // correct boundary automatically when body is a FormData.
+  const headers: Record<string, string> = {
+    "X-Civi-Auth": `Bearer ${process.env.CIVI_API_KEY}`,
+    "X-Civi-Key": process.env.CIVI_SITE_KEY!,
+    "X-Requested-With": "XMLHttpRequest",
+  };
+  if (process.env.CIVI_HTTP_AUTH_USER && process.env.CIVI_HTTP_AUTH_PASS) {
+    const creds = Buffer.from(
+      `${process.env.CIVI_HTTP_AUTH_USER}:${process.env.CIVI_HTTP_AUTH_PASS}`,
+    ).toString("base64");
+    headers["Authorization"] = `Basic ${creds}`;
+  }
+
+  const res = await fetch(url, {
+    method: "POST",
+    headers,
+    body: form,
+    cache: "no-store",
+  });
+  if (!res.ok) {
+    const text = await res.text();
+    throw new Error(
+      `CiviCRM v3 ${entity}.${action} (multipart) failed (${res.status}): ${text}`,
+    );
+  }
+  const json = (await res.json()) as {
+    is_error?: number;
+    error_message?: string;
+    values?: T[] | Record<string, T>;
+    count?: number;
+  };
+  if (json.is_error) {
+    throw new Error(
+      `CiviCRM v3 ${entity}.${action} (multipart) error: ${json.error_message ?? "unknown"}`,
+    );
+  }
+  const raw = json.values;
+  const values: T[] = Array.isArray(raw)
+    ? raw
+    : raw && typeof raw === "object"
+    ? Object.values(raw)
+    : [];
+  return { values, count: json.count };
+}
+
 /**
  * Validate a contact checksum (cid + cs) against CiviCRM.
  *