Upload: route through Civi extension multipart endpoint
Every JSON-based upload path on this Civi stores the `content` field
verbatim on disk — confirmed against both APIv4 File.create AND APIv3
Attachment.create (both came back as base64 text in hex dumps). The
multipart `file` part to /civicrm/ajax/rest is also a dead end: APIv3
Attachment.create on this install doesn't see $_FILES (rejected with
"Mandatory key(s) missing: id or content or options.move-file").
The one path Civi honors is APIv3 Attachment.create + options.move-file
— pointing at a filesystem path the Civi server can read. So expose a
tiny multipart endpoint in the WebForm-mw Civi extension that copies
PHP's $_FILES['file']['tmp_name'] into the API call, then return the
new file id as JSON. PHP's $_FILES preserves binary natively.
Civi extension (requires admin deploy):
- CRM/WebformMw/Page/Upload.php : multipart POST handler. Validates
the upload, requires `access CiviCRM`, whitelists entity_table to
civicrm_contact|civicrm_activity, calls Attachment.create with
move-file pointing at the tmp upload, returns {id, name} JSON.
- xml/Menu/webform_mw.xml : registers civicrm/webform-mw/upload.
WebForm-mw side:
- lib/civicrm.ts : new civiMultipart() helper. POSTs multipart to an
arbitrary Civi path (not /civicrm/ajax/rest) with the same AuthX
headers. Returns the parsed JSON body.
- app/api/upload/route.ts : send the upload's bytes via civiMultipart
to civicrm/webform-mw/upload. Comment-block now records all four
upload paths we tried so a future reader doesn't repeat the cycle.
Deploy: admin syncs the updated civi-extension/webform-mw/ directory
and Disable/Re-enables the extension (or runs cv flush) so the new
menu route is registered.
This commit is contained in:
Joel Brock
+30
-34
@@ -25,7 +25,7 @@
|
||||
*/
|
||||
|
||||
import { NextResponse } from "next/server";
|
||||
import { civi3, verifyChecksum } from "@/lib/civicrm";
|
||||
import { civiMultipart, verifyChecksum } from "@/lib/civicrm";
|
||||
import { allFields } from "@/config/form";
|
||||
import { rateLimit, clientIp } from "@/lib/rate-limit";
|
||||
|
||||
@@ -205,48 +205,44 @@ export async function POST(req: Request) {
|
||||
);
|
||||
}
|
||||
|
||||
// APIv3 Attachment.create with `content` as base64. Path history:
|
||||
// Path history exhausted before we got here:
|
||||
// 1. APIv4 File.create + content:base64 → stores base64 text on disk.
|
||||
// 2. APIv3 Attachment.create + content:base64 → same; v3 doesn't decode either.
|
||||
// 3. APIv3 Attachment.create + multipart file → /civicrm/ajax/rest drops $_FILES.
|
||||
//
|
||||
// 1. APIv4 File.create + content:base64 → stored base64 *text* on disk
|
||||
// (Civi v4 doesn't decode). Every file came back corrupt.
|
||||
// 2. APIv3 Attachment.create + multipart file part → Civi rejected with
|
||||
// "Mandatory key(s) missing: id or content or options.move-file".
|
||||
// The /civicrm/ajax/rest endpoint on this install doesn't expose
|
||||
// $_FILES to the v3 action; only params['content'] is consulted.
|
||||
// 3. APIv3 Attachment.create + content:base64 (this code) → Civi's v3
|
||||
// Attachment.create has historically been the file-upload entry
|
||||
// point used by Civi's own form widgets, and it auto-decodes the
|
||||
// content field.
|
||||
// The one path Civi reliably honors is options.move-file: APIv3
|
||||
// Attachment.create with a server-side filesystem path. PHP's $_FILES
|
||||
// preserves binary natively, so the WebForm-mw Civi extension exposes
|
||||
// a tiny multipart endpoint that copies the upload's tmp_name into
|
||||
// Attachment.create as options.move-file. We POST the file there.
|
||||
//
|
||||
// Attachment.create requires (entity_table, entity_id); our custom-field
|
||||
// flow stores the file id directly in the custom column (no entity_file
|
||||
// linkage needed for prefill/download), but the API still mandates an
|
||||
// entity context, so we anchor to the form-filler's contact id. The
|
||||
// resulting civicrm_entity_file row is metadata-only — Civi's custom-
|
||||
// field joins ignore it.
|
||||
// flow uses the returned file id directly in the custom column (no
|
||||
// entity_file linkage needed), so we anchor to the form-filler's contact
|
||||
// id and accept the metadata-only civicrm_entity_file row.
|
||||
//
|
||||
// Orphan files: civicrm_file + civicrm_entity_file rows persist if the
|
||||
// user uploads and then abandons the form. Cleanup is handled by a
|
||||
// CiviCRM scheduled job configured separately by the Civi admin.
|
||||
// Orphan files: civicrm_file rows linger if the user uploads then
|
||||
// abandons. Cleanup is handled by a CiviCRM scheduled job (separately
|
||||
// configured by the Civi admin).
|
||||
let fileId: number;
|
||||
try {
|
||||
const res = await civi3<{ id: string | number }>("Attachment", "create", {
|
||||
entity_table: "civicrm_contact",
|
||||
entity_id: Number(cid),
|
||||
name: safeName,
|
||||
mime_type: clientMime,
|
||||
content: Buffer.from(bytes).toString("base64"),
|
||||
sequential: 1,
|
||||
});
|
||||
const idRaw = res.values?.[0]?.id;
|
||||
const id = typeof idRaw === "string" ? Number(idRaw) : idRaw;
|
||||
if (!id || !Number.isFinite(id)) {
|
||||
throw new Error("Attachment.create returned no id");
|
||||
const res = await civiMultipart<{ id?: number; name?: string; error?: string }>(
|
||||
"civicrm/webform-mw/upload",
|
||||
{
|
||||
entity_table: "civicrm_contact",
|
||||
entity_id: String(Number(cid)),
|
||||
name: safeName,
|
||||
mime_type: clientMime,
|
||||
},
|
||||
{ bytes, filename: safeName, mime: clientMime },
|
||||
);
|
||||
if (!res || !res.id || !Number.isFinite(res.id)) {
|
||||
throw new Error(res?.error ?? "Upload endpoint returned no id");
|
||||
}
|
||||
fileId = id;
|
||||
fileId = Number(res.id);
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
console.error("[upload] Attachment.create failed:", msg);
|
||||
console.error("[upload] Civi extension upload failed:", msg);
|
||||
return NextResponse.json(
|
||||
{ error: "Could not save the upload. Please try again." },
|
||||
{ status: 502 },
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
<?php
|
||||
|
||||
/**
|
||||
* File upload proxy.
|
||||
*
|
||||
* External callers (the WebForm-mw /api/upload route on Amplify) POST a
|
||||
* multipart request here with a `file` part. We hand the uploaded temp
|
||||
* path to APIv3 Attachment.create via `options.move-file`, which is the
|
||||
* one upload pathway Civi reliably honors on this install:
|
||||
*
|
||||
* - APIv4 File.create + content:base64 -> stores base64 text on disk.
|
||||
* - APIv3 Attachment.create + content:b64 -> same.
|
||||
* - APIv3 Attachment.create + multipart -> /civicrm/ajax/rest doesn't
|
||||
* expose $_FILES to the
|
||||
* action, so "file" is
|
||||
* silently ignored.
|
||||
* - APIv3 Attachment.create + options.move-file -> WORKS. Civi reads
|
||||
* the path, moves the file
|
||||
* into civicrm.files/upload,
|
||||
* writes correct bytes.
|
||||
*
|
||||
* Required POST fields:
|
||||
* file the binary (multipart `file` part)
|
||||
* entity_table e.g. "civicrm_contact" (per Attachment.create contract)
|
||||
* entity_id the entity id to link to
|
||||
* name (optional) file_name; defaults to the upload's name
|
||||
* mime_type (optional) defaults to the upload's reported type
|
||||
*
|
||||
* Returns JSON: { id, name } on success, { error } with 4xx/5xx otherwise.
|
||||
*
|
||||
* Authorization: `access CiviCRM`. AuthX is expected to authenticate the
|
||||
* Bearer + Site-Key headers WebForm-mw sends.
|
||||
*/
|
||||
class CRM_WebformMw_Page_Upload extends CRM_Core_Page {
|
||||
|
||||
public function run() {
|
||||
if (($_SERVER['REQUEST_METHOD'] ?? 'GET') !== 'POST') {
|
||||
$this->jsonError('POST required', 405);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!CRM_Core_Permission::check('access CiviCRM')) {
|
||||
$this->jsonError('Permission denied', 403);
|
||||
return;
|
||||
}
|
||||
|
||||
if (empty($_FILES['file']) || !is_array($_FILES['file'])) {
|
||||
$this->jsonError('Missing file part', 400);
|
||||
return;
|
||||
}
|
||||
$upload = $_FILES['file'];
|
||||
if ((int) ($upload['error'] ?? UPLOAD_ERR_NO_FILE) !== UPLOAD_ERR_OK) {
|
||||
$this->jsonError('Upload failed (php error ' . (int) $upload['error'] . ')', 400);
|
||||
return;
|
||||
}
|
||||
if (empty($upload['tmp_name']) || !is_uploaded_file($upload['tmp_name'])) {
|
||||
$this->jsonError('Invalid upload tmp path', 400);
|
||||
return;
|
||||
}
|
||||
|
||||
$entityTable = (string) ($_POST['entity_table'] ?? '');
|
||||
$entityId = (int) ($_POST['entity_id'] ?? 0);
|
||||
// Whitelist entity tables to mirror the redirect route's defensive scope.
|
||||
if (!in_array($entityTable, ['civicrm_contact', 'civicrm_activity'], TRUE)) {
|
||||
$this->jsonError('Invalid entity_table', 400);
|
||||
return;
|
||||
}
|
||||
if ($entityId <= 0) {
|
||||
$this->jsonError('Invalid entity_id', 400);
|
||||
return;
|
||||
}
|
||||
|
||||
$name = (string) ($_POST['name'] ?? $upload['name'] ?? 'upload');
|
||||
$mime = (string) ($_POST['mime_type'] ?? $upload['type'] ?? 'application/octet-stream');
|
||||
|
||||
try {
|
||||
$result = civicrm_api3('Attachment', 'create', [
|
||||
'entity_table' => $entityTable,
|
||||
'entity_id' => $entityId,
|
||||
'name' => $name,
|
||||
'mime_type' => $mime,
|
||||
'options' => [
|
||||
'move-file' => $upload['tmp_name'],
|
||||
],
|
||||
]);
|
||||
$fileId = NULL;
|
||||
if (!empty($result['id'])) {
|
||||
$fileId = (int) $result['id'];
|
||||
}
|
||||
elseif (!empty($result['values']) && is_array($result['values'])) {
|
||||
$first = reset($result['values']);
|
||||
if (!empty($first['id'])) {
|
||||
$fileId = (int) $first['id'];
|
||||
}
|
||||
}
|
||||
if (!$fileId) {
|
||||
throw new Exception('Attachment.create returned no id');
|
||||
}
|
||||
$this->jsonOk(['id' => $fileId, 'name' => $name]);
|
||||
}
|
||||
catch (Throwable $e) {
|
||||
$this->jsonError('Attachment.create failed: ' . $e->getMessage(), 500);
|
||||
}
|
||||
}
|
||||
|
||||
private function jsonOk(array $payload): void {
|
||||
header('Content-Type: application/json', TRUE, 200);
|
||||
echo json_encode($payload);
|
||||
CRM_Utils_System::civiExit();
|
||||
}
|
||||
|
||||
private function jsonError(string $message, int $status): void {
|
||||
header('Content-Type: application/json', TRUE, $status);
|
||||
echo json_encode(['error' => $message]);
|
||||
CRM_Utils_System::civiExit();
|
||||
}
|
||||
|
||||
}
|
||||
@@ -12,4 +12,10 @@
|
||||
<page_callback>CRM_WebformMw_Page_File</page_callback>
|
||||
<access_arguments>access CiviCRM</access_arguments>
|
||||
</item>
|
||||
<item>
|
||||
<path>civicrm/webform-mw/upload</path>
|
||||
<title>WebForm-mw file upload proxy</title>
|
||||
<page_callback>CRM_WebformMw_Page_Upload</page_callback>
|
||||
<access_arguments>access CiviCRM</access_arguments>
|
||||
</item>
|
||||
</menu>
|
||||
|
||||
@@ -244,6 +244,58 @@ export async function civi3Upload<T = unknown>(
|
||||
return { values, count: json.count };
|
||||
}
|
||||
|
||||
/**
|
||||
* POST a multipart request directly to an arbitrary Civi route. Used for
|
||||
* extension endpoints that handle multipart uploads natively (the v3/v4
|
||||
* ajax/rest path silently drops $_FILES on this install, and JSON+base64
|
||||
* stores the literal base64 text on disk).
|
||||
*
|
||||
* The caller's `path` is appended to CIVI_BASE_URL. AuthX headers are sent
|
||||
* the same way as the other helpers. Returns the parsed JSON body.
|
||||
*/
|
||||
export async function civiMultipart<T = unknown>(
|
||||
path: string,
|
||||
fields: Record<string, string>,
|
||||
file: { bytes: Uint8Array; filename: string; mime: string },
|
||||
opts: CiviApiOptions = {},
|
||||
): Promise<T> {
|
||||
if (isStubMode()) {
|
||||
console.warn(`${STUB_LOG_PREFIX} multipart ${path} — env not set`);
|
||||
return {} as T;
|
||||
}
|
||||
const base = opts.baseUrl ?? process.env.CIVI_BASE_URL!;
|
||||
const url = `${base.replace(/\/+$/, "")}/${path.replace(/^\/+/, "")}`;
|
||||
const form = new FormData();
|
||||
for (const [k, v] of Object.entries(fields)) form.append(k, v);
|
||||
const fileBuf = file.bytes.buffer.slice(
|
||||
file.bytes.byteOffset,
|
||||
file.bytes.byteOffset + file.bytes.byteLength,
|
||||
) as ArrayBuffer;
|
||||
form.append("file", new Blob([fileBuf], { type: file.mime }), file.filename);
|
||||
const headers: Record<string, string> = {
|
||||
"X-Civi-Auth": `Bearer ${process.env.CIVI_API_KEY}`,
|
||||
"X-Civi-Key": process.env.CIVI_SITE_KEY!,
|
||||
"X-Requested-With": "XMLHttpRequest",
|
||||
};
|
||||
if (process.env.CIVI_HTTP_AUTH_USER && process.env.CIVI_HTTP_AUTH_PASS) {
|
||||
const creds = Buffer.from(
|
||||
`${process.env.CIVI_HTTP_AUTH_USER}:${process.env.CIVI_HTTP_AUTH_PASS}`,
|
||||
).toString("base64");
|
||||
headers["Authorization"] = `Basic ${creds}`;
|
||||
}
|
||||
const res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers,
|
||||
body: form,
|
||||
cache: "no-store",
|
||||
});
|
||||
if (!res.ok) {
|
||||
const text = await res.text();
|
||||
throw new Error(`Civi POST ${path} failed (${res.status}): ${text}`);
|
||||
}
|
||||
return (await res.json()) as T;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a contact checksum (cid + cs) against CiviCRM.
|
||||
*
|
||||
|
||||
Reference in New Issue
Block a user