pdfjs-dist standard_fonts Missing From standalone

The short version

pdfjs-dist’s standard_fonts/ is read in a way Next.js file tracing cannot follow, so it can go missing from an output: "standalone" build.

The effect of the omission is quiet. Missing fonts do not raise an exception; only text extraction from PDFs using the standard fonts (Helvetica and the rest) breaks. Making the inclusion explicit in configuration (outputFileTracingIncludes) and adding a separate fail-fast guard that notices at production startup if it goes missing anyway is what finally made this safe to operate.

What it looks like

This repository’s PDF extractor (PdfExtractor) calls getTextContent() per page with pdfjs-dist (the Node legacy build) to pull out text. Extracting PDFs that use the standard 14 fonts (Helvetica and so on) correctly requires passing the standard_fonts/ directory pdfjs-dist ships as standardFontDataUrl.

That directory is found without trouble in development and CI, where pdfjs-dist sits in node_modules as-is. But in the output: "standalone" runtime generated by next build — the side that actually starts on Cloud Run — file tracing (NFT) statically analyses dependencies and narrows .next/standalone to just the necessary files. standard_fonts/ could fall outside that narrowing.

Even when it is missing, the extraction itself throws nothing. On pdfjs-dist v6, an unresolvable standardFontDataUrl makes getTextContent() fail with UnknownErrorException, but that is easily swallowed by a try/catch on the calling side, leaving only the PDF’s extracted text near-empty. (On v5 it stopped at a warning and extraction still succeeded, which made the behaviour change in v6 easy to miss.)

Why

The path to standard_fonts/ is resolved at runtime through dynamic file:// access.

// extractors.ts
function standardFontAnchors(): string[] {
  return [import.meta.url, pathToFileURL(join(process.cwd(), "noop.cjs")).href];
}

function standardFontsDirFrom(anchor: string): string | undefined {
  const require = createRequire(anchor);
  const pkgJsonPath = require.resolve("pdfjs-dist/package.json");
  const pkgRoot = pkgJsonPath.slice(0, pkgJsonPath.length - "package.json".length);
  return `${pkgRoot}standard_fonts`;
}

standardFontAnchors() holds two candidate starting points because of how Next.js’s bundler works. In development and vitest, import.meta.url points at the source’s real location and resolves directly, but in a production server bundled by Turbopack, import.meta.url points at a virtual chunk location like .next/server/chunks/..., and a relative search from there cannot find pdfjs-dist. Hence the fallback that also tries a createRequire anchored at process.cwd() (the standalone runtime’s root).

The problem is that this path resolution via createRequire(...).resolve() is invisible to Next.js’s file tracing. NFT decides what to include by analysing static import statements and require() calls; it does not follow dynamic access to a string path assembled at runtime. As a result the standard_fonts/ directory itself can be missing from the .next/standalone output, or present but empty.

The awkward part is that the path resolution (require.resolve("pdfjs-dist/package.json")) can succeed even while the files are missing. The pdfjs-dist package itself is included for other reasons (the code’s own require), so package.json is found. But if only the neighbouring standard_fonts/ directory falls outside the inclusion, the path assembles while nothing is there. So this code does not judge by whether the path resolves; it checks that the real files exist.

// extractors.ts
function hasStandardFontData(dir: string): boolean {
  try {
    return existsSync(dir) && readdirSync(dir).some((name) => STANDARD_FONT_FILE_RE.test(name));
  } catch {
    return false;
  }
}

Fixing it

The response is two-stage. One is making the inclusion explicit in Next.js configuration; the other is being able to notice at production startup if it goes missing anyway.

① State the standard_fonts/ inclusion explicitly to file tracing in next.config.ts

// next.config.ts
const nextConfig: NextConfig = {
  output: "standalone",
  serverExternalPackages: ["pdfjs-dist"],
  outputFileTracingRoot: monorepoRoot,
  outputFileTracingIncludes: {
    "/api/**": ["./node_modules/**/pdfjs-dist/standard_fonts/**"],
  },
};

The keys of outputFileTracingIncludes are route patterns, and the value globs the extra files that route uses. Here ./node_modules/**/pdfjs-dist/standard_fonts/** absorbs pnpm’s nested directory structure (.pnpm/pdfjs-dist@*/node_modules/pdfjs-dist/...) with a wildcard. The route key is the broad /api/** rather than individual API paths because Next.js’s dynamic route segments (square brackets like [id]) are interpreted as glob character classes in the file-tracing world and do not match as intended.

serverExternalPackages: ["pdfjs-dist"] is needed at the same time. Including pdfjs-dist in the Next.js bundle makes the createRequire(import.meta.url).resolve(...) above try to resolve from a post-bundle chunk location and fail, and it also breaks the worker and font assets the legacy build depends on. Leaving it unbundled and letting it require from the real filesystem’s node_modules at runtime keeps the same resolution path in both next start and the standalone runtime.

② Detect it fail-fast at production startup, should the inclusion setting go missing

Not relying on the setting alone is because the inclusion path can be dropped again for some other reason, such as a Dockerfile rewrite. So register() in instrumentation.ts, called exactly once at production startup, invokes a guard that checks whether the font files resolve.

// instrumentation.ts
export async function register(): Promise<void> {
  if (process.env.NEXT_RUNTIME !== "nodejs") return;
  if (process.env.NODE_ENV !== "production") return;
  const { assertStandardFontsAvailable } = await import("./lib/pdf-extract.js");
  assertStandardFontsAvailable();
}

(In reality it imports from the internal package holding the extraction code. The point is that the dynamic import sits inside the nodejs guard, so a Node-only dependency graph is never pulled into the edge runtime.)

// extractors.ts
export function assertStandardFontsAvailable(): void {
  if (resolveStandardFontDataUrl() !== undefined) return;
  const dir = locateStandardFontsDir() ?? "pdfjs-dist 未解決";
  throw new Error(
    `pdfjs-dist standard_fonts のフォント実体を解決できません (${dir})。` +
      "標準フォント PDF の text 抽出がサイレントに空になります。" +
      "Cloud Run standalone バンドルに standard_fonts/ が同梱されているか確認してください " +
      "(apps/web/next.config.ts の outputFileTracingIncludes)。Issue #311。",
  );
}

The early return on process.env.NODE_ENV !== "production" is because development and CI unit tests have the font files in node_modules and would not normally throw; the guard is meant to catch the production-specific omission only. With this check in place, a missing inclusion changes from a hard-to-find degradation — “PDF text extraction quietly comes back empty” — into an easy-to-notice one: startup itself fails right after deploy.

Premises that are easy to miss

• A path resolving and a file existing are different things. require.resolve() succeeding does not mean the adjacent directory was included. Assets that depend on dynamic file:// access are worth suspecting of a file-tracing miss, and checking down to whether the real files exist.
• Development and CI unit tests cannot detect this kind of omission. It does not reproduce in a runtime where the whole of node_modules is present. The symptom only appears once a standalone build is actually run.
• Fixing the inclusion in configuration does not, by itself, prevent recurrence. Another change, such as a Dockerfile rewrite, can drop the same inclusion path again. The setting and the startup fail-fast guard are each insufficient alone.