Skip to main content

Upload compression

POST /convert/raw accepts the compressed body. You reduce network bytes and speed up large uploads. Indicates the algorithm with the header content-encoding.
Only /convert/raw supports content-encoding. /convert (multipart) and /convert/from-url no.
##gzip

zstd

From code

How much do you save in practice

PDFs already usually have compressed internal streams (JPEG images, text with Flate), so gzip/zstd over a typical PDF saves less than over text flat. Even so, in documents with a lot of uncompressed text (PDFs generated from Word/LaTeX without optimization) or in Markdown output, the difference is remarkable: For uploads (/convert/raw), use a low level (gzip -1 or zstd -3): the bottleneck is the network, not the CPU, and a high level takes longer to compressing what you save on transfer. For output_url (where you compress once on the server and your backend decompresses it N times), a higher level may compensate.

Recommendations

  • For large uploads, zstd -3 usually gives the best ratio/CPU balance.
  • Avoid Brotli: saves bytes but consumes much more CPU.
  • The X-Input-Compressed-Bytes header tells you how many bytes arrived over the network after compression; X-Input-Bytes is the actual uncompressed size.
A content-encoding other than gzip or zstd responds 415.

Output compression (output_encoding)

When you use output_url in /convert/from-url, the converted Markdown is uploaded compressed to your storage. The output_encoding parameter controls the algorithm: When there is output_url, the API auto-promotes to zstd if you do not specify another value. Reason: in the BYOS case there are no browsers or CDNs in the way; client and server are backends that can unpack native zstd. zstd yields about 30% fewer bytes than gzip at similar CPU cost and supports streaming decompression on the client. Do not want to depend on S3/R2 for output_url/url? See Self-hosted S3 for open source alternatives that you can self-host. Overwriting is trivial:

Raw limit measures uncompressed bytes

The raw upload limit (RAW_UPLOAD_MAX_BYTES, 12 MB by default) applies to the uncompressed size, not the bytes traveling over the network. Compression does not let you upload a larger PDF through /convert/raw: a gzip/zstd body that decompresses above 12 MB is rejected with 413. Compression only saves network and upload time. For large PDFs, upload them to your storage and use /convert/from-url, which does not apply this limit. See Limits and Raw vs storage URL.

Troubleshooting

If you don’t know which algorithm has the best ratio for your specific document, try both with gzip -c file.pdf | wc -c and zstd -3 -c file.pdf | wc -c before you decide — the difference depends a lot on the internal content of the PDF.