Formatting and encoding

Formatters and encoders look like conveniences. Two of them are not: escaping is a security control whose correctness depends on knowing where the output lands, and encoding choices have measurable size and correctness consequences. This guide covers what the tools in this category are doing underneath.

Formatting is a correctness tool

Pretty-printing JSON, SQL, or CSS produces no functional change. What it produces is visibility, and the reason to do it before debugging is that structural errors are nearly invisible in minified text and obvious once indented.

A JSON formatter is also a validator, because it cannot indent what it cannot parse. Feeding a suspect payload through one is the fastest way to locate a trailing comma, an unquoted key, or a truncated response, and the parser reports a character offset rather than a vague failure. The same applies to SQL: a formatted query makes an unbalanced parenthesis or a join condition landing in the wrong clause apparent at a glance.

Formatting also matters for diffs. Two minified files that differ in one value produce a diff of one enormous line, which tells you nothing. Formatted, the same change is a single readable line. This is why committing formatted source and minifying at build time is standard rather than optional.

What Base64 is for, and what it costs

Base64 maps arbitrary bytes onto 64 characters that survive text-only channels. It exists because a great deal of infrastructure was built assuming 7-bit ASCII and will corrupt or reject anything else: email bodies, HTTP headers, JSON string values, URL parameters, and XML text nodes.

It is not encryption and provides no confidentiality. Decoding requires no key and is a single function call. Anything sensitive that is only base64 encoded is, for all practical purposes, in plain text.

The cost is structural. Three bytes become four characters, so output is exactly one third larger than input, plus padding. Embedding a 300 KB image as a data URI produces 400 KB of markup, which is uncacheable independently of the document, blocks the parser while it is read, and cannot be lazily loaded. Below a few kilobytes, saving a request may be worth it. Above that it is usually a loss.

Two variants matter in practice. Standard Base64 uses plus and slash, both of which have meaning in URLs and must be percent-encoded, doubling the escaping. The URL-safe variant substitutes minus and underscore and usually drops padding, which is what JWTs use. Mixing them silently produces corrupt output on decode.

Escaping is contextual, and getting the context wrong prevents nothing

HTML escaping replaces the five characters that carry syntactic meaning in markup — ampersand, both angle brackets, and both quote styles — with entity references, so untrusted text is rendered as text rather than parsed as tags. Done correctly it is the primary defence against cross-site scripting.

The critical qualifier is that HTML has multiple parsing contexts and each requires different escaping. Text between elements needs the five characters replaced. An attribute value needs the same plus strict quoting, because unquoted attributes can be terminated by a space. A URL in an href needs the scheme validated, since escaping does nothing to stop a javascript: URI. Inside a script block, HTML escaping is almost useless, because the JavaScript tokeniser runs first and needs JavaScript string escaping instead. Inside a style block, CSS escaping applies.

Applying HTML escaping to a value that lands in a script block is a common and serious mistake, because it looks like a defence and is not one. The rule that follows: escape at the point of output, once, with the encoder matching the destination context. Escaping on input cannot work, because at input time the destination is unknown.

The corollary is to prefer APIs that make the context explicit. textContent cannot execute markup; innerHTML can. A templating engine that escapes by default is safer than one requiring you to remember. Parameterised queries are safer than escaped SQL for the same underlying reason.

Percent-encoding and the two-function problem

URLs permit a restricted character set, and everything else must be percent-encoded as a byte value in hexadecimal. Where this goes wrong is that JavaScript provides two encoders with different reserved sets, and the difference is not obvious from the names.

encodeURI is intended for a whole URL and deliberately leaves the structural characters intact: slash, question mark, ampersand, equals, hash, colon. encodeURIComponent is intended for one piece of a URL and encodes all of them.

Using encodeURI on a query parameter value is a bug that survives testing, because it only manifests when the value contains a character encodeURI preserves. A parameter containing an ampersand splits into two parameters. A value containing a hash truncates everything after it, since the fragment is never sent to the server. Neither shows up until real data arrives.

One character resists both: the plus sign. In the query string of a form-encoded URL, plus historically means space, so a literal plus in a value must be encoded even though encodeURIComponent leaves it alone. Phone numbers in international format break on this constantly.

Frequently asked questions

Tools referenced in this guide