JSON Lines vs SSE: Parse Streaming AI Responses Without Losing Events

A streaming response doesn't arrive in neat application messages. One network chunk can contain half a JSON object, several complete records, or the beginning of a multibyte character. Code that parses every received chunk as JSON will eventually fail even when the server sends a valid stream.

JSON Lines and Server-Sent Events solve different framing problems. Before writing the parser, identify which format the API actually sends. The useful question is where one complete application record ends, not how often the network delivers bytes.

Identify the protocol from the contract

JSON Lines conventionally places one JSON value on each line. SSE uses a text event stream with fields such as data, optional event metadata, and a blank line separating events. An SSE data field may contain JSON, but the whole SSE stream isn't one JSON document.

The JSON Lines format documentation specifies UTF-8 and one valid JSON value per line. A blank line is not a JSON value. Define how the client handles such invalid input rather than silently dropping anything inconvenient and reporting an intact stream.

The HTML standard's SSE specification defines its event framing and parsing rules. Follow the API's documented format as well. Don't assume that every streaming AI endpoint uses the same terminal marker or reconnection behaviour.

Check the response content type and provider documentation. If they disagree, preserve the response metadata and investigate in staging. Guessing between formats based on a convenient sample can create intermittent production failures.

Buffer incomplete text

Decode bytes using a streaming UTF-8 decoder so a character split between chunks isn't corrupted. Keep any incomplete text in a buffer until the framing rule identifies a complete record or event. Process complete units and retain the remainder for the next read.

For JSON Lines, a simplified stream might be:

{"type":"delta","text":"Hello"}
{"type":"delta","text":" world"}
{"type":"complete","finish_reason":"stop"}

These are fictional event records. An escaped newline inside a JSON string is different from the line delimiter between records. Your parser must respect the format rather than splitting an already decoded field arbitrarily.

Treat SSE data as event content

An illustrative SSE event can look like this:

event: delta
data: {"text":"Hello"}

The blank line completes the event. SSE can include comments and multiple data lines, so don't reduce its parser to “remove the first five characters and parse.” Use a maintained parser or implement the documented rules with dedicated tests.

When an event's assembled data is JSON, parse that data after event framing is complete. A plain-text heartbeat or documented completion sentinel may require a different branch. Never assume an arbitrary sentinel is a universal part of SSE.

Keep transport errors separate from application errors

A connection closing can mean normal completion, cancellation, network failure, or an interrupted response. Use the API's completion event or other documented contract to determine whether the application result finished successfully.

Track whether a final event was received and whether a partial buffer remains. A stream ending halfway through a record shouldn't quietly become a complete answer. Return a clear partial-result state and follow the workflow's safe recovery policy.

Don't automatically replay a request that performs side effects just because the stream broke. The remote operation may have completed while the response was lost. Reconcile its outcome using the API's supported mechanisms before repeating sensitive work.

Inspect records with the right tool

The JSON Formatter expects a JSON document, not an arbitrary sequence of JSON Lines or an SSE envelope. Extract one complete synthetic JSON record before formatting it. For a set of records, deliberately create an array in your own test environment.

Use Text Diff to compare synthetic raw stream fixtures. It can reveal a missing blank line, a changed event name, or a lost completion record. It does not act as a streaming parser or validate the protocol.

Keep real access tokens, conversation content, and private URLs out of shared debugging examples. A stream capture can contain far more sensitive information than the final answer shown to the user.

Give each fixture an expected ordered list of events and a completion state. That makes a regression visible even when the combined output text looks unchanged. Dropping a metadata event or duplicating a final event can break downstream logic without changing the sentence a user reads.

Test awkward boundaries deliberately

Split a synthetic response at every possible boundary, including inside JSON strings and multibyte characters. Feed those chunks to the parser and confirm that the same event sequence emerges regardless of chunk size.

Also test several events in one chunk, empty lines, comments, malformed records, oversized events, cancellation, and a connection ending before completion. Bound buffer growth so a broken stream cannot consume unlimited memory.

Measure performance only after parsing is correct. See LLM latency metrics for why network event pacing isn't automatically token pacing. A parser that batches events can change the apparent user experience without changing model generation speed.

Conclusion

Network chunks are delivery units, not application records. Identify the stream format, decode incrementally, buffer partial content, and apply the correct framing rules. Then distinguish complete results from interrupted streams and test the parser against deliberately inconvenient boundaries.