What broke when I rendered YouTube transcripts into EPUB 3

I have been building a pipeline that takes articles and YouTube videos, rewrites them into something shorter, and delivers the result as an EPUB you read on a Kindle in the evening. Most of it is unremarkable. The parts that were not unremarkable were nearly all cases where the system was confidently wrong rather than broken, which is the failure mode that costs the most time, because nothing throws.

These are five of them, with the numbers that are actually in the code.

1. The input cap that ate the ending

The transform sends source text to a model with a character cap. Mine was 12,000. A forty-minute talk transcribes to something closer to 60,000 characters, so the model was reliably summarising the first two thirds of the video and never seeing the rest.

What made this expensive to find is that the output looked fine. A summary of the first two thirds of a talk is a coherent summary. It has an introduction, it has themes, it reads like someone paid attention. The only symptom was that the conclusion was missing, and nothing in a summary tells you a conclusion is missing. I noticed because a video ended with the point the whole thing had been building towards, and the digest ended with a middle section.

MAX_INPUT_CHARS = 40_000

Raising the cap fixed it for the lengths I care about, and truncation is now marked in the output rather than silent. The general lesson is duller than the bug: when a component degrades by producing less rather than by failing, you need a signal that says how much of the input it saw. A log line with the truncated length would have found this in a day rather than a fortnight.

2. Confusing an output ceiling with a bill

I had set max_tokens to 4096, on the theory that a lower ceiling meant a cheaper call. It does not. You are billed for generated tokens, so a ceiling is a limit on what you allow, not on what you pay for. What a low ceiling actually buys you is a truncated response.

The output is JSON. Truncated JSON is not partially useful, it is a parse error, and my first instinct was to add a retry. That was the real mistake. The retry ran under the same ceiling against the same input, so it truncated in the same place, and I had turned one wasted call into two.

stop_reason = getattr(message, "stop_reason", None)
if stop_reason == "max_tokens":
    # The JSON was cut off mid-structure. The correction retry below cannot
    # help — it would run under the same ceiling and truncate in the same
    # place — so record it and skip straight to degrading.

The ceiling is now 12,000 and the same for every source, and truncation is detected from stop_reason rather than inferred from the parse failure. Those are two separate fixes and the second one matters more. A retry is only worth making when something about the next attempt differs; if nothing does, the retry is a way of spending money to reach the same conclusion more slowly.

3. The model renumbered a list, and covered its tracks

This is the one that changed how I think about the whole contract.

A video titled as twelve lessons produced a digest with twelve lessons in it. The numbering was continuous, one through twelve, no gaps. Two of the lessons in the middle were missing, and the ones after them had been shifted up to fill the space.

Nothing about the output looked wrong. It was internally consistent, which is exactly what made it dangerous: a gap would have been visible, and a renumbering is not. I only caught it by watching the video with the digest open beside it.

The interesting part is that the information needed to prevent this was sitting in the transcript. The speaker says the numbers out loud. He says the equivalent of "the fourth lesson is" before the fourth lesson. I had been asking the model to rediscover a structure that the source states explicitly, and then trusting its answer over the source.

So a regex reads the numbering first, and the result is passed to validation as ground truth. It costs no tokens and cannot hallucinate. It is also language-specific in a way I did not expect to be comfortable with: my ordinal table is Vietnamese, because that is what my sources are, and the pattern only matches an ordinal token after the word for "th".

The part I would keep if I rewrote this is not the regex. It is the set of conditions under which the regex is allowed to be authoritative:

• at least three markers, so that one stray phrase is not a structure
• a strict contiguous run from one to N, with no gaps
• positions that increase through the text, so a recap cannot masquerade as a section
• at least 200 characters between consecutive markers, so a passing mention is not a heading

If any of those fail, the extractor returns nothing at all and the pipeline falls back to the plain contract. That was deliberate. A heuristic that is right most of the time is useful; a heuristic that is right most of the time and is treated as authoritative the rest of the time is worse than not having it, because it converts a visible failure into an invisible one. Articles and unstructured talks have no numbering, and they must not get a guessed one.

4. A validation rule that taught the model to quote

Each extracted idea carries an artifact: a quote, a number, a copy-ready prompt, something concrete the reader can take away. To keep those honest, every artifact had to carry a verbatim span from the source, and validation checked that the span appeared inside it character for character.

The twelve-lesson video came back with six artifacts, and all six were quotes.

That is not a coincidence and it is not the model being lazy. Quoting was the cheapest way to satisfy the rule I had written. A number like "1% a day compounds to 37x a year" is a real artifact and does not repeat any span of the source word for word; a rewritten prompt does not either. I had built a validator that accepted exactly one kind of output and then wondered why I got one kind of output.

# Containment applies to quotes ONLY. Requiring every kind to carry
# its span verbatim is what collapsed a 12-lesson video into 6/6
# `quote` anchors: quoting is the cheapest way to satisfy the rule,
if anchor.get("kind") == "quote" and evidence:
    if normalize_for_match(evidence) not in normalize_for_match(content):
        errors.append(...)

Containment now applies to quotes and nothing else. The other kinds stay honest differently: the composer prints the source span as a caption underneath, so provenance is visible on the page without forcing every artifact to be a quotation. They still have to clear a length floor, because a bare label is not an artifact.

The general shape of this bug is worth naming, because I do not think it is specific to language models. A validation rule applied to a generator does not only reject bad output. It describes the cheapest acceptable output, and anything optimising against it will find that description and sit on it.

5. Images for a screen with no colour and no patience

A video chapter reads better with a frame from it, so the pipeline pulls the thumbnail with yt-dlp and then cuts frames with ffmpeg at each chapter mark. Videos without chapters get a frame every five minutes, and the whole thing is capped at eight frames.

The cap is not about file size in the abstract. It is about what an EPUB is for. Eight images already make a document that takes a noticeable moment to open on an e-reader, and a chapter illustrated four times is not four times as clear.

Every frame is converted to greyscale, resized to at most 800 pixels wide, and saved as JPEG at quality 60 before it goes anywhere near the book. Greyscale first, because the device cannot show anything else and a colour image is bytes spent on information the screen will discard. 800 pixels because that is around the panel width, and a larger image is downsampled on a device with far less headroom than the machine that made it.

One EPUB detail that cost me an evening: images have to be added as EPUB resources with their own manifest entries, not referenced or inlined as data URIs. Readers vary in how much they tolerate here, and the failure is not an error message. It is a book that opens with a broken-image placeholder on someone else's device and looks perfect on yours.

The pattern underneath all five

Four of these five produced output that a reasonable person would have accepted: a coherent summary of two thirds of a talk, a continuously numbered list with two entries missing, six well-chosen quotes, a book that renders on the machine that built it. The fifth threw a parse error and was fixed in an hour.

The thing I would tell myself at the start is that when a pipeline degrades by producing less rather than by failing, the correctness question is not "did it work". It is "how would I know". Most of what I changed was not the model, the prompt, or the ceiling. It was giving each stage something to check its own output against: a marker table read from the source, a stop reason read from the response, a length recorded next to a truncation.