Personal Research Notes / 04

Speech, Dubbing, and Representation in the Age of AI

What Should Survive Localization?

Changing the language without losing the relationships that make the character.

Published 11 October 2026Working essay · Edited from Japanese

A translated line can be accurate word by word and still change a scene.

It may make a cautious person sound certain, turn restraint into aggression, or remove the difference between two speakers' social positions. The mouth can match while the character no longer does.

The earlier notes moved from the conditions of an utterance to the work of connecting a new voice with an existing body, then to the representation that connection produces. This final note asks what should remain coherent when the language—and potentially the voice and image—can be changed.

The question is not whether translation transforms a work. It does. The question is what those transformations are allowed to do.

A changing set of production choices

Some changes can be observed without claiming that film and television dubbing have become fully automated.

In February 2026, YouTube described automatic dubbing in 27 languages, Expressive Speech in eight, and a lip-sync feature being tested as a pilot. These were different stages of availability, not one uniform capability. Its May explanation again described 27-language support and eight languages with Expressive Speech. These are dated platform statements, not independent tests of performance in every language. (February announcement, May explanation)

Meta's October 2025 announcement described translation and dubbing for Reels that imitate a creator's vocal sound and tone, with optional lip syncing and eligibility conditions. That is a specific platform use case, not evidence that dramatic performance can be preserved without loss in every production. (Meta announcement)

What matters for these notes is the redistribution of choices. Translation, voice, timing, and image no longer have to be treated as one indivisible replacement operation. A workflow can potentially preserve one element while changing another.

That makes the old question more explicit: what is being preserved, and why?

When the image is no longer the fixed constraint

Scott Mann, the filmmaker and Flawless co-founder, has described seeing a foreign-language version of Heist as a reason for pursuing visual dubbing. In a SYFY WIRE interview, he explained that a scene with Robert De Niro no longer carried the dialogue and performance he intended. He described using a scene from the film as a test, recording the dialogue again and altering the visuals to synchronize with it. This is his account of the motivation and test, not an independent evaluation of the resulting version. (Interview with Mann)

The interesting change is not merely a more convincing match between lips and sound. It is a change in where the constraint sits.

If the image is fixed, translated dialogue and performance must fit its existing mouth movements. If part of the image can be adapted, there may be more room to establish the translated line's function before adjusting the mouth to it.

Neither arrangement is automatically preferable. Altering the visible performance can also change timing, expression, or the viewer's reading of the person. Moving a constraint does not abolish responsibility. It creates a new decision about what may be altered and how the result is checked.

This is how I read the example: a technical option makes a previously fixed hierarchy negotiable. It does not supply the criterion by which the negotiation should be settled.

Continuity is not identical form

The translated version cannot reproduce every feature of the original. Languages differ in word order, rhythm, forms of address, humour, and cultural implication. A demand for perfect surface identity would misunderstand translation.

But the fact that change is necessary does not mean all changes are equivalent.

A line can change its wording while preserving what the character is trying to do. A different rhythm may retain the hesitation that makes an answer meaningful. Another verbal device may carry a social distinction that has no directly corresponding accent in the target language.

Conversely, literal wording can coexist with a changed relation: an apology becomes a demand, an uncertain reply becomes a declaration, or a socially specific person becomes a stock type.

I use continuity of the work for the persistence of the relationships and causal connections through which the characters and scenes remain intelligible as the same work. This is an interpretive criterion, not a claim that one correct translation can be derived mechanically from the source.

It requires asking what a choice does to the work, not only what it resembles.

What prompts the utterance? What does the speaker want from the other person? What does the reply enable? What social relation is made audible? Does a later scene still follow from this one?

The intended continuity should be part of the localization brief before a tool is selected. It cannot be inferred from a smooth output alone.

Guidance already contains competing priorities

Netflix's public dubbing guidelines offer one concrete example of the judgments involved. They prioritize preserving dialogue intent and rely on translators and adapters when perfect lip sync cannot be achieved at the same time, while identifying key moments where synchronization should not be compromised. They also distinguish vocal matching from performance, prioritizing a credible experience when both cannot be achieved. Their guidance addresses casting, inclusive terminology, and research alongside recording, editing, and mixing. These are Netflix's production expectations, not universal rules. (Dubbing Creative Guidelines)

I read this as evidence that quality involves an ordering of priorities, not one maximized number. The existence of guidance does not establish that every delivered dub fulfils it.

The creative and technical questions also overlap. A timing decision can affect intention. A pronunciation decision can affect expertise. A casting or delivery decision can alter how a character is socially understood.

Representation is not necessarily an issue introduced after the production is finished. It can be present within the specification of what the production is meant to achieve.

What a human performance has to justify

If a workflow can generate usable translated speech, employing a human actor is no longer justified solely by the need to make words audible. The relevant question becomes what that performance contributes to the particular work.

This does not mean that an actor is required to add conspicuous originality to every line. Restraint, continuity, and the decision not to impose an effect may be precisely the contribution.

Nor does a human voice automatically enrich the source. If the new performance replaces a specific person with a familiar local type, human intervention can become a distortion. A generated version might preserve an important relationship more successfully in a particular case. The reverse may also be true.

The comparison has to be made at the level of the work, under its actual production conditions. A claim that “human voices have warmth” does not tell us which version preserves the character's action. A claim that “the model preserves the voice” does not establish that it preserves the scene's meaning.

Both can become shortcuts around evaluation.

Structure-preserving character reconstruction

In my Japanese draft, I called the relevant expertise the ability to reconstruct a character while preserving its structure.

In English, I describe it more plainly: the ability to re-create a character across languages while preserving the relationships that make the character intelligible.

“Structure-preserving character reconstruction” is the shorter working term I use after that explanation. It is a proposal in these notes, not the name of an established certification or metric.

The skill includes reading the original performance, recognizing the constraints of the new language, deciding which differences matter, and making an expression that can carry them. It requires knowing where a change is productive, where it is necessary, and where it breaks a relation the work depends on.

This responsibility is not the actor's alone. Translators, adapters, directors, editors, engineers, producers, and those approving the final version can all affect the continuity being sought. The workflow needs a way to communicate the intended relationships and return a failed choice for revision.

An actor can read a scene carefully and still be given an adaptation that changes the character's action. An accurate translation can be undermined by direction. A good performance can be altered through editing or synchronization. Evaluation at the final waveform is therefore insufficient if the preceding decisions are invisible.

What matters is whether the judgment can travel across the handoffs.

What remains to be read

There is a temptation to end with a neat division: technology preserves the surface; people read the structure.

That is too tidy. People also work on surfaces, and computational systems may support some kinds of contextual analysis. The responsibilities cannot be assigned once and for all by naming the producer of a sound.

The distinction I want to retain concerns the object of judgment. Similar timbre, fluent speech, and aligned mouths are not the whole object. The character's action and the work's relations also need to survive the transformation.

In that sense, new tools do not invent a duty that dubbing never had. They make it harder to hide the duty behind a technical limitation. When a constraint becomes adjustable, a compromise that once appeared unavoidable becomes a choice someone must explain.

The series began with a scripted line. It ends with the conditions under which another version can still belong to the same character and work.

Who is speaking? To whom? From what situation and social position? What is this utterance trying to do? What must remain coherent when it moves into another language?

Those questions do not disappear when the voice becomes easier to generate. They become more concrete places at which someone has to make a decision—and say why the result is acceptable.


Method note: the platform examples are dated statements by their providers. Mann's account is a filmmaker's and vendor co-founder's testimony. Netflix's guidance is a specific production framework. None is used here as an independent benchmark or proof that all the proposed capabilities operate reliably across contexts. The continuity and reconstruction criteria are the author's interpretive proposals. Sources were checked on 11 October 2026.

Previous: Representation Is Part of the Craft · Afterword: A Performance That Sent Me Looking