When I watch a film, I am not constantly inspecting how its voices were produced. I am listening to what the character in front of me is saying, whom they are speaking to, and what they might be hiding. Once I am absorbed in the scene, recording and editing recede into the background of my attention.
Yet when I began looking into AI dubbing, the question of how a voice was made suddenly became difficult to ignore. Performers' work, the quality of generated speech, translation, costs, rights: all of these matter. But if the discussion stops there, something is missing. What is happening to the person watching the film?
Even with human-performed dubbing, some viewers will stop watching if they cannot make sense of a scene or if the voice does not fit the character's behavior. With generated speech, there may also be viewers who can hear it as the character's words and follow the story without worrying about how the voice was produced.
Considering that possibility does not mean treating humans and machines as the same. We can distinguish the process that produces a voice from the role it plays in the experience of watching a film. It is the relationship between those two things that interests me.
Hearing the Words and Understanding the Character
A dubbed line does not necessarily convey everything it needs to convey simply because its words are intelligible.
Consider a line such as “I'm fine.” Is the character genuinely all right? Trying to reassure someone? Hiding how badly they are doing? The words may be the same, but what the character is doing with them is different.
When a tired-looking character speaks in a calm voice, we might understand them as making an effort to maintain their composure. If the surrounding scene does not support that interpretation, however, the voice might seem detached from the character. A voice and a face that do not express the same emotion are not automatically a failure.
This is an illustrative example, not an assessment of a particular film. Does the line convey not only its verbal meaning, but also what the character is trying to do, and to whom? That judgment includes the relationship between sound and image.
Even the perception of speech sounds is not independent of vision. In the experiment known for the McGurk effect, an auditory “ba” combined with the lip movements for “ga” was reported as “da.” The visible movement of the mouth influenced the judgment of what sound had been heard. 1
In a study of the Kuleshov effect, changing the contextual footage paired with identical faces altered judgments of their expressions and mental states. 2
These findings do not establish a single shared mechanism. Perceiving a speech sound and interpreting a character's state are different questions, and neither study directly demonstrates the acceptance of AI dubbing. They do, however, offer a starting point for considering why evaluations of isolated voices or expressions cannot fully capture how they are received within a film.
Different Processes, Overlapping Functions
An actor works through their body and voice, taking account of the character's situation and relationship to others. Breathing and hesitation may emerge through that action, or they may be deliberately shaped as part of the performance.
We need not imagine human acting as entirely dependent on accident or emotional collapse. Using conventions, coordinating timing, and performing the same line repeatedly are also part of the craft. A controlled performance is not, for that reason alone, an empty one.
Describing generated speech as nothing more than reading out a string of text is also too narrow an account of how it is made. Some synthesis methods learn from speech and use reference audio to condition their output. 3 Producing a sound that suggests a performance is nevertheless not the same event as an actor physically performing that scene.
That difference remains. But a difference in production does not determine, in a one-to-one fashion, what a listener will receive.
In live-action dubbing, the original actor's work is still present in the image: expression, gaze, posture, distance from another person. Speech in another language is connected to that existing performance to construct the character's utterance. A change in how the voice is produced does not suddenly remove human acting from the entire film.
My question here is not whether the waveforms become identical. It is how voices made through different processes can be heard, in particular scenes, as particular actions performed by a character.
A similar voice quality, natural-sounding speech, and a convincing character are different things. A single scale of “closeness to a human” does not readily account for those distinctions.
Filling In Is Not the Same as Effortless Viewing
Viewers do not merely receive the information presented to them. What I mean here by “filling in” is the work of interpreting a line's meaning and intention through the surrounding scene and the relationships between characters.
But filling in is not a magic way to erase flaws.
Understanding what is happening is not the same as watching with ease. If I have to reconstruct a character's intention every time a line is spoken, I may be able to follow the story and still find it tiring. Conversely, a slight sense that something is off might not prevent me from understanding the important information.
Not noticing a discrepancy, noticing it without minding, and interpreting it as a feature of the character are also different responses. If we gather all of them under “the brain fills it in automatically,” we lose track of what worked and what was compromised.
Familiarity requires the same care. Viewers may come to accept a particular style of dubbing as a convention. Getting used to it, however, is not the same as liking it, and it does not correct a mistranslation. Finding a voice natural is also different from approving of the method used to produce it.
It is difficult to prescribe one correct response here. Language proficiency, expectations of the film, previous viewing experience, and personal taste all leave room for different ways of receiving the same work.
That does not mean everything can be dismissed as a matter of taste. We can examine whether meaning is conveyed, whether a character's intention has been changed, and whether a mismatch obstructs understanding. We can then keep separate the questions of whether someone wants to continue watching, enjoys the experience, or comes to like it.
When We Are Absorbed, Production Recedes from View
When I am absorbed in a film, I am more concerned with “Why did this character say that now?” than with “Who recorded this voice, and how?” This experience during viewing is what I mean, first of all, when I say that production recedes into the background.
That is different from forgetting about production. After the film, I may want to know who performed the role, how the performance was directed, and how it was edited. If something jars, my attention may also move away from the scene and back toward how the voice was made.
Hearing a voice as a character's speech and knowing how it was generated are not necessarily mutually exclusive. It may be possible to know that a voice was made by a machine and still receive it as the words of a character within the story. Attributing a line to a fictional character should also be distinguished from attributing personhood to the machine itself.
I do not want to describe this response simply as being deceived, or as a cognitive bug. Cinema already offers us separately produced images and sounds that we experience as events within a scene. Would treating that way of watching as a defect, the moment AI enters the picture, not narrow our account of the viewing experience?
Of course, a natural-seeming result does not guarantee consent, provenance, permitted use, or accountability in production. Working as a viewing experience and being legitimately produced are both important, but they are separate questions.
The absence of a body that performed a particular utterance is not the same as an absence of human labor or responsibility. Generated speech can also involve people who supply material, translate, direct, edit, select, and approve it. Precisely because these processes can be difficult to see, there is still a reason to record who made what, who authorized its use, and who judged the work complete.
Viewing Experience Is Not an Afterthought to Production
In discussions of AI dubbing, improving production efficiency and protecting human performance can appear to be opposing concerns. But if that opposition becomes the entire discussion, the experience of the person receiving the work can fall outside it.
What reaches an audience is not determined solely by what the makers provide. Nor can audience satisfaction alone determine the whole value of a work or its production.
It is the relationship that I want to examine. What do we want to convey as this character's utterance? How are voice, image, dialogue, and timing combined to do that, and what does the viewer make of them? Viewing experience is not merely a matter of adding reactions once production is complete. It can also be a starting point for deciding what we are trying to make.
A body actually producing an utterance, and a character being heard as speaking: how far can perception accommodate the difference? Even where it can, what value do we continue to place on the lived experience and singularity of an actual performance? These questions cannot be answered by comparisons of accuracy or cost alone.
Without AI dubbing, I might not have examined so closely what I meant by “that was a good voice performance.” The voice itself, the acting in the image, translation and editing, the viewer's memories: having another means of production to compare with has made me look again at how these elements were connected all along.
I do not want only to determine whether a voice is human or machine-generated. I want to ask what reaches us, where we stumble, and what we enjoy. Neither the differences in production nor the experiences that arise in viewing should be lost from the discussion. It is the dynamics between them that I want to keep observing.
References and Scope
This essay begins with the experience of watching films. It is not a report of an experiment measuring the acceptance of AI dubbing or viewer retention. The “I'm fine” scene is an invented illustration. The distinctions between filling in, effort, familiarity, and preference are a framework for inquiry in this essay, not a set of conclusions all established by one experiment.
- McGurk & MacDonald (1976), Hearing lips and seeing voices. The publicly available abstract of the original paper was consulted for the Japanese edition. It is cited as an example of visual influence on speech-sound perception, not as evidence of acceptance of AI dubbing or an emotional response to it.
- Mobbs et al. (2006), The Kuleshov Effect: the influence of contextual framing on emotional attributions. The publicly available abstract was rechecked for the revised Japanese edition; the full text had also been consulted while preparing the earlier draft. It is cited for experimental changes in judgments of identical faces' expressions and mental states under different contextual framing.
- Wang et al. (2023), Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers. The publicly available abstract was consulted while preparing the earlier Japanese draft. It supplies an example of synthesis conditioned on more than text, not an account of every TTS model or evidence of an understanding of acting.
First published 13 October 2026. English translation of the Japanese column, retaining the author's first-person perspective and the limits of the cited evidence. Written and edited by the LAYER SHIFT editor; published by Ausreißer Verlag.