Election 2 (2006): Music scoring at Milkyway is a pressurized but creatively open practice (see Endnotes 12 and 16).
PTU (2003): Sergeant Ho (Simon Yam) questions a suspect at a games arcade.
PTU: The defiant suspect refuses to inform on his comrade.
PTU: As Ho repeatedly strikes the youth, the blows are synched to video game sounds.
Chappell states that he tries to place the viewer in the character’s “earspace,” as in Lam Suet’s anguished gunfight in PTU…
…Nicholas Tse’s interrogation in Time and Tide (Tsui Hark, 2000)…
…and the prelude to Edison Chen’s restaurant rampage in Dog Bite Dog.
The opening sequence of Where a Good Man Goes (1999) blends ambient sounds and music score.
Johnnie To’s laconic shopping mall shootout in The Mission (1999).
The Mission: Johnnie To ratchets suspense through a series of tension-filled tableaus.
The Mission: Sporadic gunshots puncture the stillness.
The Mission: Chappell dilates the gunshots in an effort to accelerate the action.
The Mission: An assassin disguised as a janitor wheels a screeching cart into the melée.
In Jackie Chanís Project A (1983), a coastguard holds a tray of pasta as he observes a tavern brawl. Unexpectedly, a figure springs up from the bottom frameline.The unfortunate coastguard is soaked in spaghetti sauce. Director Chan exploits the limitations of the frame for comedic effect, in a way that Arnheim might have admired (see Endnote 30).
Milkyway regular Lau Ching-wan as the eponymous Mad Detective (2007).
As stated, the sound track in most Johnnie To productions is designed by Martin Chappell, whose firm Fork Media operated out of the Milkyway premises until 2011. The pair first met in 1997, when Chappell was employed at MGM’s offices in Hong Kong. Hired to provide sound tracks for MGM “promo” trailers, Chappell gained a reputation as a fast, efficient, and creative sound designer, proficient in Fairlight editing software. On the advice of Fairlight executives, Johnnie To asked Chappell to join Milkyway’s sound unit. Milkyway Image was still a fledgling studio, and its financial resources unstable. Keen to consolidate his own position, Chappell launched Fork Media, which would remain an independent firm even while housed in the Milkyway production building.
Founded in 1998, Chappell’s company had to adjust rapidly to the bracing work tempo of the Milkyway milieu. Stringent deadlines tightened the postproduction schedule, abbreviating the standard time allotted for postproduction sound. Most Hong Kong studios devote three or four weeks to postproduction sound, but Chappell mixed PTU (Johnnie To, 2003) in five days. Love on a Diet / Sau sun nam nui (Johnnie To, Wai Ka-fai, 2001), Wu Yen / Chung mo yim (Johnnie To, Wai Ka-fai, 2001), and Help!!! / Lat sau wui cheun (Johnnie To, Wai Ka-fai, 2000) each consumed less than a week of postproduction sound editing. [open endnotes in new window] During these intensive periods, reels are pored over tirelessly, and working hours far exceed those that Hollywood unionists endorse. Exacerbating the process is a sometimes heaped set of tasks, as sound tracks for several films may be prepared simultaneously. Chappell does not tackle these tasks single-handedly, of course, but his cadre of technicians is pocket-sized compared with Hollywood sound crews. Buoyed by no more than four workers, Milkyway’s sound unit adopts a fairly loose division of labor, parceling out internal duties of foley recording, ADR, sound mixing and the like. Chappell supervises all activities and takes charge of sound design and editing. He also collaborates closely with Johnnie To, experimenting within stylistic parameters set by the director. In all, the Milkyway sound studio flourishes under some exacting pressures. Yet, however rigorous the working situation might be, creativity is hardly stymied. Invariably, To and Chappell produce intricate and inventive soundscapes that belie the pressurized demands of postproduction.
An intense production schedule calls for flexible technical facilities. Here again, the resources are modest. Until 2006, Milkyway sound tracks were assembled on Fairlight MFX and Yamaha O2R mixing consoles, both fairly common platforms for audio engineering in European and U.S. studios. Though remarkably stable, the Fairlight system possessed inadequacies. In the late 1990s loading sounds onto the console was highly time-consuming, and the system offered few options in the way of reverbs or creative effects. Consequently, Chappell began exploring an alternative workstation and sequencer—MOTU Digital Performer—while editing Tsui Hark’s Time and Tide / Shun liu Ni liu (2000). He finally deployed it extensively in Soi Cheang’s Dog Bite Dog / Gau ngao gau (2006) and thereafter the software was assimilated into Milkyway’s auditory arsenal. Today Digital Performer is entrenched as the studio’s chief digital audio workstation. As one would expect, cost-cutting drives the choice of technology—Digital Performer is a more economical option than ProTools, the international industry standard. But the cheaper option also brings aesthetic advantages that, until very recently, were unavailable to the industry favorite. One expedient is its non-destructive region gain, which enables greater control of sound levels and signal quality. From this a finer-grained synthesis of acoustic dimensions can be achieved—for example, dialogue, ambience, and foley can be subtly blended. (We’ll see presently that Chappell and To exploit this technical advantage for dramatic effect.) Recent Milkyway productions have consolidated other low-budget programs as well. Since Turn Left, Turn Right / Heung joh chow heung yau chow (Johnnie To, Wai Ka-fai, 2003), the Propellerhead Reason sampler has become a constant resource. And AudioFinder software yields practical benefits. Providing quick access to audio files, the program simplifies portability of sound clips, making archived sounds faster to import and export. Inevitably, Milkyway’s lean postproduction phase profits from such time-conserving applications.
Cooperation between To and Chappell is based on a steady routine. From the start, Chappell is empowered with broad creative license. Under To’s general auspices, he initially outfits two or more reels with sound. Subsequently, To reviews the temp track, supplying guidelines for fine-tuning. Chappell modifies ambient noise, adjusts reverb on dubbed speech, and executes other refinements specified by To. Particular effort goes toward embedding the music score among other sounds. Director To completes music spotting prior to sound editing, establishing the sound level for each music cue, and marking the precise junctures that certain cues will “sneak” in and out of the sound mix. Consequently Chappell’s sound design must partly be oriented to prearranged music cues that constrain sound spotting to some degree, but which also provide Chappell an initial mood around which to organize foley and effects. As a default principle, Chappell strives to dovetail diegetic sound and music score—a design tactic honed in the advertising trade. When in PTU a nondiegetic music cue fizzles into a single bass note, Chappell loops and therefore dilates the tone, repurposing the cue to furnish the scene’s ambient sound. In Sparrow, he recruits diegetic noise into a percussive rhythm, blending rainfall and clicking traffic signals with a jaunty music track. Similarly, the opening brawl in Where a Good Man Goes / Joi gin a long (1999) yields thuds and slaps rhythmically attuned to the score’s percussion; as music augments the scene’s ambient texture, sound effects become rhythmic components of the music. “I learned in the promo world to blur the line between sound effects and music,” Chappell says. At its limit, this synthetic principle engenders a wholly integrated acoustic canvas, an organically unified sound track. As such, it belies the discrete stages of music production and sound effects editing, while endowing the sound track with an integrity that is rare for Hong Kong film.
If Chappell’s sonic choices spring partly from the music score, they are triggered most directly by To’s evocative imagery. The silent narrative image harbors clues for sonic elaboration. “A lot of the information I need is in the visual image,” Chappell states. “Hopefully I can spot it and link a few things together.” Johnnie To’s shots contain a kind of acoustic suggestiveness, instantiating what Sergei Eisenstein called “sound-pictures” or “audio-visual compositions.” This is one oblique facet of Chappell’s collaboration with To—the sound designer creatively interfaces with the image structure of the film. As before, default heuristics guide Chappell’s design strategies. Most basic is the necessity to steer attention to salient regions of the image. The mute composition, couched within a broader narrative context, hierarchizes zones of interest; Chappell’s default task entails impressing the most narratively significant zones upon the spectator’s consciousness. Still another heuristic promotes a congruence of sound and movement. Synchronizing sound with movement within or between images not only funnels attention, but it charges the visual field with material and expressive force, intensifying the image’s silent drama. “Ambient sounds can be timed with the movement of the camera,” Chappell notes. Movement yielded by editing can also find acoustic embellishment, as when a horizontal shot wipe in The Mission / Cheung fo (Johnnie To, 1999) is underscored by a filigreed whoosh.
None of these tactics is especially innovative or peculiar to Hong Kong film. The point is that such maneuvers are heuristics and default strategies, and as such they constitute an indelible component of film practice. The strangulating production schedules of Hong Kong cinema force filmmakers to rely on tried-and-proven methods and tacit craft knowledge. Just as important—given time pressures—is the filmmaker’s “spontaneous” reliance on instinct and intuition, crucial yet often downplayed factors in any national or regional filmmaking practice. Eisenstein, writing in the early 1940s, stressed the degree to which “compositional ‘intuition’ is responsible for correct audio-visual structures,” arguing that “those senses of ‘instinct’ and ‘feeling’ can materialise sound-picture” combinations. Intellectualizing the creative process is, for Eisenstein, the province of “post-analysis” whereas the filmmaker at work thinks chiefly and “directly in terms of manipulating his resources and materials.” What Eisenstein summarizes as “spontaneity” is ingredient to any filmmaker’s craft practice, but the exigencies of Hong Kong film production exacerbate it. From this angle, the constraints imposed on local filmmakers are not mere corollaries of Hong Kong film production. They are mediating forces that explicitly shape film practice.
Artistic intuition ought not to be construed as mere accident, anathema to deliberate intention, reflection, or artisanship. On the contrary, it relies upon the filmmaker’s personal storehouse of practical experience. Intuition, spontaneity, and artisanal rules of thumb are informed by experience of trial-and-error as well as by an effort toward novelty and variation, and the artist applies conscious “testing mechanisms” to artistic solutions during the process of composition. Artists, writes Brian Boyd, “solve particular artistic problems in individual ways, using repertoires of materials and methods accumulated over years of creative effort.” All things being equal, these intuitive aspects of practice—which are not antithetical to conscious artistic design—are intensified by contextual mediations such as economic constraints and production deadlines. Though production constraints impose determining conditions upon film practice, they may of course serve as enabling factors, engendering the expressive enterprise and ingenuity of the filmmaker.
So it is with the practice of postsynchronization. The imperatives to construct the soundscape from new; to simulate sounds implied in the “sound-picture;” to add specificity to a diegetic space clean of direct sound; to interweave voices, noise, and music score—in all such ways, the practice of postsynchronization inclines the sound designer toward creative invention. Presented with a purely visual array, Chappell confronts a vast number of options. Functionality and dramatic clarity are baseline priorities, but a sound may settle anywhere on a continuum of denotative, expressive, and abstract effects. The designer’s creative engagement becomes most salient when a sound swerves onto expressive or abstract terrain. Consider a sequence set in a games arcade in PTU. An interrogating officer smites a youth with a barrage of stinging blows. As the assault intensifies, Chappell displaces palpable slaps with ambient arcade noise, synchronizing video game pyrotechnics (explosions, swooping missiles) with each strike to the suspect’s face. Had To shot the scene using sync sound, with the audio track requiring only cosmetic reprocessing, the acoustic effects would have been far more prosaic. (Note that ingenuity here piggybacks on a reliable heuristic. The sound-image relation is creatively embellished, but still it rests on the synchronization of sound and figure movement.)
Here, then, is one source of the distinctiveness of To’s cinema. Postsynchronization — constructing the soundscape afresh — encourages the sound designer toward creative elaboration, to go beyond purely referential sound effects. And, as already noted, it confers upon filmmakers total control of the acoustic field, a circumstance particularly prized by doyens of cinematic style such as To.
Johnnie To’s critical reputation as a film stylist is by now well-entrenched, but it rests principally on ingenuities of composition and cinematography—that is, on the pictorial dimension of cinematic style. Are To’s sound tracks any less meticulously designed than his shot structures? In turning to the stylistic treatment of sound, we concern ourselves with a cluster of questions. What traits and tendencies characterize sound design in To’s films? How do these acoustic principles relate to broader aesthetic traditions? In what ways does To’s sound track intersect with other dimensions of style? And in what other ways is the soundscape determined or facilitated by the (post)production practices itemized above?
Perhaps intuitively, To and Chappell have established a recurring set of acoustic traits. A favorite device involves the subjective use of sound (or “putting the audience in between the character’s ears,” to borrow Chappell’s phrase). The climactic shootout in PTU provides a flagrant instance, as Lam Suet’s muffled scream, and the deepened resonance of gunshots, evokes a traumatized mental state. Chappell would develop the subjective aural cue in films by other directors—most strikingly, Tsui Hark’s Time and Tide and Soi Cheang’s Dog Bite Dog—and as we will see presently, he puts the device to effective narrational use in Mad Detective. More broadly, To’s collaborations with Chappell put sound and image in tension. An example is the celebrated shopping mall shootout in The Mission. Pitting the central band of bodyguards against a team of assassins, Johnnie To ratchets suspense by exaggerating the pauses between bursts of gunplay. Consequently the sequence is marked by an austere stillness fraught with tension. As was customary, Johnnie To presented the sequence to Chappell with the music cues spotted. Chappell, struck by the sequence’s longueurs, pursued an opposite inclination, to “speed the scene up”—an “emotional instinct,” he says, shaped by his background in advertising. He set about experimenting with Fairlight’s time stretch algorithm, distending certain sounds; and he made exaggerated use of stereo effects, ricocheting gunfire through different channels. Whereas the scene’s image track flaunts stasis and inaction, the sound track strives to fill in the ellipses. Gunshots are dilated, sometimes morphing into a low-level electronic hum. A single gun blast registers a sonic “tail,” ostensibly reducing the gaps between gunfire. The clatter of bullet shells striking the floor further dynamizes the sound track. And when a janitor wheels a squeaking cart into the fray, Chappell primes his entrance several shots before the character is shown on screen. The sequence ends with a horizontal shot wipe, characteristically embroidered by a sonic whoosh (another hangover from Chappell’s advertising days).
The shopping mall sequence in The Mission highlights two recurring features of To and Chappell’s work—a demonstrable tension between sound and image; and a tendency for sound to prefigure the image, as in the case of the faux janitor’s squeaking cart. We might describe the latter device as a sound cue that is “retroactively communicative,” and in several films To and Chappell exploit this narrational device for a variety of effects.
In Turn Left, Turn Right, for instance, Takeshi Kaneshiro sits in a public park, playing a violin. Director To frames Kaneshiro in a series of close views (deleting a communicative establishing shot), so Chappell adds ambient noise—principally the cooing of pigeons—to evoke the park locale and to underscore the violin melody. Presented in medium shot, Kaneshiro concludes his performance, stands, and executes a deep bow. Is he addressing an audience outside our field of vision? To’s narration cuts to a wider framing, revealing Kaneshiro’s “audience” (and delivering the punchline)—a congregation of pigeons has gathered at the musician’s feet, cooing appreciatively.
The payoff is a comic one, arising from a play with the spectator’s knowledge. Exploited here is a stylistic gambit that reaches back into film history—the delimiting boundaries of the frame are requisitioned to suppress story information and yield humorous effect. Johnnie To thus revives a visual tactic from early cinema, one that Rudolf Arnheim esteemed in the work of the American silent comedians. Though Arnheim famously disdained film sound, he might have admired its use here. To and Chappell generate humor by exploiting not only the medium’s limitations of visual presentation, but also the restrictions of the auditory field (e.g. the omnidirectional nature of sound, which fails to discriminate a general morass of birdsong from a specific chorus of coos beneath the lower frameline).
In retrospect, we notice that image and sound have pursued divergent narrational paths. Whereas To’s image track suppresses information, Chappell’s ambient noise hints at the payoff to come. This disparity comes forward as a recurrent feature of To’s films with Chappell, plumbed in several works for a range of effects. A more intricate instance occurs in Mad Detective. Bun (Lau Ching-wan), a brilliant former detective, is visited at his apartment by Inspector Ho (Andy On). Ho tries to coax Bun back into service. As the men converse, Bun’s wife May (Kelly Lin) obliquely conveys her disapproval by producing a distracting din in the adjacent room. The scene climaxes upon a major plot revelation: May is a hallucination born of the eponymous Bun’s psychosis. Until this disclosure, the scene’s visual narration is strategically suppressive. We assume that May is perceptible to Ho, but a calculated montage of shots loosely focalizes May around Bun’s experience. That is, May is never rendered “objectively,” but is tacitly moored to Bun’s interior perception. The narration thereby generates a kind of “inattentional blindness” in regard to Inspector Ho; not expecting a narrative feint, we simply fail to register his non-acknowledgment of May. In short, To’s visual narration is deliberately restricted, occluding any suggestion of the impending plot twist.
Now consider the scene’s acoustic narration. Prior to the surprise revelation, sounds associated with May become unusually prominent, magnified in volume. They hook the viewer’s auditory attention by flouting the primacy of dialogue, intruding upon the cops’ conversation. Moreover, May’s speech registers an unnatural and dematerializing timbre, quite distinct from the other characters’ unvarnished vocalizations. Such sonic distortions tacitly evoke what Michel Chion calls “point of audition sound,” aural elements anchored to a character’s auditory perception. By such means, Chappell’s sound track obliquely hints at character subjectivity, and primes the upcoming revelation. Here again acoustic elements react against the visual narration, not in the form of audiovisual counterpoint (i.e., where sound contradicts image content) but in terms of a differing “disposition” toward the viewer’s range and depth of knowledge. With hindsight, we notice that the sound track provides the scene’s most communicative clues to character subjectivity and narrative surprise. These clues, being tacit, do not disclose the revelation, but rather generate the narrational principle of “fair play,” allowing the most astute of viewers to predict the deus ex machina. In this case, To and Chappell tap a favorite audiovisual tactic—the tacitly communicative cue—for fresh narrational effect: the foreshadowing of plot revelation.
PTU elaborates the anticipatory cue in still another way. Early in the film, To presents a uniformed special unit officer, Sergeant Ho (Simon Yam), in medium shot. As Ho investigates a crime scene, he pivots in place, arcing his body toward a colleague beside him. Chappell lends the gesture materiality by postsynching the creak of Ho’s leather boots. This minute detail adds a subtle texture, but nothing in the image mandates this particular sonic choice. Once more the sound track evinces a certain independence of the image, though the cue does animate a default principle of audiovisual linkage—the yoking of sound to movement. In The Film Sense, Eisenstein writes of a perfunctory mode of synchronization that simply “matches the boot with its creaking,” but Chappell goes beyond such elementary combinations to subtly augur a forthcoming visual motif. The creaking noise is revived in a subsequent scene, but now the acoustic cue is motivated by To’s visual action—a medium shot provides a knees-down framing of two officers descending steps to the basement arcade. (The image of uniformed figures in lockstep carries quasi-Nazi connotation, hinting at the PTU officers’ herd-based morality.) Still later, an officer pointedly removes one boot before brutally kicking a helpless informant. And when a female gang member receives a kick to the chest, To wryly imprints a boot stain on her shirt. Again, To and Chappell launch an aesthetic of acoustic foreshadowing, braiding a sonic and visual motif throughout the film’s structure. The overall strategy is not a play with the spectator’s knowledge, as in the preceding examples, but rather a kind of formal robustness, the sound track launching what will emerge as a chiefly visual story motif.