Speech is converted into text, then into a British English phonetic transcription.
The transcription is split into five musical tracks: vowels, plosives, fricatives,
nasals, and liquids/glides.
Vowel pitch is guided by intrinsic F0 research: close vowels, often called high
vowels, tend to have a higher average fundamental frequency than open, or low,
vowels. F0 is the vibration rate heard as pitch; F1, F2, and F3 are formants, or
resonant bands that shape vowel colour. The mapping also follows perceived
brightness: front vowels such as /i/ have higher F2 than
back vowels such as /u/, so a higher pitch is assigned to
/i/ and other front vowels.
- Vowels
- sustained voice sounds such as i æ u; electric piano melody
- Diphthongs
- moving vowels such as aɪ eə ʊə; grace-note motion
- Plosives
- stop consonants such as p t k b d ɡ; drum attacks
- Fricatives
- noisy consonants such as s ʃ h; brush, hiss, breath
- Nasals
- sounds such as m n ŋ; minor-jazz cadence ending on Fm9
- Liquids / glides
- smooth consonants such as l ɹ j w; muted organ gestures
The nasal track controls harmony. One nasal gives an Fm9 pedal; two create
C7b9–Fm9; three create Gm7b5–C7b9–Fm9; four or more cycle through
Dbmaj7–Gm7b5–C7b9–Fm9.
Vowels define the main time span. The other phoneme tracks are distributed across
that span like a small DAW session.