8: Chapter Making Sounds by Combining Pieces
1
2
We know that natural sounds are often the combination
of multiple sounds.
Adding waves in physics or math is hard.
In computer science, its easy! Simply add the samples
at the same index in the two waves:
r srcSample in range(0, getLength(source)):
destValue=getSampleValueAt(dest, srcSample)
srcValue=getSampleValueAt(source, srcSample)
setSampleValueAt(source, srcSample, srcValue+destValue)
3
a
The first two are sine waves.
The third is just the sum of the
first two columns.
b
a+b=c
4
Uses for adding sounds
We can mix sounds
We even know how to change the volumes of the two
sounds, even over time (e.g., fading in or fading out)
We can create echoes
We can add sine (or other) waves together to create kinds
of instruments/sounds that do not physically exist, but
which sound interesting and complex
5
def addSoundInto(sound1, sound2):
for sampleNmr in range(0, getLength(sound1)):
sample1 = getSampleValueAt(sound1, sampleNmr)
sample2 = getSampleValueAt(sound2, sampleNmr)
setSampleValueAt(sound2, sampleNmr, sample1 +
sample2)
Notice that this adds sound1 and sound2 by
adding sound1 into sound2
6
>>> c4=makeSound(getMediaPath("bassoon-c4.wav"))
>>> e4=makeSound(getMediaPath("bassoon-e4.wav"))
>>> g4=makeSound(getMediaPath("bassoon-g4.wav"))
>>> addSoundInto(e4,c4)
>>> play(c4)
>>> addSoundInto(g4,c4)
>>> play(c4)
7
def makeChord(sound1, sound2, sound3):
for index in range(0, getLength(sound1)):
s1Sample = getSampleValueAt(sound1, index)
setSampleValueAt(sound1, index, s1Sample )
if index > 1000:
s2Sample = getSampleValueAt(sound2, index - 1000)
setSampleValueAt(sound1, index, s1Sample +
s2Sample)
if index > 2000:
s3Sample = getSampleValueAt(sound3, index - 2000)
setSampleValueAt(sound1, index, s1Sample + s2Sample
+ s3Sample)
-Add in sound2 after 1000 samples
-Add in sound3 after 2000 samples
Note that in this
version were
adding into
sound1!
8
def echo(sndFile, delay):
s1 = makeSound(sndFile)
s2 = makeSound(sndFile)
for index in range(delay, getLength(s1)):
echo = 0.6*getSampleValueAt(s2, index-delay)
combo = getSampleValueAt(s1, index) + echo
setSampleValueAt(s1, index, combo)
play(s1)
return s1
This creates a delayed echo sound, multiplies it by 0.6 to
make it fainter and then adds it into the original sound.
9
def echoes(sndFile, delay, num):
s1 = makeSound(sndFile)
ends1 = getLength(s1)
ends2 = ends1 + (delay * num)
# convert samples
secs = int(ends2/getSamplingRate(s1)) # to secs
s2 = makeEmptySound(secs + 1)
amp = 1.0
# make amplitude smaller for each
echo
for echoCount in range(0, num):
amp = amp * 0.6
# amplitude 60% smaller each
time
for posns1 in range(0, ends1):
posns2 = posns1 + (delay*echoCount)
val1 = getSampleValueAt(s1, posns1)*amp
val2 = getSampleValueAt(s2, posns2)
10
setSampleValueAt(s2, posms2, val1 + val2)
How sampling keyboards
work
They have a huge memory with recordings of lots of
different instruments played at different notes
When you press a key on the keyboard, the recording
closest to the note you just pressed is selected, and then
the recording is shifted to exactly the note you requested.
The shifting is a generalization of the half/double
functions we saw earlier.
11
Why +1 here?
def double(source):
len = getLength(source) / 2 + 1
target = makeEmptySound(len)
targetIndex = 0
for sourceIndex in range(0, getLength( source), 2):
value = getSampleValueAt( source, sourceIndex)
setSampleValueAt( target, targetIndex, value)
targetIndex = targetIndex + 1
play(target)
return target
Heres the
piece that
does the
doubling
12
This is how a sampling
synthesizer works!
def half(source):
target = makeEmptySound(getLength(source) * 2)
sourceIndex = 0
for targetIndex in range(0, getLength( target)):
value = getSampleValueAt( source, int(sourceIndex))
setSampleValueAt( target, targetIndex, value)
sourceIndex = sourceIndex + 0.5
Heres the
play(target)
piece that
return target
does the
halving
13
Can we generalize shifting a
sound into other frequencies?
This way does NOT work:
def shift(source, factor):
target = makeEmptySound(getLength(source))
sourceIndex = 0
for targetIndex in range(0, getLength( target)):
value = getSampleValueAt( source, int(sourceIndex))
setSampleValueAt( target, targetIndex, value)
sourceIndex = sourceIndex + factor
play(target)
return target
14
Itll work for shifting down, but not shifting up. Why?
>>> hello=pickAFile()
>>> print hello
/Users/guzdial/mediasources/hello.wav
>>> lowerhello=shift(hello,0.75)
>>> higherhello=shift(hello,1.5)
I wasn't able to do what you wanted.
The error java.lang.ArrayIndexOutOfBoundsException has occured
Please check line 7 of /Users/guzdial/shift-broken.py
15
def shift(source, factor):
target = makeEmptySound(getLength(source))
sourceIndex = 0
for targetIndex in range(0, getLength( target)):
value = getSampleValueAt( source, int(sourceIndex))
setSampleValueAt( target, targetIndex, value)
sourceIndex = sourceIndex + factor
if sourceIndex > getLength(source):
sourceIndex = 0
play(target)
return target
16
For a desired frequency f we want a sampling interval
like this:
17
How the original sound
synthesizers worked
What if we added pure sine waves?
We can generate a sound that is just a single tone (see the book)
We can then add them together (perhaps manipulating their
volume) to create sounds that dont exist in nature
Dont have to use just sine waves
Waves that are square or triangular (seriously!) can be heard and
have interesting dynamics
We can add together waves of lots of types to create unique
sounds that cant be created by physical instruments
We call this additive synthesis
Additive synthesis as-is isnt used much anymore
18
Sampling as an Algorithm
Think about the similarities between:
Halving the sounds frequency and
scaling a picture larger.
Doubling the sounds frequency and
scaling a picture smaller.
19
def half(filename):
source = makeSound(filename)
target = makeSound(filename)
sourceIndex = 1
for targetIndex in range(1, getLength( target)+1):
setSampleValueAt( target, targetIndex,
getSampleValueAt( source,
int(sourceIndex)))
sourceIndex = sourceIndex + 0.5
play(target)
return target
def copyBarbsFaceLarger():
# Set up the source and target pictures
barbf=getMediaPath("barbara.jpg")
barb = makePicture(barbf)
canvasf = getMediaPath("7inX95in.jpg")
canvas = makePicture(canvasf)
# Now, do the actual copying
sourceX = 45
for targetX in range(100,100+((200-45)*2)):
sourceY = 25
for targetY in range(100,100+((200-25)*2)):
color = getColor(
getPixel(barb,int(sourceX),int(sourceY)))
setColor(getPixel(canvas,targetX,targetY), color)
sourceY = + sourceY 0.5
sourceX = sourceX + 0.5
show(barb)
show(canvas)
return canvas
20
Our programs (functions)
implement algorithms
Algorithms are descriptions of behavior for solving a
problem.
A program (our Python function) is an executable
interpretations of algorithms.
The same algorithm can be implemented in many
different languages.
The same algorithm can be applied to many different data
sets with similar results.
21
Both of these functions implement a sampling
algorithm
Both of them do very similar things:
Get an index to a source
Get an index to a target
For all the elements that we want to process:
Copy an element from the source at the integer value of the
source index
to the target at the target index
This is a
Increment the source index by 1/2
description of the
Return the target when completed
algorithm.
22
Adding sine waves to make
something completely new
We saw earlier that complex sounds (like
the sound of your voice or a trumpet) can
be seen as being a sum of sine waves.
We can create complex sounds by
summing sine waves.
These are sounds made by mathematics,
by invention, not based on anything in
nature.
23
Basic idea: Build a sine
wave
If we want a 440 Hz
sound wave, then we
need one of these cycles
every 1/440th of a
second.
We need to break this
wave into the number of
pieces in our sampling
rate.
24
25
Our Code
def sineWave(freq ,amplitude ):
# Get a blank sound
mySound = getMediaPath(sec1silence.wav)
buildSin = makeSound(mySound)
# Set sound constant
sr = getSamplingRate(buildSin) # sampling rate
interval = 1.0/ freq # Make sure its floating point
samplesPerCycle = interval * sr # samples per cycle
maxCycle = 2 * pi
for pos in range (0, getLength(buildSin )):
rawSample = sin((pos / samplesPerCycle) * maxCycle)
sampleVal = int(amplitude*rawSample)
setSampleValueAt(buildSin ,pos ,sampleVal)
return buildSin
26
Adding pure sine waves
together
>>> f440=sineWave (440 ,2000)
>>> f880=sineWave (880 ,4000)
>>> f1320=sineWave (1320 ,8000)
>>> addSounds(f880 ,f440)
>>> addSounds(f1320 ,f440)
>>> play(f440)
>>> explore(f440)
>>> just440=sineWave (440 ,2000)
>>> play(just440)
>>> explore(f440)
Adding together
440Hz, 880Hz,
and 1320Hz,
with increasing
amplitudes.
Comparing to a
440Hz wave
27
Comparing the waves
Left, 440 Hz; Right, combined wave.
In Explorer
In the Spectrum view in
MediaTools
440, 880, 1320
28
Making more complicated
waves
Using square waves,
# use float since interval is fl point
samplesPerCycle = interval * samplingRate
instead of sine waves,
# we need to switch every half-cycle
can be a richer sound
samplesPerHalfCycle = int(samplesPerCycle / 2)
sampleVal = amplitude
def squareWave(freq,amplitude):
s =1
# Get a blank sound
i=1
mySound = getMediaPath("sec1silence.wav") for s in range (0, getLength(square)):
square = makeSound(mySound)
# if end of a half-cycle
# Set music constants
if (i > samplesPerHalfCycle):
samplingRate = getSamplingRate(square)
# reverse the amplitude every half-cycle
seconds = 1 # play for 1 second
sampleVal = sampleVal * -1
# Build tools for this wave
# and reinitialize the half-cycle counter
# seconds per cycle
i=0
interval = 1.0 * seconds / freq
setSampleValueAt(square,s,sampleVal)
i=i+1
return(square)
29
Building sounds with square
waves
>>> sq440=squareWave(440,4000)
>>> play(sq440)
>>> sq880=squareWave(880,8000)
>>> sq1320=squareWave(1320,10000)
>>> writeSoundTo(sq440,"square440.wav")
>>> addSounds(sq880,sq440)
>>> addSounds(sq1320,sq440)
>>> play(sq440)
>>> writeSoundTo(sq440,"squarecombined440.wav")
Basic square wave
Summed square wave
30
Sound synthesis techniques
Adding sine and square (and triangle) waves is additive
sound synthesis.
Most common modern synthesis technique is frequency
modulation (FM) synthesis.
Much richer sound.
Just about any way you can imagine to fill a sound
mathematically can lead to an interesting synthesis
technique.
Create random noise, then filter parts out: Subtractive
synthesis
31
Adding envelopes
Most real synthesizers today also allow you to manipulate
envelopes
An envelope is a definition of how quickly the aspects of
the sound change over time
For example, the rise in volume (attack), how the volume is
sustained over time (sustain), how quickly the sound decays
(decay): The ASD envelope
Pianos tend to attack quickly, then decay quickly (without
pedals)
Flutes tend to attack slowly and sustain as long as you
want.
32
Why write sound programs?
Arent there audio tools that can do many of these
things?
Sure, and thats good enoughif thats good enough.
If you just want to use a sound, then simply using tools to
generate the noise/instrument/sound you want is fine.
33
Communicating process
What if you want to tell someone else how you got that
sound, so that they can replicate the process, or even
modify the sound in some way, or make it better?
You could write down all the steps in a sound application
tool.
Tedious, error prone.
Or you could provide a program.
A succinct, executable definition of a process.
34
What is MP3?
MP3 files are files encoded according to the MPEG-3
standard.
They are audio files, but they are compressed in special ways.
They use a model of how we hear to get rid of some of the sound.
If there is a soft sound at the same time as a loud sound, dont record the
soft sound
They use various compression techniques to make the sound
smaller.
WAV files are compressed, but not as much, and dont use any
smart models to make themselves smaller.
35
What is MIDI?
MIDI is a standard for encoding music, not sound.
MIDI literally encodes For this instrument (track), turn key
#42 on then later For this instrument (track), turn key #31
off.
The quality of the actual sound depends entirely on the
synthesizerthe quality of the instrument generation
(whether recorded or synthesized).
MIDI files tend to be very, very small.
Each MIDI instruction (Play key #42 track 7) is only
about five bytes long.
Not thousands of bytes long.
36
Playing MIDI in JES
The function playNote allows you to play MIDI piano
with JES.
playNote takes three inputs:
A note number
Not a frequencyits literally the piano key number
C in the first octave is 1, C# is 2, C in the fourth octave is 60, D in
the fourth octave is 62.
A duration in milliseconds (1/1000 of a second)
An intensity (0-127)
Literally, how hard the key is pressed
37
MIDI Example
def song():
playNote(60,200,127)
playNote(62,500,127)
playNote(64,800,127)
playNote(60,600,127)
for i in range(1,2):
playNote(64,120,127)
playNote(65,120,127)
playNote(67,60,127)
38