Building a Musical Instrument with the Web Audio API

It's been a while since I've written anything due to some personal concerns that I might write about later, but don't worry, I'm still around and I'm still coding. Recently, I went to Texas and bought a three-row diatonic button accordion. Diatonic accordions are popular for a lot of different types of folk music, which is generally learned by ear. This is good for me, because I don't really know how to read music anyway.

The accordion has 34 buttons on the treble side and 12 buttons on the bass side. Unlike a piano accordion, which has the same logical, chromatic layout as a piano, the diatonic accordion just has a bunch of buttons and I didn't really know where to start. Also, every note is different whether you're pulling the bellows out or pushing them in, so there are actually 68 notes on the treble side (albeit some are repeated). Also, as I'm sure you might be aware, accordions are loud. Very loud. In order to not piss off my neighbors too much, and to learn how the layout of this box works, I decided to make a little web app.

That web app is KeyboardAccordion.com, which like everything else I create in my free time is open source. I noticed that there are just enough keys on a computer keyboard to correspond to the accordion layout, and they're arranged in a similar pattern. With this, I can keep track of the notes, scales, and chords and start figuring out how to put it all together.

Here's what one of the accordions looks like:

I decided to make this app in Svelte, because I've used React and Vue professionally but have no experience with Svelte whatsoever and wanted to know what everyone loves about it.

Web Audio API

KeyboardAccordion.com only has one dependency, and that's Svelte. Everything else is done using plain JavaScript and the built-in browser Web Audio API. I'd never really used the Web Audio API before, so I figured out what I needed to to get this working.

The first thing I did was create an AudioContext and attach a GainNode, which controls the volume.

const audio = new (window.AudioContext || window.webkitAudioContext)()
const gainNode = audio.createGain()
gainNode.gain.value = 0.1
gainNode.connect(audio.destination)

As I was figuring everything out, I was experimenting with making new AudioContext for every note because I was trying to fade out the sound, but then I kept realizing that after 50 notes, the app would stop working. Fifty is apparently the limit for the browser, so it's better to just make one AudioContext for the entire app.

I'm using waves with the Audio API and not using any sort of audio sample, and I used the OscillatorNode to make each note. There are various types of waves you can use - square, triangle, sine, or sawtooth, which all have a different type of sound. I went with the sawtooth for this app because it worked out the best. Square makes an extremely loud, chiptune-esque sound like an NES which is kind of nice in its own way. Sine and triangle were a bit more subdued but if you don't fade the sound out properly, it makes a really unpleasant kind of cutting sound due to how your ear reacts when a wave gets cut off.

Waveforms

So for each note, I'd make an oscillator, set the wave type, set the frequency, and start it. Here's an example using 440, which is a standard tuning for "A".

const oscillator = audio.createOscillator()
oscillator.type = 'sawtooth'
oscillator.connect(gainNode)
oscillator.frequency.value = 440
oscillator.start()

If you do that, the note will just play until infinity, so you have to make sure you stop the oscillator when you want the note to end.

oscillator.stop()

For me, this meant event listeners on the DOM that would listen for a keypress event to see if any button was pressed, and a keyup event to determine when any button was no longer being pressed. In Svelte, that's handled by putting event listeners on svelte:body.

<svelte:body
  on:keypress="{handleKeyPressNote}"
  on:keyup="{handleKeyUpNote}"
  on:mouseup="{handleClearAllNotes}"
/>

So that's really everything there is to the Web Audio API itself when it comes to setting up the app - creating an AudioContext, adding a Gain, and starting/stopping an Oscillator for each note.

You could paste this into the console and it'll play a note. You'll have to either refresh or type oscillator.stop() to make it stop.

const audio = new (window.AudioContext || window.webkitAudioContext)()
const gainNode = audio.createGain()
gainNode.gain.value = 0.1
gainNode.connect(audio.destination)

const oscillator = audio.createOscillator()
oscillator.type = 'sawtooth'
oscillator.connect(gainNode)
oscillator.frequency.value = 440
oscillator.start()

Data Structure

I had to figure out how I wanted to lay out the data structure for this application. First of all, if I'm going to be using the Web Audio API with frequencies directly, I had to collect all of them.

Frequencies

Here's a nice map of notes to frequencies with all 12 notes and 8-9 octaves for each note, so I can use A[4] to get the 440 frequency.

export const tone = {
  C: [16.35, 32.7, 65.41, 130.81, 261.63, 523.25, 1046.5, 2093.0, 4186.01],
  Db: [17.32, 34.65, 69.3, 138.59, 277.18, 554.37, 1108.73, 2217.46, 4434.92],
  D: [18.35, 36.71, 73.42, 146.83, 293.66, 587.33, 1174.66, 2349.32, 4698.64],
  Eb: [19.45, 38.89, 77.78, 155.56, 311.13, 622.25, 1244.51, 2489.02, 4978.03],
  E: [20.6, 41.2, 82.41, 164.81, 329.63, 659.26, 1318.51, 2637.02],
  F: [21.83, 43.65, 87.31, 174.61, 349.23, 698.46, 1396.91, 2793.83],
  Gb: [23.12, 46.25, 92.5, 185.0, 369.99, 739.99, 1479.98, 2959.96],
  G: [24.5, 49.0, 98.0, 196.0, 392.0, 783.99, 1567.98, 3135.96],
  Ab: [25.96, 51.91, 103.83, 207.65, 415.3, 830.61, 1661.22, 3322.44],
  A: [27.5, 55.0, 110.0, 220.0, 440.0, 880.0, 1760.0, 3520.0],
  Bb: [29.14, 58.27, 116.54, 233.08, 466.16, 932.33, 1864.66, 3729.31],
  B: [30.87, 61.74, 123.47, 246.94, 493.88, 987.77, 1975.53, 3951.07],
}

Button layout

Figuring out exactly how to arrange all the buttons into a data stucture took a couple of tries for me. The data that had to be captured was:

• The row on the accordion
• The column on the accordion
• The direction of the bellows (push or pull)
• The name and frequency of the note at that row, column, and direction

This means that there are different combinations for all three sets of these things. I decided to make an id that corresponds to each possible combination, such as 1-1-pull being row 1, column 1, direction pull.

This way, I could create an array that holds the data for any note that is currently being played. If you press the button to reverse the bellows, it would take all the currently playing notes and reverse them, thus changing 1-1-pull and 1-2-pull to 1-1-push and 1-2-push.

So ultimately I had an object that contained the data for all three treble rows like so:

const layout = {
  one: [],
  two: [],
  three: [],
}

My particular accordion is tuned to FB♭Eb, meaning the first row is tuned to F, the second row is tuned to B♭, and the third row is tuned to E♭. The example for the first row looks like this:

const layout = {
  one: [
    // Pull
    { id: '1-1-pull', name: 'D♭', frequency: tone.Db[4] },
    { id: '1-2-pull', name: 'G', frequency: tone.G[3] },
    { id: '1-3-pull', name: 'B♭', frequency: tone.Bb[3] },
    { id: '1-4-pull', name: 'D', frequency: tone.D[4] },
    { id: '1-5-pull', name: 'E', frequency: tone.E[4] },
    { id: '1-6-pull', name: 'G', frequency: tone.G[4] },
    { id: '1-7-pull', name: 'B♭', frequency: tone.Bb[4] },
    { id: '1-8-pull', name: 'D', frequency: tone.D[5] },
    { id: '1-9-pull', name: 'E', frequency: tone.E[5] },
    { id: '1-10-pull', name: 'G', frequency: tone.G[5] },
    // Push
    { id: '1-1-push', name: 'B', frequency: tone.B[3] },
    { id: '1-2-push', name: 'F', frequency: tone.F[3] },
    { id: '1-3-push', name: 'A', frequency: tone.A[3] },
    { id: '1-4-push', name: 'C', frequency: tone.C[4] },
    { id: '1-5-push', name: 'F', frequency: tone.F[4] },
    { id: '1-6-push', name: 'A', frequency: tone.A[4] },
    { id: '1-7-push', name: 'C', frequency: tone.C[5] },
    { id: '1-8-push', name: 'F', frequency: tone.F[5] },
    { id: '1-9-push', name: 'A', frequency: tone.A[5] },
    { id: '1-10-push', name: 'C', frequency: tone.C[6] },
  ],
  two: [
    // ...etc
  ],
}

There are notes 1 through 10 in row one, and each one has a name and frequency associated with it. Repeating this for two and three, I now have all 68 notes on the treble side.

Keyboard layout

Now I had to map each key on the keyboard to a row and column of the accordion. Direction doesn't matter here, since z will correspond to both 01-01-push and 01-01-pull.

export const keyMap = {
  z: { row: 1, column: 1 },
  x: { row: 1, column: 2 },
  c: { row: 1, column: 3 },
  v: { row: 1, column: 4 },
  b: { row: 1, column: 5 },
  n: { row: 1, column: 6 },
  m: { row: 1, column: 7 },
  ',': { row: 1, column: 8 },
  '.': { row: 1, column: 9 },
  '/': { row: 1, column: 10 },
  a: { row: 2, column: 1 },
  s: { row: 2, column: 2 },
  d: { row: 2, column: 3 },
  f: { row: 2, column: 4 },
  g: { row: 2, column: 5 },
  // ...etc
}

Now I have all the keys from z to /, a to ', and w to [ mapped out. Very auspicious that the computer keyboard and accordion keyboard are so similar.

Pressing keys, playing notes

As you might recall, I have an event listener on the entire page listening for the key press event. Any key press event that happens will go through this function.

First, it has to check both lowercase and uppercase keys in case shift or caps lock are pressed, otherwise the keys won't work at all. Then, if you're pressing the button to toggle the bellows (which I made q), it has to handle that separately. Otherwise, it will check the keyMap, and if one exists, it will find the corresponding id by checking the current direction and getting the row and column from the keymap.

let activeButtonIdMap = {}

function handleKeyPressNote(e) {
  const key = `${e.key}`.toLowerCase() || e.key // handle caps lock

  if (key === toggleBellows) {
    handleToggleBellows('push')
    return
  }

  const buttonMapData = keyMap[key]

  if (buttonMapData) {
    const { row, column } = buttonMapData
    const id = `${row}-${column}-${direction}`

    if (!activeButtonIdMap[id]) {
      const { oscillator } = playTone(id)

      activeButtonIdMap[id] = { oscillator, ...buttonIdMap[id] }
    }
  }
}

The way I'm tracking each currently playing note is putting them in the activeButtonIdMap object. In Svelte, in order to update a variable you just reassign it, so instead of what you might do in React with useState:

const [activeButtonIdMap, setActiveButtonIdMap] = useState({})

const App = () => {
  function handleKeyPressNote() {
    setActiveButtonIdMap(newButtonIdMap)
  }
}

You have to declare it as a let and reassign it:

let activeButtonIdMap = {}

function handleKeyPressNote() {
  activeButtonIdMap = newButtonIdMap
}

This was mostly easier, except when all I wanted to do was delete a key from the object. As far as I could tell, Svelte only rerenders when a variable is reassigned, so just mutating some value within wasn't enough and I had to clone it, mutate it, the reassign it. This is what I did in the handleKeyUpNote function.

function handleKeyUpNote(e) {
  const key = `${e.key}`.toLowerCase() || e.key

  if (key === toggleBellows) {
    handleToggleBellows('pull')
    return
  }

  const buttonMapData = keyMap[key]

  if (buttonMapData) {
    const { row, column } = buttonMapData
    const id = `${row}-${column}-${direction}`

    if (activeButtonIdMap[id]) {
      const { oscillator } = activeButtonIdMap[id]
      oscillator.stop()
      // Must be reassigned in Svelte
      const newActiveButtonIdMap = { ...activeButtonIdMap }
      delete newActiveButtonIdMap[id]
      activeButtonIdMap = newActiveButtonIdMap
    }
  }
}

Maybe someone knows a better way to delete an item from an object in Svelte, but this is the best I could come up with.

I also made a few functions that will play through the scales, starting with F, B♭ and E♭ being the main diatonic keys of the accordion, but there are more options. To play the scales, I simply looped through all the ids that correspond to the notes in the scale and used a JavaScript "sleep" command of 600ms between each note.

Rendering

Now that I have all the data structures set up and the JavaScript, I just need to render all the buttons. Svelte has #each blocks for looping logic, so I just looped through the three rows of buttons and rendered a circle for each button.

<div class="accordion-layout">
  {#each rows as row}
    <div class="row {row}">
      {#each layout[row].filter(({ id }) => id.includes(direction)) as button}
        <div
          class={`circle ${activeButtonIdMap[button.id] ? 'active' : ''} ${direction} `}
          id={button.id}
          on:mousedown={handleClickNote(button.id)}
        >
          {button.name}
        </div>
      {/each}
    </div>
  {/each}
</div>

Each circle has its own mousedown event so you can click on them in addition to using the keyboard, but I didn't put the mouseup event on the circle itself. This is because if you move your mouse somewhere else before lifting it up, it won't correctly determine the mouseup and the note will play forever.

And of course, I just used plain CSS because I don't usually feel like anything fancier is necessary for small projects.

.circle {
  display: flex;
  align-items: center;
  justify-content: center;
  border-radius: 50%;
  height: 60px;
  width: 60px;
  margin-bottom: 10px;
  background: linear-gradient(to bottom, white, #e7e7e7);
  box-shadow: 0px 6px rgba(255, 255, 255, 0.4);
  color: #222;
  font-weight: 600;
  cursor: pointer;
}

.circle:hover {
  background: white;
  box-shadow: 0px 6px rgba(255, 255, 255, 0.3);
  cursor: pointer;
}

.circle.pull:active,
.circle.pull.active {
  background: linear-gradient(to bottom, var(--green), #56ea7b);
  box-shadow: 0px 6px rgba(255, 255, 255, 0.2);
}

.circle.push:active,
.circle.push.active {
  background: linear-gradient(to bottom, var(--red), #f15050);
  box-shadow: 0px 6px rgba(255, 255, 255, 0.2);
  color: white;
}

Conclusion

I hope you liked my write-up for the Keyboard Accordion app! Of course, the full code is available on GitHub.

There are a few little bugs here and there, such as if you use keyboard shortcuts while also pressing other keys, it will get stuck on a note forever. I'm sure if you try to find more bugs you'll be able to.

This app was fun to make, I learned how to use both Svelte and the Web Audio API, and it's helping me and hopefully some other afficionados to understand the squeezebox a little better. Maybe it'll inspire you to build your own little online instrument, or make an app for one of your hobbies. The best part about coding is that you can make anything you want!