Kawai K1: I dumped the K1m Wave ROM!

I was not 100% happy with my waveform recordings, I outlined it briefly in one of my previous articles, where I explained how I recorded them.

The recordings still had the issue of being quite large in size, and more importantly, they contained noise. It’s not just simple noise from some amplifiers (that one is also there, but very low). The K1 resamples the waveforms pretty badly and the aliasing that this produces is clearly audible.

Don’t get me wrong, of course, I want to recreate the aliasing and the dirtyness, as it is part of the K1s character, but it does not help if it is already part of the waveforms, which should be as pure as possible as they are pitched during playback. The aliasing needs to be added at a later stage in the audio generation process. Details about it can be found here.

Furthermore, working with recordings from a device introduces a lot of errors in general: The data has to go through The K1s D/A converters & amplifiers, the cables that go from the K1 to the recording interface and the A/D converters of recording interface itself all introduce small unwanted modifications to the signal

K1 Wave ROM Chip Data

I was browsing the K1 service manual, my intention was to repair my broken K1m, which still suffers from broken envelope attacks, when I noticed the pinout of a chip that had the letters WAVE ROM behind it.

K1 Wave ROM pinout, taken from the service manual

If you have read some of my other articles, you might know that I’m a bit into Home Automation things. For this, I not only buy ready-to-use devices, but also create my own, using Arduinos or similar microcomputers. Given that the pinout of this chip is pretty straightforward, just some address lines and some data lines, I wanted to give it a try.

The service manual lists the used chip as being a HN62304BP. I’ve searched the internet and found a data sheet. I verified that the pinout was identical to the Kawai layout.

HN62304BP data sheet

More information that I’ve taken from it: Its size is 4Mbit, or 512KiB and the operating voltage is 5V.

5 Volts perfectly fits an Arduino, but having 19 address lines and 8 data lines, my Arduinos and NodeMCU devices that I have laying around lack plenty of GPIOs for this job. Therefore, I ordered an Arduino Mega 2560 R3 which has a lot of them, 54 digital GPIOs in total.

Preparing the Arduino Mega

I wrote the Arduino code upfront before my order arrived. I used a Google Spreadsheet to help deciding which pins of the Arduino I use to keep the wiring chaos at a minimum. Then, I created some mapping tables in the Arduino Code.

Google Spreadsheet

After it arrived, I wired a breadboard to the prototyping shield that came with the Arduino according to the specifications of the ROM chip. After having everything wired up, I was quite happy that it didn’t look too chaotic, definitely maintainable.

A prototyping shield sitting on a Arduino Mega 2560 R3, connected to a breadboard, ready to hold the K1 Wave ROM chip

I did a dry run to verify that the code works fine, fixed some issues here and there and then started to desolder the K1m Wave ROM.

Desoldering the Wave ROM

Given that the chip is not socketed , I had some work to do 🤔 I didn’t want to risk ruining the K1 at all costs so I had to be extra careful! The plan was to put the chip back into place after having finished reading it.

I started by adding fresh solder to the pins and furthermore connected each pair of two pins together. This was a preparation for the next step, I heated the new joints up again and used a pump to remove it. The result was pretty good already, some pins required some manual extra work but after a while I was able to remove the chip.

A work-in-progress image
Here we go, the K1m wave rom chip
K1m Wave ROM chip on a breadboard

First look & listen at the ROM data

After the dump had finished, I had a first look at it, mainly to verify that my code to dump the ROM was not faulty, I didn’t swap any address lines and such.

I have not been able to find any text or something that would tell me if the dump was correct or not. I verified that all 8 bits are not always zero and not always one, but I couldn’t guess if the data is correct just by looking at it in a hex editor. As it is a Wave ROM, assuming it might contain mostly audio data, I converted the text dump into a binary and loaded it in a wave editor.

When I saw and heared it I was so excited! 😍 Although something was obviously wrong, at the same time, it verified two important of things:

  • I didn’t swap any address lines or data lines
  • The ROM had no encryption that I would’ve had to fight against
Wave ROM loaded as PCM data

Though I recognized some of the used waveforms, they appeared to be heavily distorted. I imported them both as signed PCM and unsigned PCM data, but that didn’t help.

I browsed a bit in the ROM and found the sine single cycle waveform.

Broken Sine wave form

Apparently all negative waveform values were inverted. I added a small code snipped to the wave ROM text-to-binary converter to fix it:

After I did that and loaded the wave ROM again I got goosebumps! 👍All waveforms were present, in a quality that is, obviously, much better than anything I had before.

K1m Wave ROM

Wave ROM data analysis

Having the data as audio data, I was able to verify that there is only audio data in it, there is no meta information, such as offsets, any kind of init pattern, whatever. The whole ROM consists of audio data only.

PCM data

I began analyzing the file by adding markers to the waveforms. What I got quite quickly is, that every waveform has a length that is a power of two. The lengths are different, ranging from 4096 samples (some of the drums) to 32768 samples (choir, strings etc).

Knowing that every length is a power of two helped a lot to set the markers at their correct positions, resulting in perfect loop points.

What I also noticed: There are only 30 PCM waveforms in it, although there are 52 PCM waveforms that you can select. This is because the waveforms are used multiple times. For example, there is a one-shot Voice (Wave 233) and a Voice Loop (Wave 238). They are based on the same waveform. This is the same for all loops.

The reversed waveforms that you can select are not present, the K1 generates them in realtime by just playing the waveform backwards.

All „Alt“ loops are not present either. The K1 plays them as forward-backward-forward-… loops.

All „Omnibus“ loops just play through a larger range of the wave ROM. If you listen to the audio above, you can even hear all of them. For example, Omnibus Loop 8 plays a loop consisting of 6 waveforms: F. Guitar, A. Guitar, Piano Noise, Pan Flute Attack, String Attack and Bowed String. Its easy for the K1 because that is the layout in the Wave ROM. The picture below illustrates it.

Omnibus Loop 8 in the Wave ROM

On the left side of the picture, you can see how I named my markers. I use these markers to map segments of the Wave ROM to waveforms 1-256 when I load the Wave ROM in the VSTi plugin.

Single Cycle Waves

The single cycle waves are stored in the Wave ROM, too, but in a much different way.

Single Cycle Waveforms in the Wave ROM

It took me a while to understand what I am looking at. Initially, I thought that my conversion code is broken and that there is still a signed/unsigned/something mismatch.

One of these cycles is always 128 samples long, something that I’ve read previously somewhere. And each cycle seems to exist 5 times. Why? And these snippets are definitely not loopable.

What helped me to understand this is to compare the wave ROM data against my recordings:

Recording of wave 14 (First Saw)
Wave ROM data of wave 14 (First Saw)

What the K1 does is, it uses the cycle twice. To form a loop of one waveform, it uses it once in forward mode, the second part is the same cycle, but inverted and negated. If I do this manually in the wave editor, the result looks like this and is a perfect match to the recording:

So what is stored in the ROM are half-cycles only.

The next topic was to figure out why there are always 5 repeats for each waveform. I quickly verified that this is always the case for all single cycle waveforms by calculating the following:

  • The total data length for all single cycle waveforms in the Wave ROM is 130560 samples
  • Divided by 5 (number of repeats), the result is 26112
  • Divided by 128 (length of one cycle), the result is 204. This is the value that we expect as the K1 contains 204 single cycle waveforms

In one of my previous articles, I talked about that some of the single cycle waves are multisamples, and this is what the 5 variations of each waveform are for.

Now the only problem was to find out when to use which multisample. I had another look at my multisample markers that I’ve put into my recordings and noticed that, if there is a multisample transition, they always happen at the same notes. Something that I just didn’t notice. After looking at each of my recordings and writing down the notes, I had a list of 5 different notes that were used, exactly what I was looking for. The final mapping for multisamples is:

MultisampleMidi Note Number Range
10-47
248 – 59
360 – 71
472 – 83
584 – 127

You may have noticed that the range of one multisample is one octave and that there is support for 5 octaves in total. The reason why I didn’t recognize all multisamples while recording is, that not every waveform uses them. For many waveforms, the data in each multisample is identical.

Conclusion

After adding support to load the Wave ROM in my emulation, the quality increased significantly! 😎
Some of the not-really-good sounding singles now perfectly match the real K1. Especially bell-type sounds that use the amplitude modulation were missing the very low and very high frequencies, now they are as present as on a real K1. I’m going to post new audio demos soon so you can hear the improvements.

One final step is missing: I still have to put the Wave ROM chip back into my K1m. Instead of adding it directly, I’m going to add a socket to the K1m board and insert the Wave ROM there. The socket didn’t arrive yet so I have to wait some days before doing so.

Update: The socket has arrived and the K1 is fine again.

Kawai K1: Aliasing and K1s native sampling rate

The K1 is full of aliasing, which is obvious when taking a look at the waveform recordings (see previous article).

Nowadays, it would be pretty straightforward to eliminate the aliasing by doing bandlimited interpolation of the waveforms. But…

What we have to be aware of is that a lot of singles actively make use of aliasing to produce their unique sound. So instead of preventing aliasing, the goal is to recreate the aliasing behavior of the original K1.

What is Aliasing?

To do that, firstly, we have to know what happens when aliasing occurs. Every digital audio device works at a specific sampling rate. For example, a CD player works at a sampling rate of 44100 Hz, or 44,1 KHz. The nyquist theorem states that, given a sampling rate of n, a maximum frequency of n/2 can be represented in an audio stream. In the CD player example, the maximum frequency that can be stored on a CD is 22050 Hz.

If a frequency is generated at runtime, special care needs to be taken to ensure that every frequency is always less than half the sampling rate, aka the nyquist frequency. If the frequency is too high, it starts to mirror at the nyquist frequency, giving unexpected results.

An example: If your audio interface runs at a frequency of 44100 Hz but you try to generate a sine wave of 30000 Hz, it is mirrored at 22050 Hz and begins to fall downwards. Lets do the math:
30000 Hz – 22050 Hz =7950 Hz, so the desired frequency is 7950 Hz too high to be represented correctly.
Mirrored at the nyquist frequency: 22050 Hz – 7950 Hz = 14100 Hz.

So, although we wanted to play a sine wave at 30000 Hz, what we get to hear is a sine wave at 14100 Hz and this is what we call aliasing.

Aliasing on the K1

To see aliasing on the K1, take a look at this image:

Midi notes being played from 0 to 127, aliasing occurs and frequencies are mirrored at nyquist

This is one of the waveform recordings where I played every note from 0 to 127. It starts to alias, but at a frequency that is not known yet.

If you look closer, I added two markers here. The first one is an arbitrary note, the second one is 12 notes away from the first one. I did this because that is exactly one octave higher (note: frequency doubles with each octave) so we can expect that the frequency should be twice as high as the first one, but this is not the case, obviously.

At the first marker, we have a sine wave of roughly 16630 Hz. This should result in the second sine wave being at 33260 Hz. But at the second marker, we observe a sine wave with the frequency of about 16820 Hz instead.

Having both of these values, we can calculate the frequency at which the second sine is mirrored: 33260 Hz – 16820 Hz = 16440 Hz. Half of this value is 8220 Hz and if we add this to the frequency of the first marker the result is 25040 Hz.

The result is our nyquist frequency, which is equal to half sampling rate. If we double this value we get 50080 Hz.

As the FFT frequency analysis is never completely precise and I expect that Kawai has chosen some easy to remember value, I rounded the result which gives us a sampling rate for the K1 of 50000 Hz

Knowing that the sampling rate of the K1 is 50 KHz, I adjusted my emulation to run at the same sampling rate. Furthermore, the waveform playback interpolation has to be kept very basic, for now I have chosen to use very simple linear interpolation. The result is aliasing, identical to the original K1.

A single that makes use of the aliasing is, again, IA-5 Visitors. I added the single to the audio examples, feel free to listen to it here.

Just in case you wonder if that limits the VSTi plugin in any way, don’t worry. It will work on any host sampling rate, I added high quality band limited resampling code that is based on CCRMA/Stanford, as written here: https://ccrma.stanford.edu/~jos/resample/

The K1 emulation engine will always run at 50 KHz and the result will be converted to match the sampling rate of the host.

Kawai K1: Why I recorded all waveforms again

While the waveform pack by Chvad was pretty good to have something to start with, I had some issues, both for the PCM based waveforms and the single cycle waveforms.

PCM waveforms

Chvad sampled the waveforms at note number D4, which sounds good when being played back in a wave player, but the problem it causes is that you lose a lot of the high frequency spectrum when being played on lower midi notes.
Secondly, there is no chance to filter any noise that is emitted by the K1.

Therefore, I decided to sample the PCM waveforms on note D1 instead.

Recording was pretty straightforward, I played the same midi note over and over again and selected the different waveforms inbetween my note pressed. The result looked like this:

PCM waves 205-256 recorded at midi note D1

To make my life a little easier, I wrote a tool to help extracting single files out of this big recording.
What the tool does is, firstly, it detects blocks of silence and audible data. What I use this for is to know when a new waveform starts. This gives a total amount of 52 blocks of audio data, with silence in between them, which matches the PCM waveforms 205-256.
Secondly, as there are waveforms that need to be looped during playback, I created code to detect them. My tool can read wave markers and work on loops according to these markers. There are two types of loops that I need to support:

  • Loops that I define manually with a specified start and end point
  • Loops where I only define a region where a loop is, together with a maximum length. The loop start and end is then detected automatically
A loop that I define manually
A loop that is auto-detected

After I had defined all required markers, the whole file looked like this:

All PCMs with loop markers

After executing my tool, the output was a nice list of loopable PCM waveforms that were ready to be used in my K1 emulation VSTi

Single Cycle Waveforms

What always made me wondering is the fact that the single Visitors appears to have some sine wave sound, whose pitch is modified by the LFO, but when inspecting how the single is made, you’ll notice that there is no sine wave selected as a source at all. The waveform that is driven by the LFO is wave 204, which sounds quite different and looks like this:

It is not a sine wave at all, it sounds more like some synthesized waveform, frankly, the K1 manual names this waveform SYNTH_01

While testing something else on the real K1, I noticed something strange. I pressed a note and modified the coarse tune while holding the note down. When I released the note and retriggered it, the sound was different! I tested this a couple of times and every time, the sound was different after retriggering when I modified the coarse tune while holding a note.

I had an idea what that means and after doing a test recording of a waveform for all notes, ranging from 0 to 127 I had the proof: Single cycle waveforms are multisamples!

Waveform 19 (SAW) from note 0 to note 127

If you take a closer look at the image, especially at the frequency spectrum in the lower part, you’ll see that some harmonics get lost as the note number is rising.

That explained a lot. No wonder that I never had that original K1 sound. As the recordings by Chvad were only one version of each waveform, I had to record every single cycle waveform again, but this time for each note number.

It took a lot of hours to get this done but it was definitely worth it. I’ve setup a simple track in Cubase to play all notes from 0 to 127 and recorded all 204 single cycle waveforms.
It turned out that not every waveform contains multisamples. And for some of them, the way they use them is really strange. Usually, if you do this, you cut harmonics before they reach the nyquist frequency to prevent aliasing. But in the case of the K1, this is not (properly) done. Some multisamples clearly change the tone when you play on the keyboard, for others, they even make it worse by adding harmonics instead of removing them, which increases the aliasing even more.
Some examples:

Waveform 8 (Sine variant), no multisamples, large aliasing issue on higher notes.
Waveform 14 (Saw variant), highest overtones are dropped too early, making it sound dull
Waveform 37 (Square variant), no multisamples and huge aliasing
Waveform 94 (piano / electric piano), a new subharmonic is added
Waveform 98 (piano, electric piano), highest harmonics are increased in volume just before nyquist, increasing aliasing even more

As you can see, there are a lot of variantions. If you want an exact recreation, you need to take all different multisamples into account.

Single Cycle Waveforms conversion tool

Once more, I extended my tool to deal with these recordings. I manually added markers to the notes that I want to extract. You can see the markers in the screenshots above. My tool uses these markers to extract loops of the waveforms at their respective positions and exports to single files.

Notice the note numbers on the left. The tool names these markers to make sure that the root note of the loop is known, so the playback is done at the correct pitch.

After all 204 files are created, they are merged into a single file, which allows me to load them more easily. The result is a 1,66 MB wave file with over 500 markers in total, named as wave120_note84, etc.

The sound has improved a lot, in many areas, its very similar to a K1 now, just as it should.
What I didn’t do yet, as it is a lot of manual work, is to clean up the recordings a bit. There is a bit of K1 noise that could be easily filtered out, but it would need to be a manual process for each waveform and for each multisample, as the used frequencies are different for each one, leading to over 500 manual tweaks.

Another minor thing is that, although the original K1 wave rom consisted only of 512kb of data, my data size is about 24mb in total as I have every waveform as 32 bits floating point data. It is not much nowadays for a plugin to be that large, but an optimization would be nice.

Kawai K1 VST: Audio Demo

Update: New Audio Demo is here

Recently, I worked in lots of different areas. I re-recorded all wavetables (separate article will follow), reworked the envelope timings, implemented velocity curves, measured LFO speeds and more.

I thought its about time for some audio demos. I recorded this using the current state of the VST plugin. Of course, there are still a lot of issues, but a lot of presets are running fine already.

Kawai K1 VST – some of the factory singles

In the audio demo above, you hear the following factory presets (in order of appearance):

  • iA-5 Return Home
  • IB-3 Jazz Harp
  • IA-5 Visitors
  • IA-3 String Pad
  • IC-7 Terminator

The only thing I added is a bit of reverb and EQ. Let me know what you think!

Kawai K1: Analyzing Envelope Decay, Release, Level & Velocity using tools

Next step for me was to analyze the envelope values and velocity mappings. Because the LFO speed analysis was quite cumbersome, I decided to create a tool to analyse further parameters to make it a little easier for me.

For the envelope decay & release, I noticed that several singles share the same setting for both values. This is mostly used for drum presets but for some others, too. I verified it on the K1m and confirmed that this was the case.

Then, I used my simple init preset again and recorded note presses while adjusting the envelope decay value from 0 to 100. The result was this very long recording:

Largest envelope decay time is nearly three minutes!

The largest envelope decay/release time is 2:44 minutes! But that was not the only thing I observed, I also noticed that the envelope decay is not purely logarithmic, but is a mixture of some logarithmic key points with linear interpolation between them. For longer release times, this is pretty obvious when looking at the waveform, but luckily not very noticeable when listening to it.

Having the recording ready, I created a little wave analyzer whose purpose is to dump envelope durations. It analyzes the waveform to find where a note begins and tracks the duration until the wave falls below an adjustable threshold and logs the time.
While this doesn’t work properly for the very short attack times (I measured them manually instead), it saved a lot of time.

Unfotunately, I had no chance to analyze the envelope attack times, as my K1m is broken here, but I assumed that they are identical and first tests confirm that, so I have all of three (Attack, Decay, Release) finished by now.

I extended this little tool to output gain values. I used it for the various velocity curves, envelope levels, sustain levels and so on.

Velocity Curve 1

As you can see in the picture above, the velocity curve is a little bit steppy. Apparently the resolution is not very good. Anyway, I used the same low resolution in my emulator to closely match the original device.