Nils‘ K1v: VST3 & Apple M1 support

Nils‘ K1v version 1.23 has been released, which brings the following two new features:

VST3 Plugin

The plugin is now available as VST3 for Windows, Mac OS and Linux. The VST3 version is identical to the VST2 version but adds support for multichannel output.

Note that, by default, only the Stereo Out is active. If you want to use Multi Channel Output, activate the buses in your DAW manually.

Note: As soon as one of the multi outputs is activated, the Stereo Output is silenced.

The VST3 plugin has the following outputs:

Stereo OutDefault Stereo Output
Source / Multi 1Source 1 in Single mode, Single 1 in Multi mode
Source / Multi 2Source 2 in Single mode, Single 2 in Multi mode
Source / Multi 3Source 3 in Single mode, Single 3 in Multi mode
Source / Multi 4Source 4 in Single mode, Single 4 in Multi mode
Multi 5Single 5, only used in Multi mode
Multi 6Single 6, only used in Multi mode
Multi 7Single 7, only used in Multi mode
Multi 8Single 8, only used in Multi mode
VST3 Multi Channel Outputs

Issues? Feedback is welcome! 👍

Apple M1 Support

Support for Apples new ARM-based SoC named M1 has been added to all Mac OS plugin versions. There are no separate downloads as the latest build now contains a so called „universal binary“ which contains the binary for both the x86_x64 and ARM64 based processors.

This should give a performance boost on Apples new SoC. It would be nice if you would report back if everything works as intended 👍 as I don’t have an Apple M1.

Download Page

Kawai K4: Really a 16 bit synthesizer? Only partially!

I started to analyze the K4r. The first thing I did is to dump the contents of the Firmware EPROM and the Wave ROMs.

I dumped the firmware just to verify that the content on the chip matches the firmware image that I already had on disk, which it does.

Wave ROM dumps

In contrast to the K1, the K4 has three Wave ROMs in total.

From left to right: Kawai K4 Wave ROM chips U40, U39, U38

Nothing unexpected so far. As the K4 claims to be a 16 bit sampler, it was expected that it needs more storage for the samples.

After having dumped all of them, which was very easy due to them being socketed this time, I loaded them into my wave editor to take a first look. I imported the raw data as 16 bit and listened.

Analyzing Wave ROM chip data

While I had no issues with chips U39 and U40, chip U38 (the rightmost one in the picture above) sounded like garbage. What I also noticed is, that the former two chips sounded a bit distorted, had a lot of noise and the pitch seemed to be pretty high.

I looked at the data in a hex editor and was surprised to find content that much more looked like 8 bit samples rather than 16 bit samples. This can be seen by adjacent bytes constantly rising or falling, as below:

I changed the import format and loaded them again.

To my surprise, chips U39 and U40 sounded just fine now! A little bit of 8 bit noise, but nothing unexpected.

K4 Wave ROM chip U39

I guessed that the K4 somehow creates 16 bit samples by combining the contents of the chips together. But I had three chips only, where are my remaining 512k of data that I would need? Maybe the samples are 12 bit?

Chip U38 is special

I took a closer look at chip U38, the chips that only produced random noise so far.

The end of the chip data raised my attention as it looked like sample data:

Chip U38 data

What do we see here? We see eight least significant bits of a sample. This proved that this is not „random“ data but actually raw sample data. It furthermore cleared up that this chip does not contain a nibble for each of the other two chips to create 12 bit samples.

If this chip contains the LSB of another chip, to which chip does this data belong?

This can be tested by looking at the frequency spectrum of all three chips. Obviously, for chip U38 this is mostly noise, but not everywhere. For U39 and U40, we see regular frequency spectrums of sample data.

U38 -Frequency Spectrum
U39 -Frequency Spectrum
U40 -Frequency Spectrum

Do you see it? There is a relationship between the two chips U38 and U39.

Getting 16 bit samples out of two chips

I wrote a small program which combines the contents of these two chips to form 16 bit samples and loaded the result into my wave editor.

Gotcha, now we have 16 bit samples. If you look at these two pictures, you can see the difference immediately. The lower picture has got a much lower noise floor and it sounds much cleaner.

Chip U39 – 8 bit samples
Chip U38 and U39 combined, 16 bit samples. U39 = MSB, U38 = LSB

What about chip U40 then? I also tried to merge chips U38 and U40 but this did not work out, as expected. As there is no data left, the conclusion is that chip U40 is used as a source for 8 bit samples.

Conclusion

Although Kawai claimed that the K4 is a 16 bit synthesizer, this is only partially true as only half of the sample data is 16 bit, the other half is 8 bit.

What they did to hide it is to move all samples with lots of noise, like flutes, the choir, cymbals, snares and others to chip U40 which holds 8 bit sample data.

Samples without lots of noise like the piano, basses, acoustic guitar and others are stored as 16 bit samples. Chip U38 carries the 8 bit LSB while chip U39 8 bit MSB.

What I also found out is that the single cycle wave forms have more multisamples and the PCM wave forms are now partially multisamples, too! Which should give a quality boost compared to a K1.

Nevertheless, the K4 is more similar to a K1 than one might think. Most chips are identical, the firmware dumps look quite similar (same velocity curves, KS curves etc.) and the K4 even has got lots of 8 bit samples still!

Stay tuned, more to come! 👍

Evan Sirchuk donated a Kawai K4r! ❤

To my surprise, my Kawai K1 emulation, started out of a personal need, was a tremendous success. Something that I never expected, I’ve got tons of (good) feedback, some magazines even interviewed me and I’ve got loads of traffic on my web page.

One thing that stood out is the request to emulate the K1 successor, the K4. What prevent me from even thinking about it is that I would need to have a K4 to analyze it. I have read some documentation and the K4 is – more or less – a K1 with filters and improved samples. But as long as I didn’t have a K4 there wouldn’t be a chance for me to come up with something that is even close to a K4.

Then, all of a sudden, a mail from Evan Sirchuk arrived, saying that he would like to donate a K4r 😍 That was around Christmas. Even though I was blown away by this offer I didn’t have the time to react immediately but I replied some weeks later, I was interested of course!
Some mails went forth and back about how to ship it, customs questions (he is US citizen while I am German/EU citizen) but he obviously had a lot of shipping experience and he did it!
Some additional weeks later, I picked up the K4 at customs. Very well packaged and fully functional! Absolutely amazed by this. Of course, I sent him the shipping costs via PayPal. He didn’t even ask for it, what a great guy.

Look at this beauty:

Kawai K4r

I want to take this opportunity to say big thanks to Evan. If you want to say thanks to him, visit his web page or his Twitter. He deserves it!

K4 Emulation coming?

Most probably, yes. Hereby confirmed. The K4 is technically quite similar to a K1. It will be challenging to model the K4s filters though and how similar it will sound, I can’t give any guarantee yet, but I already have an idea how to do it.

But I cannot give any ETA. I still have some private things going on that I need to finish before I can dive deeply into the K4 and at the moment, free time is a rare good (Covid etc).

There’s one thing though that I couldn’t resist doing with the K4r. People that looked closely at the picture above might have noticed that the K4 is missing its rack ears. This is because I already opened it. 😎

I dumped the wave roms (3 chips). Luckily, they have socketed them in the K4r so this was no big deal. I wanted to know upfront if I can make use of the content, and yes, the content is not encrypted. More on that in a separate article.

Of course, you will be able to watch my progress in my K4 development blog, this is the first of many articles in this category. And the resulting VST/AU will be free, for sure 👍

Once again, big thanks to Evan Sirchuk for this great opportunity!

Nils‘ K1v: Version 1.19 released

I’ve just released an update for Nils‘ K1v, now at version 1.19. This is a maintenance release with the following fixes:

  • [Fix] Dropped midi events caused notes to be hanging or not played at all in some hosts
  • [Fix] Neither SysEx send nor receive worked in Mac AU version
  • [Fix] UI did not update if single parameter was changed via MIDI SysEx
  • [Fix] Pitch Bend changes were processed even if turned off in global settings

Please head over to the product page to download the latest version.

Download

Thanks a lot for your ongoing support! 👍

Nils‘ K1v in a browser as web assembly using Emscripten

Lately, I experimented a bit with VSTGUI again and the result is a running Kawai K1, directly in your web browser 😃

TL;DR if you want to try it: Click Here

🛑Needs a decent browser that supports web assembly! 🛑

If you want to know more about the implementation details, continue reading below.

Nils‘ K1v – VSTGUI, OpenAL, SDL2 – compiled as web assembly using Emscripten.

This idea evolved while I thought about how to release my other project, Heat Synthesizer as a native VST/AU including a UI. At the moment, it is an Android app with a small proxy VST for Windows only. The reason for not having a UI in the VST version is that I created it for Android using the native Android UI system in Java. Not an optimal solution to port it to different platforms.

To be able to port Heat Synthesizer to VST & AU, I was in need of a UI system that supports a wide range of platforms:

  • Android – Obviously I don’t want to drop support here so this is a must have
  • VST on Windows – I want to use my synthesizer directly in my DAW that I personally use, without the need to fire up my Android tablet to edit sounds
  • AU/VST on Mac – Would be beneficial if I were able to release it there, too

The Heat Synthesizer engine code compiles just fine for a wide range of platforms. I even had it running on an iPad already many years ago, just for testing purposes. But having a UI framework that covers all these platforms is not easy to find.

Actually, I recreated the UI in Unity3D some years ago. I thought this would be a good candidate to be more flexible with regards to targeting different platforms. Which is true for both iOS and Android, but Unity3D even nowadays lacks embedding support into runtime libraries so bringing it to VST and AU plugins wouldn’t be possible.

Heat Synthesizer in Unity3D

For the K1v, I’ve chosen a different strategy right from the start. Just in case you missed my development blog so far, K1v is created in C++ and the UI is made with VSTGUI. It’s a UI system by Steinberg, optimized for audio software. VSTGUI is open source and has become quite flexible throughout the years. It is no longer tightly bound any kind of existing VST SDK but also works with AU, VST3 or even in standalone mode.

What I came up with was the question if it was possible to create a fork of VSTGUI that supports Android. Knowing that I have to redo the Heat Synthesizer UI completely from scratch, my K1v project came in very handy here. K1v is a VSTGUI-based project that already compiles on a large variety of platforms (Win, Mac, Linux, both 32 and 64 bit each) that I could play around with.

K1v in Standalone Mode

Obviously, the first step was to create a standalone app of Nils‘ K1v. An Android app would be a „Standalone“ app, not a plugin. That step was pretty much straightforward and was finished after one evening.

The internal structure of K1v is that there is a generic „K1Plugin“ and a „K1Editor“ and these two are then wrapped for the different plugin platforms such as VST2, VST3 (unreleased yet) and AU.

I created an executable that opens a VSTGUI-based standalone window, embedded the K1v editor and created the K1v plugin instance.

Audio Output? MIDI?

To test the standalone app, I needed an audio output and an audio processing loop. Luckily, I solved this years ago as part of the Heat Synthesizer project. There is a simple audio output interface class with a lot of different implementations:

  • OpenSLES – Used on Android
  • Jack – Used for Samsung Soundcamp on Android. Unfortunately they stopped development and dropped support, but this implementation might come in handy on Linux in the future
  • OpenAL – I used this for testing purposes as the SDK is very small and I already knew it from game development
  • PortAudio – I added this some years ago to have lower latency on Windows
  • CoreAudio – Used to test on Mac & iOS

I opted for PortAudio and OpenAL for now as I have been able to test them directly on Windows.

As for MIDI, there is just no support included. To test that everything works, I injected a Note ON event on Note C3 at startup, should be enough. I didn’t plan to release a standalone version anyway so no need to implement MIDI here.

Emscripten & WebAssembly

You may ask why there is a web version now even though I kept writing about Android?

For me, Emscripten is an intermediate step. I didn’t want to deploy to my Android tablet over and over again to test my progress so I thought that Emscripten as a first challenge would be a good candidate to start with.

As I plan to keep most of the code in C++ anyway, both platforms are more similar than expected. Android an Emscripten are both „based“ on Linux (more or less), they have OpenGL ES as possible renderer and as I use CMake as a build system on all platforms differences are very minor during build.

Additionally, I’m not sure if an Android version of K1v is really useful. the role of Heat Synthesizer is different here, as it contains lots of online functionality (preset exchange) and can connect to a VST proxy for proper DAW integration, but as K1v is a VST already that doesn’t make much sense.
Releasing it for the web on the other hand is a nice challenge and furthermore something that didn’t exist before, at least I’m not aware of any existing VSTGUI to Emscripten ports.

Porting VSTGUI to Emscripten

While my K1v code compiled more or less directly, a lot of effort was needed to get VSTGUI up and running.

Everything I write here is based on VSTGUI 4.9, to be more specific, the develop branch at 2020.06.24 commit dc95f6a2. This is the version I used to develop the UI for Nils‘ K1v. I did some bug fixes and fixed Linux compilation errors at this time, but other than that, this is the state I’m currently working with / what the Emscripten version is based on.

The VSTGUI core code (everything that is platform independent) only had smaller compilation problems in one or two cases, but the largest issue was that I didn’t have anything for this new platform: No frame, no window, no run loop etc.

I decided to start with the Linux version. The Linux version is based on Cairo for rendering, uses X11/XCB for input handling and GDK for the main loop in Standalone builds.

Cairo for rendering

I knew that Cairo can be compiled in a way to use OpenGL ES for rendering, which both Android and Emscripten support. So i decided to fork Cairo and compile it for Emscripten using CMake.

This brought a lot of dependencies: zlib, libpng, pixman, libpixman.

More projects that I had to fork. Some of them already had CMake build scripts, which was great, for others, I created CMake build scripts from scratch. It took some time but worked eventually.

Although the web assembly based executable linked fine, I still had a lot of work to do. I didn’t solve the standalone part yet, i.e. the window handling and mouse & keyboard input.

I created some stub classes to be able to test it. At last, I heard the K1v preset IA-1 Ahh playing Note C3 in a browser window, yeah 😎

Using SDL2 to have a render target for Cairo

The next problem I had to solve was to define some sort of „window“ as that is what a standalone app in VSTGUI is based on. The problem is that you do not have any sort of window in Emscripten, as you’re running in a web browser. You get a canvas that you can paint on, that’s it.

Luckily, the Emscripten team already solved this problem by porting the SDL library to Emscripten. This was great news as the SDL library also supports Android.

I spent quite some time connecting an SDL „window“ to Cairo. I wanted to give hardware acceleration to Cairo so I compiled Cairo with OpenGL ES support and created a GL Surface in SDL. Unfortunately, I had to drop this idea for Emscripten as only a subset of OpenGL ES 2.0 is implemented and Cairo makes use of these missing features (for example pbuffers).

So instead of using the GL based backend, Cairo runs in software at the moment. Nevertheless, I was blown away when I saw parts of my UI for the first time:

Font Support

As you might have noticed, there are no texts in the UI. The reason for this is that the Linux build of VSTGUI uses fontconfig to enumerate the available system fonts at startup. I had to comment out that whole code block as there is no fontconfig support in Emscripten and adding this library doesn’t make sense as there are no „system fonts“ available anyway. So instead of using fontconfig, I did the following:

I created a virtual file system for Emscripten at compile time with a folder named „fonts/“ that contains all fonts that I need for K1v. These are not many and the resulting data blob is around 1Mib in size only.
At load time, this font list is built by enumerating through all files in this folder. Then, I load the files with FreeType to query for the family name and style name to finally rebuild the original fonts list that VSTGUI uses. This worked out pretty well and is completely customizable as the available fonts depend on what you put into the virtual filesystem. No hardcoded hacks were necessary.

Web assembly version of K1v with libcairo rendering fonts
Input Handling

As I had the SDL library, adding input handling was pretty straightforward. I use the SDL for mouse, keyboard & window events. I also use the SDL for timers.

Missing Features

The project is highly experimental and in a very early stage. VSTGUI support lacks the following:

  • file selector is not available
  • no alert boxes
  • no support window styles as there is no real window
  • no drag & drop
  • no tooltips
  • no clipboard support

The web version of Nils‘ K1v doesn’t currently support:

  • State is not preserved, if you reload the page, you’re going to have factory patches again, no matter what you edited before
  • MIDI – The only possible input is the virtual keyboard on the web page
  • Load or Save of files due to missing VSTGUI file selector

Most of the above could be implemented easily. Another thing worth mentioning is that I don’t currently use AudioWorklets as it seems to be quite cumbersome for now to get them running in Emscripten. The audio output is done by OpenAL and everything is single threaded. You can expect some audio pops & clicks here or there, it depends on your browser & computer, mostly noticeable when large UI parts are redrawn. Comments about how it works for you would be nice.

Future Work

After finishing the port to Emscripten, its time for some cleanup work, obviously. After I will have finished cleaning up the VSTGUI parts, I’m going to make my VSTGUI fork open source. I might decide to do a pull request, maybe Steinberg is interested to integrate official Emscripten support? Who knows.

To summarize, I had to create implementations for the following VSTGUI interfaces to make VSTGUI work with Emscripten:

  • VSTGUI::IPlatformTimer – SDL timers
  • VSTGUI::IPlatformFrame – SDL <=> Cairo frame
  • VSTGUI::Standalone::Platform::IWindow – SDL Window
  • For standalone mode, an SDL application which drives VSTGUI::Standalone::Detail::Application
  • Some groundwork such as an SDL run loop to drive redraws & inputs.

These are all great preparation steps to have an Android port of Heat Synthesizer. I’m going to continue this journey with the K1v project though, as I want to proof that my concept works before doing a complete UI redesign of Heat Synthesizer with VSTGUI.

I’m confident that the Android version of the SDL/libcairo combination will support hardware accelerated rendering, which is even more important on mobile. As Android has full OpenGL ES 2 support, this should work fine.

I’m going to post updates on this topic. Leave a comment and tell me what you think 👍