🛑Needs a decent browser that supports web assembly! 🛑
If you want to know more about the implementation details, continue reading below.
This idea evolved while I thought about how to release my other project, Heat Synthesizer as a native VST/AU including a UI. At the moment, it is an Android app with a small proxy VST for Windows only. The reason for not having a UI in the VST version is that I created it for Android using the native Android UI system in Java. Not an optimal solution to port it to different platforms.
To be able to port Heat Synthesizer to VST & AU, I was in need of a UI system that supports a wide range of platforms:
Android – Obviously I don’t want to drop support here so this is a must have
VST on Windows – I want to use my synthesizer directly in my DAW that I personally use, without the need to fire up my Android tablet to edit sounds
AU/VST on Mac – Would be beneficial if I were able to release it there, too
The Heat Synthesizer engine code compiles just fine for a wide range of platforms. I even had it running on an iPad already many years ago, just for testing purposes. But having a UI framework that covers all these platforms is not easy to find.
Actually, I recreated the UI in Unity3D some years ago. I thought this would be a good candidate to be more flexible with regards to targeting different platforms. Which is true for both iOS and Android, but Unity3D even nowadays lacks embedding support into runtime libraries so bringing it to VST and AU plugins wouldn’t be possible.
For the K1v, I’ve chosen a different strategy right from the start. Just in case you missed my development blog so far, K1v is created in C++ and the UI is made with VSTGUI. It’s a UI system by Steinberg, optimized for audio software. VSTGUI is open source and has become quite flexible throughout the years. It is no longer tightly bound any kind of existing VST SDK but also works with AU, VST3 or even in standalone mode.
What I came up with was the question if it was possible to create a fork of VSTGUI that supports Android. Knowing that I have to redo the Heat Synthesizer UI completely from scratch, my K1v project came in very handy here. K1v is a VSTGUI-based project that already compiles on a large variety of platforms (Win, Mac, Linux, both 32 and 64 bit each) that I could play around with.
K1v in Standalone Mode
Obviously, the first step was to create a standalone app of Nils‘ K1v. An Android app would be a „Standalone“ app, not a plugin. That step was pretty much straightforward and was finished after one evening.
The internal structure of K1v is that there is a generic „K1Plugin“ and a „K1Editor“ and these two are then wrapped for the different plugin platforms such as VST2, VST3 (unreleased yet) and AU.
I created an executable that opens a VSTGUI-based standalone window, embedded the K1v editor and created the K1v plugin instance.
Audio Output? MIDI?
To test the standalone app, I needed an audio output and an audio processing loop. Luckily, I solved this years ago as part of the Heat Synthesizer project. There is a simple audio output interface class with a lot of different implementations:
OpenSLES – Used on Android
Jack – Used for Samsung Soundcamp on Android. Unfortunately they stopped development and dropped support, but this implementation might come in handy on Linux in the future
OpenAL – I used this for testing purposes as the SDK is very small and I already knew it from game development
PortAudio – I added this some years ago to have lower latency on Windows
CoreAudio – Used to test on Mac & iOS
I opted for PortAudio and OpenAL for now as I have been able to test them directly on Windows.
As for MIDI, there is just no support included. To test that everything works, I injected a Note ON event on Note C3 at startup, should be enough. I didn’t plan to release a standalone version anyway so no need to implement MIDI here.
Emscripten & WebAssembly
You may ask why there is a web version now even though I kept writing about Android?
For me, Emscripten is an intermediate step. I didn’t want to deploy to my Android tablet over and over again to test my progress so I thought that Emscripten as a first challenge would be a good candidate to start with.
As I plan to keep most of the code in C++ anyway, both platforms are more similar than expected. Android an Emscripten are both „based“ on Linux (more or less), they have OpenGL ES as possible renderer and as I use CMake as a build system on all platforms differences are very minor during build.
Additionally, I’m not sure if an Android version of K1v is really useful. the role of Heat Synthesizer is different here, as it contains lots of online functionality (preset exchange) and can connect to a VST proxy for proper DAW integration, but as K1v is a VST already that doesn’t make much sense. Releasing it for the web on the other hand is a nice challenge and furthermore something that didn’t exist before, at least I’m not aware of any existing VSTGUI to Emscripten ports.
Porting VSTGUI to Emscripten
While my K1v code compiled more or less directly, a lot of effort was needed to get VSTGUI up and running.
Everything I write here is based on VSTGUI 4.9, to be more specific, the develop branch at 2020.06.24 commit dc95f6a2. This is the version I used to develop the UI for Nils‘ K1v. I did some bug fixes and fixed Linux compilation errors at this time, but other than that, this is the state I’m currently working with / what the Emscripten version is based on.
The VSTGUI core code (everything that is platform independent) only had smaller compilation problems in one or two cases, but the largest issue was that I didn’t have anything for this new platform: No frame, no window, no run loop etc.
I decided to start with the Linux version. The Linux version is based on Cairo for rendering, uses X11/XCB for input handling and GDK for the main loop in Standalone builds.
Cairo for rendering
I knew that Cairo can be compiled in a way to use OpenGL ES for rendering, which both Android and Emscripten support. So i decided to fork Cairo and compile it for Emscripten using CMake.
This brought a lot of dependencies: zlib, libpng, pixman, libpixman.
More projects that I had to fork. Some of them already had CMake build scripts, which was great, for others, I created CMake build scripts from scratch. It took some time but worked eventually.
Although the web assembly based executable linked fine, I still had a lot of work to do. I didn’t solve the standalone part yet, i.e. the window handling and mouse & keyboard input.
I created some stub classes to be able to test it. At last, I heard the K1v preset IA-1 Ahh playing Note C3 in a browser window, yeah 😎
Using SDL2 to have a render target for Cairo
The next problem I had to solve was to define some sort of „window“ as that is what a standalone app in VSTGUI is based on. The problem is that you do not have any sort of window in Emscripten, as you’re running in a web browser. You get a canvas that you can paint on, that’s it.
Luckily, the Emscripten team already solved this problem by porting the SDL library to Emscripten. This was great news as the SDL library also supports Android.
I spent quite some time connecting an SDL „window“ to Cairo. I wanted to give hardware acceleration to Cairo so I compiled Cairo with OpenGL ES support and created a GL Surface in SDL. Unfortunately, I had to drop this idea for Emscripten as only a subset of OpenGL ES 2.0 is implemented and Cairo makes use of these missing features (for example pbuffers).
So instead of using the GL based backend, Cairo runs in software at the moment. Nevertheless, I was blown away when I saw parts of my UI for the first time:
As you might have noticed, there are no texts in the UI. The reason for this is that the Linux build of VSTGUI uses fontconfig to enumerate the available system fonts at startup. I had to comment out that whole code block as there is no fontconfig support in Emscripten and adding this library doesn’t make sense as there are no „system fonts“ available anyway. So instead of using fontconfig, I did the following:
I created a virtual file system for Emscripten at compile time with a folder named „fonts/“ that contains all fonts that I need for K1v. These are not many and the resulting data blob is around 1Mib in size only. At load time, this font list is built by enumerating through all files in this folder. Then, I load the files with FreeType to query for the family name and style name to finally rebuild the original fonts list that VSTGUI uses. This worked out pretty well and is completely customizable as the available fonts depend on what you put into the virtual filesystem. No hardcoded hacks were necessary.
As I had the SDL library, adding input handling was pretty straightforward. I use the SDL for mouse, keyboard & window events. I also use the SDL for timers.
The project is highly experimental and in a very early stage. VSTGUI support lacks the following:
file selector is not available
no alert boxes
no support window styles as there is no real window
no drag & drop
no clipboard support
The web version of Nils‘ K1v doesn’t currently support:
State is not preserved, if you reload the page, you’re going to have factory patches again, no matter what you edited before
MIDI – The only possible input is the virtual keyboard on the web page
Load or Save of files due to missing VSTGUI file selector
Most of the above could be implemented easily. Another thing worth mentioning is that I don’t currently use AudioWorklets as it seems to be quite cumbersome for now to get them running in Emscripten. The audio output is done by OpenAL and everything is single threaded. You can expect some audio pops & clicks here or there, it depends on your browser & computer, mostly noticeable when large UI parts are redrawn. Comments about how it works for you would be nice.
After finishing the port to Emscripten, its time for some cleanup work, obviously. After I will have finished cleaning up the VSTGUI parts, I’m going to make my VSTGUI fork open source. I might decide to do a pull request, maybe Steinberg is interested to integrate official Emscripten support? Who knows.
To summarize, I had to create implementations for the following VSTGUI interfaces to make VSTGUI work with Emscripten:
For standalone mode, an SDL application which drives VSTGUI::Standalone::Detail::Application
Some groundwork such as an SDL run loop to drive redraws & inputs.
These are all great preparation steps to have an Android port of Heat Synthesizer. I’m going to continue this journey with the K1v project though, as I want to proof that my concept works before doing a complete UI redesign of Heat Synthesizer with VSTGUI.
I’m confident that the Android version of the SDL/libcairo combination will support hardware accelerated rendering, which is even more important on mobile. As Android has full OpenGL ES 2 support, this should work fine.
I’m going to post updates on this topic. Leave a comment and tell me what you think 👍
I’ve just released version 1.14 of Nils‘ K1v – The Kawai K1 emulation VSTi/AU.
Biggest change is that it now also supports Multi presets. I added the 32 factory Multis and all Multis from the ROM cards, resulting in 384 Multi presets in total. As a sum, Nils‘ K1v now contains 1352 presets.
Of course, Multis can be edited, too. To add all of the new features, I modified the UI slightly. The Source Mute buttons have been moved to the Single Edit page, making space for the new Buttons MULTI and SYSTEM.
I’ve been able to fit four Multi sections on one page, arrow buttons allow to switch between sections 1-4 and 5-8.
Multis work as they do on the Kawai K1 hardware. Up to eight Singles can be combined in various ways to create either rich layered sounds or multiple zones on the keyboard.
The only difference between a K1 and K1v is that in Poly Mode VR (meaning variable), the polyphony is unlimited, while it was still limited to 16 or 8 voices in total on a K1 hardware.
The SYSTEM page is a new page that allows to change global parameters, basically it’s a copy of the K1 Midi Settings and supports specifying a Midi receive channel (previous versions always ran in Omni mode) and to disable Program Change processing, Sysex & more.
Global settings are saved as part of the DAW project.
Besides the improvements as pointed out above, some additional things have been added, such as cursor key preset navigation (Hallo Jürgen!) and of course, bugs have been fixed, too. The full change log can be found below.
The most noticeable change is that I have added all 200 PHm Pop Synth Module presets to the plugin, resulting in a total of 968 Singles. As the list of Singles became even larger, I reworked the play mode section to make Single selection easier. There is now a dropdown to allow to select a bank directly and there are additional navigation buttons.
Another interesting change for some people might be one accuracy fix: Key Scaling =>Envelope Time didn’t modify Decay but only Attack, causing some Singles to sound incorrect.
More details & all fixes and changes can be found in the change log excerpt below. The new version is available for all platforms and can be obtained from the Download Page
[Fix] Drop down menus were hard to read on Linux because of the font being too tiny
[Accuracy] Key Scaling => Envelope Time modulation did not affect Decay but only Attack, causing some Single to sound incorrect when compared to the K1 hardware. This change fixes the Single ‚KillDa Mix‘ and others, mostly Pianos.
[Imp] All 200 PHm presets are now part of the plugin, causing it to have 968 total built in presets.
[Imp] Single selection screen has been reworked, for easier navigation it now includes a dropdown to select the Bank and has more navigation buttons.
[Imp] Plugin Logo has been reworked to match the K1 hardware logo. Furthermore, the „KAWAI“ logo has been integrated as Kawai gave official permission for the plugin.
I just released Nils‘ K1v at version 1.12. It contains two bug fixes and one important improvement, as I mentioned earlier in my development blog: Velocity Curves now very closely match those of the Kawai K1 hardware.
The release is available for all platforms and can be obtained from the Download Page
[Fix] One-shot waveforms (drums & others) were played looped
[Fix] Mac AU version crash when saving the plugin state (100% crash in Digital Performer 10)
[Imp] Velocity Curves are now properly implemented and very closely match those of the K1 hardware. If you are interested in details, feel free to visit the K1 development blog
Privacy & Cookies Policy
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.