High-level alternative to KSP

Hi everyone, I decided to make a new thread for a discussion which originated in the "Kontakt 2 update plan" thread to not disturb the discussion there. Here's a copy of what was written in that thread just to give some background. If you followed that thread please skip the first two posts here.

I too was a bit surprised not to find that in KSP. Therefore I wrote my own language with support for user-defined functions, and user-defined data structures (like struct/record/class in C/Pascal/Java) which can be nested. Using local variables in functions and data structures really lets you organize and get rid of the mess of variables in "on init" in KSP scripts. It also has support for for-loops (including for-each style loops) and "else if" - constructs which I missed in KSP. The language is based on Python so I can use a standard editor and get syntax highlighting. Also there is no need to use USE_CODE_IF inside callbacks/functions, since if and else statements are optimized so that parts which clearly will never be run are removed during optimization.
Finally the code is compiled to readable and pretty efficient KSP-code.

It's not completely ready yet (still perfectly usable for my needs and larger scripts though). Seeing that I could now write complex scripts in very short time, I was a bit let down when I discovered that you can only have six rows and columns of UI controls. Please NI, consider fixing that as well as introducing functions in KSP like Bob suggested.

If there's an interest I'm thinking of making my compiler publicly available in some form. Maybe that could also be a way to inspire NI to do some improvements on KSP...

Best Regards,
kotori

 

And I got this answer from Big Bob:

 

Ok, so now to my answer (not in the original thread)

Bob, you're absolutely right. My compiler outputs a regular KSP script that follows the usual syntax. I work outside of Kontakt in a normal text editor, compile my language to KSP-code and then paste that code into Kontakt, like this:

Script in my language --> compiler --> script in KSP --> paste into Kontakt and press apply.​

A little bit of extra work of course, but not much compared to what I gain in productivity.

Regarding avoiding duplicate code in the KSP code that the compiler outputs, I don't do that (because of KSP limitations). Every function call is replaced by the body of the function (inline). But this is not really problem since I'm writing the scripts in my own language and not in KSP, so I never see those duplicates and they don't make the script any less efficient. Inlining also has the small limitation that a function cannot call itself directly or indirectly. Maybe I should post some examples of what the code in my language looks like.

The code you posted were interesting. You use some smart tricks there indeed. I considered designing a very general function call system without the limitation mentioned above using the same kind of code you use to select which function to call. Reading your code it also strikes me that you really could use an "else if" construct (can't really understand why NI left that out).

About the effort I put down into the compiler I'm sure it's not very much more than you put into your script (counting lines of code your script is about half as many as my compiler).

Best Regards,
Kotori

 

Well,

I think every approach towards a bit more "higher" programming would be nice. This would allow better scritping, as everything becomes more structured.

But also more access to the Kontakt engine and the samples would be very nice and helpful. At the moment that is pretty limited or needs work-arounds (via midi cc).

As I am only an occational programmer/scripter I would prefer the scripting language itself to mature, because then I do not have to think in two languages when scripting. But I do not now, maybe your solution is better then scripting in Kscript.

Best

 

Hi Kotori & Steff3,

If we don't watch out, we're going stimulate each other into some deep thinking ;-) But, since the ball is rolling, I'll just add a few comments.

Actually, it isn't any extra work at all! Until NI gets the KSP editor working in a more friendly way, I don't think anyone would try to write anything but a trivial program by typing it directly into K2, I know I don't. I usually just use something like Notepad for typing and editing and then paste it into the KSP when it's ready to compile and test. My point here is that most of us will be using some 'outside' tools anyway, so I don't see that it would be any more difficult 'mechanically' to use the 'kotori compiler' and paste to K2 after each compilation. However ...

Steff3's concern here may well represent the concerns of many others, since most musicians aren't too comfortable with programming. So this gives rise to the following 'food for thought'. Instead of trying to make make the KC ('kotori compiler') simulate a richer language, maybe there's a case for simply making it more musician friendly. By this I mean, construct sort of a pre-processor that allows musicians to say what they want to do in a less demanding way and then have the pre-processor spit out a KSP script (with all its nasty details) that will do the job.

Agreed. In fact, more efficient since subroutine calls incur overhead. The problem with this is that one must have the KC and its source code in order to have an easier time of reading the script. Someone who simply has the .nkp file and loads it into the KSP text editor only gets to see the KC's object file (ie the script) and that's still messy. (I assume you don't also intend to produce some kind of 'reverse translator' to convert KSP script into KC source code)

I wouldn't worry about this because I think we can safely live without recursion.

Maybe so, but unless the KC has high-level instructions like: 'Do a formant-corrected pitch bend' ;-), I don't see how having a more powerful language would have helped much. The program logic design is mostly language indepedent, and I always design before I code.

Steff3 brings up a topic here that is very important. Regardless of how powerful the KSP scripting language is insofar as its language elements are concerned, in the end, it can do no more than K2 'interface functions' allow. I would like to see more of these functions and more uniformity among those we already have.

One of my pet peeves is the non-uniform way of allowing the KSP to 'intercept' some but not all MIDI controllers. And why not allow the KSP to generate arbitrary MIDI messages such as creating program changes, poly aftertouch and such. As soon as I finish posting this response, I'm going to hunt down a previous posting of mine that discusses the MIDI CC intercept problem. When I find it, I'll come back and edit this post to include a link to that discussion.http://www.native-instruments.de/site/forum_us/showthread.php?t=29141

Because of all the problems with K2 (including the KSP), I didn't want to say too much about future enhancements yet until we get the basic product working right. However, it might not be such a bad idea to start building a KSP Wishlist. Who knows, maybe NI will pick up some these ideas and implement them in future releases.

 

>>
Steff3's concern here may well represent the concerns of many others, since most musicians aren't too comfortable with programming. So this gives rise to the following 'food for thought'. Instead of trying to make make the KC ('kotori compiler') simulate a richer language, maybe there's a case for simply making it more musician friendly. By this I mean, construct sort of a pre-processor that allows musicians to say what they want to do in a less demanding way and then have the pre-processor spit out a KSP script (with all its nasty details) that will do the job.
>>

Well, you might be right with that one, but I wanted to say the opposite
I would be more comfortable to have the scripting code in a more 'C'ish/'C++'ish language (which is quite common for scripting languages I think), so I do not have to learn another (stupid) pseudo-code.
I script a little bit in different programs, use Max/MSP-Jitter and Pd, etc. C is not my world, but I'd rather prefer thinking in such a rough and edgy C instead of another non logical pseudo-code - the pseudo-code of KSP turned me off first - another useless thing to learn.
If everybody thought of his own pseudo-code, programming would even be a greater mess then it is already. (Look at all that bugs around!)

C code would be much cleaner, would have more functionality (if-then-else, structs,...) etc. C++ would even be nicer (classes and overloading functions )

Why not connect to a standard - makes lives much easier from time to time. And if I have to mess around with a scripting language at all - a I said, I would prefer when I would have to learn routines that are commonly used and therefore much more useful than otherwise unuseful, proprietary, non-logical pseudo-code.

Best

 

Big Bob and steff3, thank you for your comments.

I put together a web page explaining how the scripting language works with examples and a list of features and what differs from KSP. For anyone interested, please check it out here.
Please tell me what you think. I decided to call the language Kotori with a captial K (after my nickname here).
If I find the time to finish it and get it to work well I gurantee that it will be free to use at least for scripts that are free, and will remain so.

I agree that standards are nice and I completely understand your point of view. That I didn't make my language more similar to C/C++ is partly because I like Python better and partly because it was easier to do it this way. This way the language will also be easier to use for non-programmers. By the way if-then-else-if and structs are part of the my language (but structs are written a little bit differently than in C).

Actually I think my language is at least as musician friendly as KSP Script, if not more. Most rich features of Python (on which the syntax is based) cannot be implemented anyway. One of the most important properties of a programming language is for it to enable the programmer/scripter be able to quickly get an overview of things. That I think my language does much better than KSP.

But your idea about a more simple pre-processor, eg. to make user-defined functions possible, is also worth to consider.

That said, I should point out that although NI left some things out I think the design of KSP is actually quite good. KSP is easy to grasp and simple enough that Kontakt2 should be able to execute it very fast. But as you say there's still lots of things that cannot be controlled from KSP.

Oh, maybe that didn't sound like I intended it too. I'm sorry. What I really meant to say was that I'm impressed by the amount of work that you've put into it. I'm actually not very skilled in KSP so it was very interesting to see a script like that. I don't think using my language would had made it very much shorter. However, it could potentially have made it a bit more readable.

Best Regards,
kotori

 

Hey Kotor1

I couldn't agree with you more. While we all have our favorite languages (I personally like Borland's Delphi/Pascal myself), the K2's scripting language is actually quite adequate for most things and it isn't difficult to get used to. Actually, I'm glad that it's as powerful and comprehensive as it is.

The most important 'new feature' for the KSP right now is to eliminate the bugs and noise problems. Then, if and when new features are added, hopefully some of them will give us more control over K2 and a few of the language holes like the lack of user-defined functions will also be addressed.

I haven't checked out your 'Kotori' compiler link yet, but I'll do so after I post this response. Thanks for your willingness to share your tools with us.

Hi Steff3

I hear you! I didn't mean to imply that such a simplified interface would be what you were looking for. Since you have some programming skills already, you're probably more interested in a friendlier, but still general-purpose language. But, I wasn't thinking of another generalized language. Rather something more like a very high-level interface, sort of like a 'wizard' that would allow non-programmers to easily specify what they want to do. Such an interface might be able to handle 85% of the things most K2 users want to do with scripting by simply 'filling in some blanks' on a form so to speak. It was just a wild thought that popped into my head as I was writing my response.

As I've already stated, right now I'd gladly settle for the KSP just the way it is MINUS THE PROBLEMS.

Bob

 

>>
But your idea about a more simple pre-processor, eg. to make user-defined functions possible, is also worth to consider.
>>

Well, I do not know. I think the coders will code and the UI-user will not. If you look at CSound and so on - there are UI-Shells/skins or however they are called. So I think that would be a solution for a simple approach. And I am sure that if KSP (man, they - NI - whine about somebody because he was using the name "Generator" but they just steel names themselves - Kurzweil Signal Processor - that is a real product!) becomes powerful enough there will be front end solutions.
Or all the openGL Shader and other ShaderEngine front-ends.

I do not believe that pseudo-code helps usability.

The people I know are often just scared when they hear the word scripting or programming. KSP is simple enough, really, even easier than HTML or so.

And yes, I also code in textedit and copy&paste, this window is only good for changing some variables on the fly - at least for my limited programming skills. And yes of course, the functionality is not bad, and it is great to have it around - definitely.

I will check out the Kotori-scripting page later - thanks for your efforts - both of you (we are still only three in here, right?)

best

 

Hi again!
For those brave enough to have read this far:
I noticed that the line breaks in the code snippets on the Kotori Language web page were shown wrong in IE. Now I've fixed this.

It felt a little strange to talk so much about something I cannot yet offer, so here is something which is ready to use: a Kontakt Script syntax highlighter. Feed it any KSP script and you get it back with keywords, comments, variables and other special parts highlighted. Bob, I tried feeding it your PCE script and it looked quite nice I think. I hope it can be of use to someone.

kotori

 

I put together a preprocessor so that one can define ones own functions in Kontakt Script. Just to clarify, this has nothing to do with the Kotori Language in the posts above.
Using the preprocessor you can write scripts in the same KSP language that you're used to with the additional benefit of being able to define and use your own functions. I made the preprocessor open source and you are free to use, download or modify it as you like.

Here's an example of what your scripts could look like:

Code:

[i]{Definition of function: }[/i]
function play_octave($note, $velocity)         
    play_note($note + 12, $velocity, 0, -1)
end function

[i]{Now it can be used like any other built-in function:}[/i]
on note       
    play_octave($EVENT_NOTE, $EVENT_VELOCITY)  
end on

Comments and suggestions are all very welcome.

Best regards,
kotori

 

Hey Kotori,

Thanks for posting all this stuff. I won't have a chance to look at it for a couple of days because I've had a truly 'wild' experience here. The power transformer feeding my home conked out yesterday at around noon (so of course we lost our power). But before Edison could get out to fix it, it quite literally blew up, actually splitting the power pole into pieces and sending a high voltage (about 10KV) power surge through our house. Power was finally restored around noon today, but lots of stuff in our home was literally 'fried' and I'm still trying to assess all the damage. As an example (one light bulb in a lamp literally exploded and sent glass fragments all over the house). Anyway, as soon as we get Humpty Dumpty back together again, I'll take a look at the stuff you posted and try to get back to you about it.

Bob

 

Hey Kotori,

Just tried out your syntax highlighter and user-function pre-processor. Pretty Spiffy! Can you furnish these as Windows executables? BTW are you Nils?

Bob

PS I'm still trying to put Humpty Dumpty back together but, things are kind of getting back to normal (still no telephone service (voice) but oddly enough my DSL service is OK). Phone company can't get here for another week yet :-(

 

Hello Bob,
sorry to hear about your troubles. That seems like a major electrical blow! I really hope your computer and other valuable things survived it.

About the tools: thanks. I'm thinking about maybe doing a small editor (based on scintilla) with the preprocessor builtin and package that as an windows executable (I only use cross platform tools, so it would work on mac too, would anyone be interested). I'll see if I can find the time however. And yes, I'm Nils so you can call me that too. Kotori is just a nickname meaning "Little Bird" in Japanese. By the way, funny that you too use Delphi - besides Python that's what I mainly do my programming in.

Nils (kotori)

 

Hi Nils,

Well you know what they say about 'great minds' and all that ;-).

I've never joined the Mac crowd (and probably never will now because I'm getting 'too old to learn new tricks") but I would be interested in the syntax highlighted editor if it was made available as an .exe for Windows XP (assuming it's at least as capable as 'Notepad' -- I'm not familiar with scintilla). Let me know if you ever 'whip one out'.

Bob

 

>>
I've never joined the Mac crowd (and probably never will now because I'm getting 'too old to learn new tricks") but I would be interested in the syntax highlighted editor if it was made available as an .exe for Windows XP (assuming it's at least as capable as 'Notepad' -- I'm not familiar with scintilla). Let me know if you ever 'whip one out'.
>>
Well, for me it is a bit v.v. - I even joined the PC crowd once - well, no need to talk about it.

Kotori, I check out the syntax highlighter and the other, but on the Mac I could not download the .py-file. as I do rather seldomly connect with my music and media partition I also would prefer stand-alone/locale versions. Although, I will still have to check how easily I can manage the Python transition. but synthax between high languages is often not that different compared to the difference to pseudo-code like KSP.

Thanks for your effort.

best

 

I am mac based and am interested in these tools.

 

Hi Nils,

I am at the moment trying your KS Editor. That's a lot more comfortable then that utterly primitive thing Kontakt calls an editor. So, thanks.

But... I did come across something odd on your webpages about it: both the preprocessor and the colorizer seem to do only the same thing: colorize!

Am I missing something here?

Regards, Jan

 

Hi Jan,
I'm glad that you like my editor.
I guess the online colorizer is no longer needed since its functionality is included in the preprocessor. The preprocessor lets you use functions and it's also integrated in the editor and can be invoked by pressing F5.

More information:
Introduction to using functions
Big Bob's legato and vibrato script - read the source to see how functions can be used in real life.

Kind regards,
Nils

 

Ah, I see, one has to put in a function definition AND actually use it.
I suppose nothing like local or global scope for vars has to be considered?

This K2 script language is really rather inflexible, functions are a great help. I'm so very used to client and asp javascript...