REAKTOR Structure Design in Primary and Core

Discussion in 'Building With Reaktor' started by herw, Aug 15, 2019.

  1. herw

    herw NI Product Owner

    Messages:
    6,307
    If you want to create a large REAKTOR project, it is important to consider time!
    Time is important for two aspects.

    1. A large project needs time, not only 1 day :p , 1 week :cool: but months or years :rolleyes:. So you have to consider that you are working with several projects maybe with interuptions etc..
      The structure of primary and core have to be readable and reconstructable always.
    2. You use many macros often, so you need a personal framework of macros and structure parts. Recurred structures support routine and developing speed.
    So i open this thread for all interested REAKTOR users who want to demonstrate their workflow in designing REAKTOR structures.
    I would like to see different workflows.

    I hope we all will get some new ideas for our own benefit.
    HAVE FUN with REAKTOR :)
     
    Last edited: Aug 15, 2019
    • Like Like x 3
  2. herw

    herw NI Product Owner

    Messages:
    6,307
    ok - i will start with a short post about

    short and long labelings

    long labelings (mainly inputs, outputs, quickbuses and send/receives are useful, to understand a structure. I show you an example where it is useful to use short names (btw i am a friend of names with 3-10 alphanumerics).

    Cryptic labelings of inputs and outputs


    an example:

    You want to create a large ensemble, say with several oscillators, filters sample players, envelopes, sequencers etc. . You have built a mix of primary, eventually core and want to use your own DSP macros (means corecells) etc..

    So you have many GUI sources (knobs, switches, button etc.) and you want to use this in core. You reach the point, that you say: i want to use core and use primary (mainly) for GUI only; then there is the problem to send many GUI events into core. You don’t like a wire harp. So you have to use a multiplexing system.

    a bus GUI to corecell.png

    There is one: partials framework, BUT it has cryptic labelings:

    labeling of partials framework.png

    It is not important to understand the labelings c#, #, id and $ here.

    It is important the use. I myself understand the short labelings, because it is a main part of the GUI/Corecell connection. So it is not useful to have long descriptions like container number c#, number # of events, identifier id and event value $.

    But if someone want to use this too?

    As it is part of a common framework, it is made for many users. If they decide to use it too, they read the documents and will understand.

    If i want that other users use the same framework (means the macros i built with it too) it is not useful to rename the labeling. No one can read the personal structure.

    So i say: if you use a common framework and its labeling, you have to accept the names for common use.
    Nevertheless simple and meaningful names are a good choice from developer's side.

    BTW i use here short names c# and others because it is part of my own framework and i would describe c# if someone want to use it.

    So short labelings are useful, if they are described clearly once and then used often and for many users.
     
    Last edited: Aug 16, 2019
    • Like Like x 2
  3. Laureano Lopez

    Laureano Lopez NI Product Owner

    Messages:
    187
    well, i'm not sure i have a proper workflow heh. coding is something i do between projects (or when i'm too sick for other things, like now). much of it is quite ad hoc. i don't make much electronic music, and the little i do is mostly for myself, but i mix and sometimes produce, pretty much whatever comes to my hands. so much of what i use everyday are small utilities: a simple gain (amazingly cubase doesn't have a plugin for mid-chain gain), pan/autopan (mainly to have three pan rules), various mid-side mixers, an adsr (to trigger an audio source, with an inverse option for making cuts), crossovers (to work on the lows and highs of a track in separate busses), midi-controlled filters (to make eqs relative to pitch), a xfade with lfo (to xfade between tracks), etc. often i'm in the middle of a mix and there's something i can't solve with cubase or any plugin, so i hack a solution in reaktor and save it with the project -after i finish the job, some of those get revised and added to the general folder. i have another folder for core macros, which at this point is quite full of garbage, but also has the little pieces i use all the time (latch and merge, write-read-write, counters, timers, conversions, filters, waveshapers, etc), and some bigger ones i use occasionally. when i started coding it was all quite chaotic, and it took a long time until i got some maths (i still get dizzy with some basic things, like pole-zero plots). my formal education in these things is messy and incomplete -most of the relevant stuff has been self-taught and i learn things when i need them, so it's difficult for me to pre-plan because i may not have the least idea. most of the stuff i made some years ago i don't use anymore, either because it bored me or it's too ugly or both. nowadays, my process is mostly paper-reaktor-mathematica-reaktor in a loop, with long stretches of googling. i sketch some idea in diagrams, lay some dummy core macros, try to code a part that seems immediate, get stuck, go make more diagrams or solve some math or ask the internet. most ideas don't come in orderly fashion -my first versions are much more convoluted than the later ones. i let myself build additively until i have a minimally functional result, then as i understand how it actually works i make things more compact and the structure starts to "unveil". at that point sometimes i make a thorough refactoring. i like my code neatly ordered and aligned, i prefer wires to quickbusses, i mostly use short names (too short sometimes), and i tend to think of processes as geometric relations, so i'm generally not satisfied until i look at the code and get its logic from its shape. this doesn't happen in a single magical moment, and often when i'm working on some part i find some other one that always bugged me but i had forgotten.

    to put an example of this shape thing -these are formulas for 1-pole and 2-pole allpasses
    ap1.png
    and their implementations
    ap2.png
    there's something satisfying for me in that symmetry, and even though it's there already in the formulas, i don't see it until i draw it. there are other ways to place that code, but when i place it this way it makes an intuitive sense that helps me understand. it may not be relevant in this case, but in many others it triggers the mental leap where i see a coherence and start to make a structure from an arbitrary chain.
     
    Last edited: Aug 16, 2019
    • Like Like x 1
  4. Paule

    Paule NI Product Owner

    Messages:
    4,903
    You can use the info tab in preferences for longer text descriptions.
     
    • Like Like x 1
  5. herw

    herw NI Product Owner

    Messages:
    6,307
    Simpliticity, clearness and symmetry are important attributes to have found the right structure.
    symmetry2.png
    Here are longer names of quickbuses and macros helpful.
     
    Last edited: Aug 16, 2019
    • Like Like x 1
  6. herw

    herw NI Product Owner

    Messages:
    6,307
    Replications

    Creating a large ensemble is time-consuming. Often there are structure parts a number of times. So there is a point to think about a personal framework. Why should i create a personal framework?
    NI's factory library includes several frameworks: TF toolkit, ZDF toolkit, (envelope) toolkit in core or as mentioned above partials framework in primary.
    If you look into the primary and core filters sections you find different filters but always some common inputs like pitch, cutoff frequency, resonance and in.
    Bild 8.png Bild 9.png
    So you don't need to think about it because the principle setting is always the same.

    A hardware modular system like an Eurorack System is similar:
    Bild 6.png
    The front panels have different arrangement, but you know what A, D, S, R , Lev, Frq, Q mean. You can add several modules into a prepared rack without thinking about power supply and you can connect every output to every input.

    The modules are different but there is a framework: same connections on the backside to electrical power supply, common pitch or gate bus:

    Bild 7.png

    REAKTOR is a modular system.
    When i realized this in my own work i started to build a framework and - which is very important for me - i create my ensembles with the intention to support a framework.
    Maybe there are simple macros which you always need: multiple adder, multiplier, special GUI-elements like special pull down menus, knobs, switches etc..

    The best awareness for me was to realise similar signal flows. In a complex ensemble you have to have always the complete signal flow in mind. You know that my project is a modular concept, which lends itself to use similar signal flows.

    I am fascinated of building an ensemble with replicated working and thinking processes.
     
    Last edited: Aug 16, 2019
    • Like Like x 3
  7. colB

    colB NI Product Owner

    Messages:
    2,894

    You are right that it's not useful to rename the labelling used in the partials framework - it's useful to leave it because then everyone knows that when they see a macro with those weird cryptic labels, it is a component from the partials framework, or if they see other code with $ or ! or -+ inputs or outputs, it's an indication that there is some partials framework library code in use somewhere in the structure.

    But the same would have worked with e.g 'data{pf'}, 'event{pf}', 'mux{pf} etc., the same would be true - it would be clear that partials framework was in use, but also, occasional users of partials framework, or someone just maintaining code wouldn't have to wade through the partials framework manual every time to remind themselves of what those meaningless arbitrary labels mean - because data, event and mux already have universally understood meaning.

    that doesn't mean they are good - just that if we use the library, we are stuck with the bad
    design choices it forces on us.
    No, short labellings like those in partials framework are not useful - labels that are arbitrary and have no descriptive value are bad design - it's always better to use labels that have some descriptive meaning. Sometimes it's difficult for various reasons, but it's always better.
    e.g. multiplex is often shortened to mux - this is a common and well known abbreviation. so a library that has a lot of connections for multiplexed signals could label multiplexed connections 'mux', or 'rawMux' etc. - a short identifier with some expressive meaning that many engineers would understand without having to look at a manual - that's a win. Instead something like {b} was used. This has no descriptive value at all - it is an abstract identifier with no meaning.

    Short identifiers were traditionally used in software even though they are a bad idea because memory and storage were scarce and expensive resources, and screens were small with low resolution. The same is not true now, so there is no legitimate excuse.

    So definitely use the meaningless short identifiers from partials framework when using partials framework, but please don't see them as good design and start creating even more code with meaningless arbitrary labels.
     
    • Like Like x 1
  8. herw

    herw NI Product Owner

    Messages:
    6,307
    Perfective Maintenance (GUI)

    GUIs seems to be very important for many REAKTOR user’s. They want to understand a GUI at once without reading any manual or hints.
    For those it is important to build a GUI which is very common.
    Here are two examples which everyone understand.

    gruene Fee (green fee) (R4-ensemble and still running in R6.3)

    gruene fee GUI.png

    As it is not in the user library i add the file here (see below). It was only an experiment for me, so don’t set the standard of R6.3.
    A very simple basic GUI. Everyone understands it at once because it is NI-standard from early days (REAKTOR 4). I stopped developing because drawing audio table was a dead end.

    greyhound

    Another layout (modular mini (see later) was published before), so i used a similar layout but no patch cables. The principle layout is still common. There are patch cords realised by pull down menus. Nothing is spectacular.

    greyhound.png

    download below
    It was one of my first experiments with event bus. Then Max Zagler gave me partials framework for testing and i stopped my own bus system. Nevertheless it is an interesting idea because it based only on a bus-send-reveive. It sends only changes of a source. But it is only a eventbus, so i cannot use it for audio signals.
    Have a short look inside if you are interested.

    I myself like to invest time into a basic idea which comes along with me over years - from 2004!

    The idea of a modular system with independent containers which you can add and remove is fascinating. I am always in view of perfective maintenance for my work and user’s work.
    So the basic concept is, that i am able to get the idea and the progress always.

    I don’t want to go to much into details now, so only a comparison of the GUIs

    modular-mini (GUI)

    modular mini.png

    a very small modular synth. As it is no more in the user library i add it here (download below).

    Have fun because astonishingly it runs with R6.3 without any changes since 14 years. In 2005 R5 was published and it was perfectly for the first REAKTOR patching system on screen. In 2004 i used a first version with pull down menus for connections (REAKTOR 4). Another main thing was the new core level. I used it in modular mini only very rudimentary.
    new wire: first right-mouse-click on output (coloured buttons), then right-mouse-click on input (grey buttons).
    delete wire: double-mouse-click on input.

    The following modular ensembles and frameworks used similar GUI (wires) so the user knows what to do :)
    You can downoad the in the user library.

    modular m1

    modular m1.png

    modular mini 2

    modular mini 2.png

    RUHR

    RUHR.png

    EMSCHER

    EMSCHER.png

    The GUI changes only a little bit over the years. A modular mini user understand the actual GUI in 5 minutes i think. So it is common now i hope ;)

    Nevertheless there is the standard GUI from NI which works since many years. There are some additions but the principles are the same. Most users never look in the how-to-start-manual.

    Although i don’t like REAKTOR Blocks the idea is brilliant. My antipathy is more inside (monophonic, CPU usage etc. ), but that's not my point here.
    The GUI framework of REAKTOR Blocks is nice and attractive.

    and the future?

    The actual REAKTOR-GUI is frumpy.

    The future is use of modern GUI aided by browser, ipad etc.. Zooming is for me my main wish. I have in mind the GUI of C15 from Stephan Schmitt. Would like to see it in REAKTOR.

    So that are my two cents for Perfective Maintenance (GUI) :)


     

    Attached Files:

    Last edited: Aug 18, 2019 at 4:25 PM
  9. colB

    colB NI Product Owner

    Messages:
    2,894
    I don't think any of these are frameworks - including the partials framework!
    They are libraries/toolkits. 'Partials toolkit' would have been a much better name.

    A framework IMO is an abstract system of meta-code - a structure to hang other code onto. It doesn't actually contain much if any actual concrete code.
    As a result, it's not possible to create a framework in Reaktor. To create a framework, you need a programming environment/language that allows some form of meta-programming - templates, abstract classes, lambda expressions, whatever. Reaktor is so far away from that is not even funny :)
     
    • Informative Informative x 2
  10. herw

    herw NI Product Owner

    Messages:
    6,307
    thanks for clarifying, difficult to define for graphical programming language like REAKTOR.
    So is my modular framework a framework?
     
  11. colB

    colB NI Product Owner

    Messages:
    2,894
    That's cool, but can we keep this about structure design in primary and core?
    GUI design is a whole other thing with enough different approaches and competing philosophies to support multiple separate forum threads.
     
  12. herw

    herw NI Product Owner

    Messages:
    6,307
    Yes that's right. I thought about it too. But here it is important, because the wiring is combined with a new structure inside. So before i go inside i want to add these. Otherwise the post would be too long - sorry. No one wants too long posts.
    The ensembles are added to compare the inner structure later and as a little bonus.
     
  13. colB

    colB NI Product Owner

    Messages:
    2,894
    I've got a nice (I think) and generally useful approach to basic GUI implementation in Reaktor.
    I use multiplexing, but it's much simpler than Partials Framework approach.
    Each part of the GUI will have its' own separate multiplexed connection that is easy to route within the core structure with a wire, or a scoped bus distributer.
    So Primary looks a bit like this:
    mux1.PNG

    And core looks a bit like this:
    mux2.PNG

    I had a system where it all went down the same wire and was extensible etc, but it was so much more complex that maintenance was an issue.
    This approach is very simple to use and is a nice compromise - it cuts down the connections from primary by a factor of 8 (roughly) so for medium sized monolithic synths and other similar instruments is works very well. I have a 16x bus size the rarely gets used - the 8x one is almost always the best fit.
    Inside the primary macro looks like this:
    mux3.PNG
    As you can see its pretty simple - I just attach whatever GUI device (or other event generating widget) to a ctrl input on a mux module, and then in core, label (or check later very easily) which bus output that sends to...

    The bus reader I'm using wasn't the first I did - that was a single level tree that worked fine. Then Partials framework came out and I tried a version of the nested tree used there, because it seemed cool - a bit like a recursive function. Works just the same so I've kept it, but if I was doing it now, I would use a basic single level tree - because its simple rather than cool, and I've learned that that is always the best option.
     
  14. herw

    herw NI Product Owner

    Messages:
    6,307
    nice and similar to multiplexing from partials framework. Multiplexing is necessary when you use many GUI-elements. Automatic adressing makes many constant (addresses) obsolete.
     
    Last edited: Aug 18, 2019 at 8:45 PM
  15. colB

    colB NI Product Owner

    Messages:
    2,894
    Completely automatic addressing also makes things harder to just throw together, you need more planning, and it is also potentially more difficult to debug and maintain.

    My approach is a half-way compromise. Within the individual mux/demux pairs, the addressing is automatic. However these mux/demux pairs are stand-alone, they are independent of each other. Not so good when developing something like Emscher where you want a flexible modular system requiring no extra coding, but absolutely fine for most Reaktor projects where you just need something to tidy up all those connections to GUI mechanisims. And easier to learn to use correctly, and easier to replace in the future if the need arises.

    In the UK we say 'Horses for courses' - meaning different solutions are better for different situations!
     
  16. herw

    herw NI Product Owner

    Messages:
    6,307
    The benefit of this system is that you only need two events to send the value: address and value.
    So the messages are [id,value].
    Partials framework is made for different long messages, so in partials framework it is [id, 1, value].
    If you want to use only one wire you can send messages like [m,2,id,value] where m is the identifier for the macro m (ctrl vco it, ctrl vcf, etc.) it is sent from. But as you mentioned this is more useful for larger ensembles.
     
    Last edited: Aug 18, 2019 at 9:48 PM
  17. colB

    colB NI Product Owner

    Messages:
    2,894
    Yes, good points.
    I'm not sure if the usefulness is related to the ensemble size though. It's more to do with the use case.
    e.g. if you are sending streams of similar data to arrays of partials, it makes sense to package them up in a run length encoded style where you have a type identifier and a count, then overall you save potentially a lot of cpu, but if you are just processing Random access stuff from a GUI, then sending each data value with it's own ID is going to be more efficient, because the cpu hit from GUI events is minimal, so whatever makes the system lighter (less code) is better.

    One thing I'd like to know in more detail and haven't researched is how much if any cpu is saved by having e.g. 100 events flow from Primary to Core through one wire vs all of them going through separate inputs. The important thing here I think is binary size, so I guess one would need some decent analysis tools to be able to look at the memory usage of Reaktor with to different versions of an ensemble running. Not sure if windows 10 home can tell me with enough detail, or if I'd need pro, or some third party tools... Maybe someone else has already done the work?

    I wonder if a MIDI note style approach might be possible where you just assume that the type and address of a stream is the same until you receive a message that changes the type and address. That way, there is no count but you might need two connections from Primary. Although within core they would become a single bundle distributer. Of course in modern Reaktor, you can send the type, source, destination, and value in parallel through a bundle connection...
     
  18. EvilDragon

    EvilDragon Moderator Moderator

    Messages:
    14,986
    AFAIK Task Manager and Resource Monitor are exactly the same in Home and Pro...
     
    • Like Like x 1
  19. sellotape

    sellotape NI Product Owner

    Messages:
    337
    I'm using this in my current project. Here is the demux where 'M' is the master-ID for the macro an 'S' the slave-ID for the control. Values are send to tree routers in the corresponding macros. Adding or deleting controls is quite easy this way.
    upload_2019-8-19_0-25-54.png

    I think you don't have to care about that even if your machine is 20yrs old. It's just some compares that are fired one by one and i can keep switching thru the snaps without seeing any peek of cpu usage.
     
    Last edited: Aug 19, 2019 at 12:43 AM
  20. colB

    colB NI Product Owner

    Messages:
    2,894
    ...more somewhat cryptic Partials Framework style identifiers... I'd be all "Mst and Slv.... or more likely MasterID and SlaveID - I mean, why not just tell it like it is ?
    Seems like a good solution though. Are the [M] and ids packed into the same value, or kept separate? Shouldn't matter for GUI controls, but might make a difference for some stuff depending...
    EDIT: Lol, even BBcode misunderstands your cryptic identifiers, I think that's a sign ;)