Event table shared memory problem

I'm having a problem with event tables.

In order to avoid event loops in the text entry processing and patch recalling code in digit8, I'm using event tables sharing the same memory to pass data 'upstream'. This works great, and was the only way I could get the damn thing to work at all.
This approach is required because during manual editing I need to store any updates automatically in a snap array, so the snap array needs to come after the update process. But during patch recall, I need to read from the snap array into the update process, so the snap array needs to come before the update process. oops...
So I use the shared memory of the event table to break the cyclic dependency. And it work great. Until...

The problem is that the data is not only shared amongst Event Table clients in one instance of an instrument, but amongst multiple instances. This means that if someone loads two instances of digit8 at the same time and tries to use a different patch on each one, it all falls apart. E.g. load a patch on instance A, then a different patch on instance B, then if they try to edit instance A, it will respond by clearing it's memory and loading the patch from instance B...

So is there a way to prevent different instrument instances from sharing Event table memory?
Or, is there an alternative way to dodge event loops and still break the cycle?

cheers

Col

 

since neither event tables nor core arrays store snapshot data, would it be possible to use arrays?

i think this whole issue caused a minor bit of spaghetti code in automata but i can't remember.

 

No, arrays are no good because I need this in the primary layer. I'm using the Event Tables like I would use an OBC memory connection in core. Basically to create an event z-1 in primary. It might be possible to use convert to and from audio and use the audio unit delay in primary, but that would break some other stuff, and also make the spaghetti even worse.

I suppose if it becomes an issue, I can make a version with two instances, with manually edited Event tables that share a different internal file. Cant see anyone wanting to use more than two instances at a time - one is enough for me!

It would be nice to find an elegant solution though.

 

you can do that with a single delay module, right? or are the outputs quantized to the control rate when using events? i'm not sure.

EDIT - seems to delay 3 samples even when you tell it 1, strange... in fact even with a delay time of 0, it seems to delay 2 samples... something is probably wrong with my testing. i guess the timer module has a max freq. of half the sampling rate, makes sense. also makes it difficult to test, i'll make a core cell.

EDIT 2 - seems to delay 1 sample when delay time is zero, according to core, again, there may be something wrong with my tests...

 

Number of 'samples' isn't really relevant here. I need a setup in primary layer using events (not audio) that lets me use feedback without creating event loop errors. I can do this with event tables, but I get this problem of multiple instances of the instrument sharing the same memory. This is trivial to achieve in core, but I can't use core because the feedback needs to pass around a snap value array, and you can't pass OBC connections out into the primary layer.

One possibility is to use a large event table and find a way to force each instance to use a different section of the table. The difficulty here is to generate unique instance ids that always start at a fixed value. So far I have been able to generate instance Id's, but not starting at a fixed value... Might be possible to use hashing...

I really hope there is a simpler solution.

 

on initialization you could search the table for the first empty y column?

 

I think I have a solution. Not thoroughly tested, but seems to work.

I decided that a limit of a maximum of 8 instances would be way more than anyone would ever need, so I changed all the Y sizes in the tables from 1 to 8, then use the instance ID I have and mod it with 8. That way, I don't need the IDs to have a fixed start value.

I am still a little concerned that it works 'by accident'. Reaktor Primary layer is so messed up that this could be working because of some combination of the order that I wired it up in... But if it's still working after a decent amount of time and a few other tweaks, then I'll go with it

Edit: FWIW, quite a lot of the Primary code I've hacked together recently only works because I ditched the primary order module and used a core cell with the input wired straight to multiple outputs. This allows you to control the order of events like a primary order. The difference is that init events maintain their init status no matter which output the come out of.
The instance id generator depends on this, and so does some of the digit8 data entry code.



Thanks for your help.

Col

 

funny, i was wondering if that would work recently (the core order module, that is).

the order module has a funny initialization routing, as explained in the module. only two of the outputs are 'useful' in a lot of cases and that's stretching it.

 

Yes, the problem with the Primary order is that the top output sends an init event, but the other two don't. So you have to set your code up to assume that init status gets lost, and if you're stacking order modules, you can't use the 1st output (if you want to keep your sanity) just to maintain consistency.
Using a core order, you can use init event filters on any of the outputs and they work! you can also have 1 order module with lots of outs, so you don't need a huge stacked tree of order modules.
I still use the primary version where is works, because I assume its more efficient. Haven't tested that assumption empirically though.
Here's my instance id generator:

The two Event tables share memory in every instance of the instrument. The memory is read first, the value read is sent out as the ID, and is also incremented and stored back in the memory by the second Event table.
Using Primary order modules, this doesn't work, because the lower two outputs don't send during init, so initialisation must be completed in all instances before the first instance can finish processing this modules.
Using a core macro as an order module, all init events are passed through during initialisation, so I have a better chance that this ID generator module will complete in one instance before it starts in another... It doesn't matter what order they go in as long as the process is complete atomically. I'm not sure if the Event Tables are read and written during the init cycle - haven't tested that. I do know that it works using a core order, but doesn't work using primary order modules.
Even if the module instances did process in the correctly using primary order modules, the IDs wouldn't be available during initialisation, so it would be pretty useless!.
This setup also failed to work when I tried sending out the id using send/receive modules but did work using ICsend/ICreceive.

And of course, Reaktor doesn't like being pushed into doing stuff it's not familiar with - I've had more crashes to desktop today than in the previous month.

Cheers

Col

 

Perhaps it is a good idea to use the ACEW.
ciao herw

 

I use it a lot. But in this case, I didn't need it to get things working satisfactorily. ACEW is great, makes debugging possible in reaktor

 

Hey Col,
you run into a trap.
The problem with the core order is, that it doesn't work on Init step 2, as everything is synchronous...
And becahse of that a Primary Order Module with multiple Outport 1 doesn't make sense.



For the event table, it seems, that the table is read before events are written to it at init step2. That also seems to be true for a event table receive, which is located behind (Core ordering) the event table send.


cheers, mark

 

What you´re trying is really interesting. I´m not that familiar with shared tables, but very surprised that they work over different Reaktor instances, as you say! I also think the ID solution is working, but it arouses new thread issues on another level.
The new ID for a instance has to be written to it`s event table before another instance (CPU thread) could interleave that process. Else both have the same ID. I assume, that the write process for the tables is executed directly after step 2 and before step 3 of init. I´ll check that out.
That means at a Sequencer Project Load with several instances of your Ensemble, they have to start running one after another. At least for the processing time of init step 2.
I´m shure you´re aware of that. Just thinking loud...

EDIT:
as assumed, init step 2 events are written after init step 2 and before step 3 to the shared memory:



EDITIT:
Table memory is also written before Tempo Info Module initializes.
(It initializes between step 2 and 3)

 

indeed, now that you mention it, there must be some really cool ways we can take advantage of this behavior.

 

I've only been testing multiple instances of the same instrument in one instance of Reaktor. I've not tested multiple Reaktor instances yet. So far I've not had any problems. If there is a threading issue, it might not effect me in this case because I'm not using the instance IDs in any audio processing. It's being used for gui and snapshot management.

 

Here's one to chew on, the three tables are clients of the same memory.

Look at where the values are written to, and where they are read from. Something weird is happening.

 

Guys, i have to read more carefully. Sorry for that!

My test ensemble in Post 13 is completely wrong too. It doesn't allow any conclusion about the timing of write processes. That assumption was really naive.
Your last examle, Col, shows that at init written values also can be read out at init. And it shows there is an order of the event tables. Now i think it must behave this way. The shared table must be written (and read out) sequentially - at simultaneous init. There is no "merger" or something, that filters multiple written values to that one table index.

I fear it is the core order...
or the order of inserting table modules.
Do you agree, that the table is always read before written (for one module at init)?

 

After playing about for a few minutes, I have discovered that:

changing the order of the outs of the core order doesn't effect this particular example at all!
swapping the two event tables does.

This suggests that the processing of the tables happens after all the events from the core order have arrived, so their order doesn't matter in this case. And as you guessed, it also suggests that the order of creation of table modules does effect the processing order (makes sense really).

What seems to be happening is that there is some sort of internal intermediate code that handles multiple clients of shared memory at init time that does some sort of event queuing...
Intuitively, I would expect it to process all writes in table Am then in table B, then process all reads in table A then in table B. However if it did this, all the events would be either 7 or 33.
What actually happens is some sort of odd 'criss-cross' effect where the value written at Table A is output by table B and the event written at table B is read at table A, then finally the memory is set to the value written to whichever of them is processed last at the write stage...
Needs more work to understand.

 

i think what is read at table A is simply the value which was written to its memory before init.
At table B.