Oversampling, ILO tri

Discussion in 'Building With Reaktor' started by Laureano Lopez, Mar 25, 2021.

  1. Laureano Lopez

    Laureano Lopez NI Product Owner

    Messages:
    408
    Just uploaded some macros for oversampling. They're IIR filters -elliptics implemented with allpasses. Rather light for very sharp responses. They came up some days ago in another thread and I liked them enough to immerse myself again in a little storm of number crunching :eek: (without having finished the previous one, of course)

    While testing this, I tried again, and remembered I had already tried, to make an ILO hyperbolic sat with a tri kernel -I remembered because it sort of failed again :D I have to raise the threshold considerably to avoid crackle, which raises a lot the noise floor, especially on high frequencies. It seems like it may be a general thing with ILO tri -so much differentiation making it very sensitive to error... It doesn't get better with 64-bit though, which is a little baffling. I also tried checking both differences instead of their product -it was worse. Has anyone been able to solve this, or had similar issues... or maybe knows I'm missing something?
     

    Attached Files:

    • Like Like x 3
  2. colB

    colB NI Product Owner

    Messages:
    3,823
    I have some elliptic OS filters I made years ago. Not ideal, I'm sure, but I like to use stuff I made myself where possible.
    I used an awesome freebie app (no ads or BS) "Iowa Hills IIR filter designer" to generate the coefficients. Just played with the specifications, watched the fancy graphs for magnitude, phase, group delay etc. and applied my rudimentary knowledge of the meaning of those things :)

    Not sure about how allpasses would fit into that though - how do you use allpass filters for oversampling? or are they for phase correction?
     
    • Like Like x 1
    • Informative Informative x 1
  3. colB

    colB NI Product Owner

    Messages:
    3,823
    I tried to get the fancy ILO to work, but I don't think I ever quite did. My results never gave me a significant improvement over the basic ILO, and I also had threshold problems, so I assumed I'd got it wrong and stuck with the basic one combined with oversampling...
    ...Comparing your code with mine, the only difference is the threshold test. So I'll be having another look into that!
     
  4. colB

    colB NI Product Owner

    Messages:
    3,823
    Hmm, looking back at the paper, it seems that maybe we both have it wrong... watch this space! (but don't hold your breath lol)
     
  5. colB

    colB NI Product Owner

    Messages:
    3,823
    We did both have it wrong!

    You created a single average value and apply that whenever either of the thresholds was crossed. The 'correct' approach (I think) is to test each term separately for ill conditioning, and substitute an appropriate replacement for that term.
    So for the first term it's (f(x)[n]+2f(x)[n-1])/6, and for the second it's (f(x)[n-2]+2f(x)[n-1])/6, and crucially, you only replace when that term is ill conditioned, you don't need to replace both.

    ...my error was that I was replacing with the linear average between f(x)[n] and f(x)[n-1], and then between f(x)[n-1] and f(x)[n-2], although this doesn't seem to make a whole lot of difference... maybe because the waveshaping functions I tested are quite linear close zero... and strangely at least in some situations, this over simplification can reduce crackling... so more investigation is probably a good idea.

    I've attached a modified version of your ilo diff.

    Still not sure if the small improvement over the simple ILO is worth all the extra cpu.
     

    Attached Files:

    • Like Like x 2
  6. colB

    colB NI Product Owner

    Messages:
    3,823
    The real question here though is how you arrived at the F0 and F1 part for that x/(|x|+1) saturator.
    For ILO stuff, I usually stick the function into wolfram to do the calculus, but here it's all complex results - with logs of negative numbers, and pi*i etc...
    I can get a result using a piecewise version where the real part of the solution seems to match your F0, but it would be nice to know how you derived it, and particularly the F1 part?
     
  7. Laureano Lopez

    Laureano Lopez NI Product Owner

    Messages:
    408
    Yays, it does work now... with a pretty high threshold, so we're hitting it often. (I attach my simpler, inefficient version just in case). Can close this little chapter now :D I don't think I'll actually use it -it's quite a hassle for a curve that's not so good anyway. Like, the main point of ILO is tackling the high foldovers that oversampling can't.
    You have to trick Wolfram sometimes. Instead of x/(1+abs(x)) and x^2/(1+abs(x)), I integrated x/(1+abs(kx)) and x^2/(1+abs(kx)) -with both reals, or it freaks out-, then replaced k with 1.
    They make the actual filters :) The idea is that you want your subsample lanes to be full-range, so you make them allpass, but they must add in a way that cancels out above sr/4 (sr/2 of the low rate). Say you have two 1-pole allpasses, (1+az^2)/(a+z^2), with z^2 of the high rate (z of the low rate). If you delay one of them by one sample (multiply the transfer function by z^-1), you subtract w to the phase response. Plotting both responses, with and without delay, for different values of a, gives
    phase.png
    There's a range for which the curves get close at the first half, while the difference at the second half gets around pi. At the optimal position, the sum of both allpasses is an equirippled lowpass. With higher orders you get more degrees of freedom to make the match more precise.

    I traced the breadcrumb trail of papers about this up to a dense 1939 book (?) by some Darlington guy, then focused on the snippets I could get from each one. Most of what I did is covered in "DSP schemes for efficient interpolation and decimation" by Valenzuela and Constantinides. There's a couple of papers by some Finns that seem to give a better implementation for higher ratios, but it's paywalled everywhere.
     

    Attached Files:

  8. colB

    colB NI Product Owner

    Messages:
    3,823
    Yes, It's pretty useful for a few things, but maybe not this particular shaper.
    Sorry to drag it on, but I feel compelled :p
    I realised that my initial analysis of the problem was limited - the issues with your original code were maybe slightly more subtle. What you were doing was taking data from three consecutive clock ticks, and generating a single value from those three values (two differences squared, then multiplying the squares).This does mean that if either is zero, then both will be replaced, but here are other things to consider:
    What if one of those diffs is close to zero, but the other is a large value? then the final value can be larger than your threshold, so both the terms will be processed. In this case, one of the divisions can result in a very large number, causing numerical errors due to processing values of extremely different scales (floating point weakness). Testing the two individually prevents or reduces this issue.
    A more general problem is that of considering the consequences of a process that depends on results of calculations on different ticks. Obviously not unique to this type of process, but definitely more tricky to handle than just a basic shaper or folder. So if one of your diffs is zero, then you go for the average generated from all three points... but we really only want to replace with an average when the function is close to linear and close to zero. But a signal with discontinuities in it could have a diff of some large value - maybe 10 or 50 or more if you are using lots of gain in a clipping process - followed on the next clock tick by a very small diff - zero or close to zero because you hit the top (or bottom) of a filtered square wave or saw... On that tick with the large diff, you really want to be processing the ILO properly, not replacing it because of it's unruly neighbour!
    ...You could end up using the linearized approximation exactly at the point where you would want the ILO... big picture view I guess :)

    This is great - thanks. Wrestling wolfram into submission FTW.
    That is such a cool idea! but is it really better than just using a low pass :D... apart from the fact that it's cooler, which already makes it better I suppose. Is it cheaper?
     
  9. colB

    colB NI Product Owner

    Messages:
    3,823
    :mad::mad::mad:
    This pisses me off so much.
    about 15 years ago, I worked in a university library, and was involved in the process to choose which digital libraries to licence. No way we could afford them all, so I had to compare all of them to work out how to get the best coverage of all the journals we wanted... ridiculous state of affairs, and it's getting worse.

    So much of the funding for many of the papers comes from the government and is paid for by regular taxpayers, and the peer review is from academics who are not paid for that part of their work, and yet these private companies make huge profits with their paywalls. It is literally a huge scam.

    I wish I still had access though - because although it was a business library, the licences also gave access to programming and dsp related stuff - I learned a lot :cool:
     
  10. Laureano Lopez

    Laureano Lopez NI Product Owner

    Messages:
    408
    Yup, that's what I thought. It ends up replacing too many values, that couldn't be right. We're still not doing what the paper says though -it's supposed to be the function of the average, not the average of the function, also for ILO rect. I plotted both and at least in this case they're pretty close -maybe it's a good approximation in general. I tested with the function of the average and it didn't let me lower the threshold anyway :rolleyes:
    It is way cheaper than a direct form version of the lowpass. Say the "x4 lo" combination in the tests: four allpasses for the first stage and two for the second:
    x4lo.png
    If you expand the whole thing to one rational function, it's order 23 with an odd denominator, so 24+12=36 muladds, minus 1 add. That needs to run four times, so it's 284 ops. If you keep it multistage and only expand each factor, it's 9th order on z^2 for the first and 5th order on z for the second, both with an odd denominator, so 10+5=15 and 6+3=9 muladds, minus 2 adds. The first stage is doubled and the second is doubled twice, so 29*2+17*4=126 ops. The allpass based version is six 1-pole allpasses, with the second stage ones doubled, so eight in total. Each one uses 4 ops, so it's 32 ops. The 4th order Chebyshev in the too-b overdrive has 5/4 terms, so 9*2-1=17 ops four times: 68. We still need 32*2+4=68 ops for the whole up/downsampling, which is more than the 22 used by Efflam in the x4 clipper, but that's a very rough filter, with a large cut in the passband and a very gentle slope in the stopband.

    Of course that may be enough for waveshaping -you're making a lot of noise anyway, so a bit of loss here and not so much there isn't much of a difference. I limited myself to a bandwidth of 3/7 SR (18900@44100), with ~60 to 120 dB of stopband gain. I could make some more relaxed versions, with a wider transition band and less attenuation. The least amount of ops allowed by the method for a complete x4 chain is 28, and that's still a 9th order lowpass.
    It's pissing me off too lately, because most of the stuff I can't find comes from IEEE, who charges 33 dollars for each article, which at the current rate (just did the math) is like 7% of my monthly income :D
     
    Last edited: Mar 26, 2021
  11. colB

    colB NI Product Owner

    Messages:
    3,823
    Yes, that's one I was already aware of. Most times the average of the function is fine, but I have one where I've edited the inside of my ILO process module to change that because it was making a difference.
    Ok, damn, now that's another complicated, math heavy thing I need to wade into :rolleyes: thanks :D
     
    • Funny Funny x 1
  12. Laureano Lopez

    Laureano Lopez NI Product Owner

    Messages:
    408
    The concept is more or less simple, the complicated part is getting the coefficients. The Constantinides paper gives two methods. One is analytic and uses elliptic functions with some auxiliary intermediates. Elliptic functions give me headaches, so I went the numerical way. It's an iterative process -get an initial guess, then NMinimize repeatedly until it looks right. I used a different minimization to the one they suggest because it didn't work, probably because they used a different algorithm (they ran the whole thing on a CDC, it was all very custom back then).
     
  13. Paule

    Paule NI Product Owner

    Messages:
    7,325
    Laureano, who is the author of the finn papers?
    colin, can I use the foldover in your MS-20 Filter?
     
    • Like Like x 1
  14. Laureano Lopez

    Laureano Lopez NI Product Owner

    Messages:
    408
    Renfors and Saramaki. It's a pair of IEEE papers from 1987, "Recursive Nth-band digital filters". I found a book by Renfors which seems to cover this, maybe without details, I'll see. They're Nokia engineers -as usual, all this stuff comes from communications... The 1939 book was a Bell System publication, heh.
     
    • Like Like x 1
  15. colB

    colB NI Product Owner

    Messages:
    3,823
    Very nice thanks.

    What software did you use for these aliasing plots? Same as what Efflam used for the original ILO release pics. But what is it?

    aliasing plot.PNG
     
  16. colB

    colB NI Product Owner

    Messages:
    3,823
    Just a quick thing for you to check.
    Your oversampled ILO examples are connected in parallel, but isolated.
    ILO oversight 0.PNG

    This means that each lane uses it's own previous samples, so the differentiation in the ILO is running at the non oversampled rate in each lane, but offset by 1/OSrate samples.

    What I do is connect up the OBC from each z^-1 module across the lanes - like this:
    ILO oversight.PNG

    I'm not sure how much if any difference this will make, but it would be nice to see a comparison on one of those snazzy plots between the simple parallel approach, and the interconnected approach.
     
  17. colB

    colB NI Product Owner

    Messages:
    3,823
    Which MS-20 filter? and what do you mean by foldover?
     
  18. Laureano Lopez

    Laureano Lopez NI Product Owner

    Messages:
    408
    RX, with a log scale and a Kaiser-20 window. The view starts at 250 Hz. From the colors, I think Efflam used Audition. The test is different too. Efflam ran a clipper with a gain of 40. That's right for testing ILO, as the point is what happens with the high foldovers. I used an hyp sat with a gain of 8. We already know that nothing happens with the high foldovers, we want to see what happens with the lower ones.
    Right! I will test them connected and report.
     
    Last edited: Mar 26, 2021
    • Like Like x 1
  19. Paule

    Paule NI Product Owner

    Messages:
    7,325
    The MS-20 filter is iOS. Oh, do I misunderstood the ilo?
     
  20. Laureano Lopez

    Laureano Lopez NI Product Owner

    Messages:
    408
    It's (sort of) worse -I think it makes sense though. ILO filtering is not very sharp, and running it at x4
    ilo.png
    the 1 there goes from 22050 to 88200, so it has less effect.
     

    Attached Files: