Anything comparable to ANN?(logic module?)

Cwejman, Doepfer, Erica, MakeNoise, Mutable instruments, TipTop Audio, Analogue Solutions, and much more! The world’s most popular format.
Be sure to look into MANUFACTURER SUB-FORA as well..

Moderators: Kent, Joe., luketeaford

Post Reply
User avatar
Severed head
Ultra Wiggler
Posts: 923
Joined: Tue Sep 24, 2019 9:24 am

Anything comparable to ANN?(logic module?)

Post by Severed head »

What logic modules do you folks find comparable to ANN
(ARC artificial neural network)
WTB:-Mac recordings software from 2008/9
:help:
etckla
Veteran Wiggler
Posts: 670
Joined: Wed Aug 12, 2020 1:12 pm

Re: Anything comparable to ANN?(logic module?)

Post by etckla »

Ornament and crime hemisphere suite has Threshold logic neuron, Boolean logic and comparator applets
User avatar
vidret
Veteran Wiggler
Posts: 682
Joined: Sun Jul 30, 2017 10:26 pm
Location: Skogen

Re: Anything comparable to ANN?(logic module?)

Post by vidret »

Rectifiers for your ReLU activation functions and absolutely MASSIVE bipolar VCA matrices with offfsets for your weights and biases :lol:
User avatar
Severed head
Ultra Wiggler
Posts: 923
Joined: Tue Sep 24, 2019 9:24 am

Re: Anything comparable to ANN?(logic module?)

Post by Severed head »

vidret wrote: Tue Sep 20, 2022 3:06 am Rectifiers for your ReLU activation functions and absolutely MASSIVE bipolar VCA matrices with offfsets for your weights and biases :lol:
I can’t even translate this let alone hope to implement it.

—-
O&C probably has tons of uses cases but scuba gear isn’t my thing
WTB:-Mac recordings software from 2008/9
:help:
User avatar
jheronymo
Ultra Wiggler
Posts: 753
Joined: Sat Nov 22, 2014 4:11 pm
Location: Vancouver, BC

Re: Anything comparable to ANN?(logic module?)

Post by jheronymo »

vidret wrote: Tue Sep 20, 2022 3:06 am Rectifiers for your ReLU activation functions and absolutely MASSIVE bipolar VCA matrices with offfsets for your weights and biases :lol:
Analog machine learning ftw :goo:
User avatar
vidret
Veteran Wiggler
Posts: 682
Joined: Sun Jul 30, 2017 10:26 pm
Location: Skogen

Re: Anything comparable to ANN?(logic module?)

Post by vidret »

Severed head wrote: Thu Sep 22, 2022 8:31 pm
vidret wrote: Tue Sep 20, 2022 3:06 am Rectifiers for your ReLU activation functions and absolutely MASSIVE bipolar VCA matrices with offfsets for your weights and biases :lol:
I can’t even translate this let alone hope to implement it.

—-
O&C probably has tons of uses cases but scuba gear isn’t my thing
My mistake, I assumed your familiarity since you started the topic. It was a joke.

Simplified; in neural networks each neuron can be seen as an offset (bias) and a weight for your incoming signal which you tune in such a way that you get the right answer, using math. So, a*x + b, where x is your input signal, a is the weight/multiplier/attenueverter and b is your bias or offset.

These neurons usually are arranged in a large matrix for larger scale versions of this operation, and to respond to several inputs and outputs simultaneously.

This output is then fed through an activation function, such that your transformations aren't simply linear transformations (like the matrix operation). I won't get into the hows and whys, but one such activation function commonly used is a Rectified Linear Unit, a ReLU. This function cuts every value below zero and keeps every value as it is above zero.

So in essence this could be set up with a bipolar matrix mixer with lots of offsets and rectification. Stack enough of these and you have layers of what we call a deep neural network, and you could process anything using that. What you're missing to make it actual machine learning/ai that trains in eurorack is the training part - you're stuck setting all these weights and biases yourself - unless you feel like recreating some advanced math functions with eurorack modules, now there's a challenge.


Obviously, this wasn't a real suggestion :lol: :hihi:

So, an actual suggestion would to grab some ring mods, offsets and a comparator; I like the triple bipolar VCA from WMD but there's a bunch of others. Comparator 2 by joranalogue looks good for this.
User avatar
vidret
Veteran Wiggler
Posts: 682
Joined: Sun Jul 30, 2017 10:26 pm
Location: Skogen

Re: Anything comparable to ANN?(logic module?)

Post by vidret »

The arc looks like it has some kind of activation function at the end as well, so something like the instruo tanh (or any fun waveshaping actually) for that.

Why not get the arc? It seems to be very geared towards the task.
User avatar
Severed head
Ultra Wiggler
Posts: 923
Joined: Tue Sep 24, 2019 9:24 am

Re: Anything comparable to ANN?(logic module?)

Post by Severed head »

vidret wrote: Thu Sep 22, 2022 8:57 pm Why not get the arc? It seems to be very geared towards the task.
I haven’t been able to find one-
The condensed reissue either never happened or happened off the radar and was Sold out really fast.

Nah not to familiar just the very basic understanding from the YT demo/description
I appreciate the lengthy description of info!!!

I’ve never used a logic module and am looking into ways of sparking creativity that stem from interesting interactions of signal manipulation instead an “fx” module that has that flashy instant gratification vibe.
Which is always fun but not really what I’m after atm
WTB:-Mac recordings software from 2008/9
:help:
aevai
Learning to Wiggle
Posts: 48
Joined: Fri May 27, 2022 12:10 am

Re: Anything comparable to ANN?(logic module?)

Post by aevai »

How about Neuron from NLC?

https://www.nonlinearcircuits.com/modules/p/8hp-neuron

"The Neuron was inspired by a paper on neural computing and is a variation of a typical analogue neuron circuit".

Get ~60 of them and arrange them in multiple layers :lol:

Though not sure how you would do backpropagation...
User avatar
vidret
Veteran Wiggler
Posts: 682
Joined: Sun Jul 30, 2017 10:26 pm
Location: Skogen

Re: Anything comparable to ANN?(logic module?)

Post by vidret »

^^ backpropagation is the issue isn’t it. A high pass filter is essentially a differentiator, so have at it :hihi:


Maybe start off with just a comparator; input several different cv signals mixed/attenuated/inverted from various sources and then compared with some perhaps movable threshold.

Then take the output gate to trigger something interesting, like the cv inputs makes the decision för you whether to trigger an envelope of change a sample etc
User avatar
jheronymo
Ultra Wiggler
Posts: 753
Joined: Sat Nov 22, 2014 4:11 pm
Location: Vancouver, BC

Re: Anything comparable to ANN?(logic module?)

Post by jheronymo »

vidret wrote: Thu Sep 22, 2022 8:52 pm What you're missing to make it actual machine learning/ai that trains in eurorack is the training part - you're stuck setting all these weights and biases yourself - unless you feel like recreating some advanced math functions with eurorack modules, now there's a challenge.

Obviously, this wasn't a real suggestion :lol: :hihi:
Interestingly the Mark 1 Perceptron in the late 50s, which (I believe) was the first hardware implementation of these ideas, used motorized potentiometers which the operator would set manually to an initial state, and would then be updated automatically during training.

Not that it was an immediate success...


perceptron mark 1.png
User avatar
vidret
Veteran Wiggler
Posts: 682
Joined: Sun Jul 30, 2017 10:26 pm
Location: Skogen

Re: Anything comparable to ANN?(logic module?)

Post by vidret »

damn now I want electric motors for my pots for when they receive CV, that'd be insane.
User avatar
grrrwaaa
Wiggling with Experience
Posts: 436
Joined: Wed Sep 16, 2020 8:00 pm
Location: Toronto
Contact:

Re: Anything comparable to ANN?(logic module?)

Post by grrrwaaa »

For the fun of rebuilding perceptions then yes, just a chain of mixers and rectifiers or tanh (sigmoid like) shapers would do it, at leas for training by manual knob turning. To process sequential data the input could use an analog shift register, and all of the updates synchronized via s&h. But it seems more fun to work with continuous signals, otherwise why do this in analog at all?

Training automatically instead means vca mixers not attenuator mixers: not adjusting weight by hand, but by it self-adjusting stored signal values. Backpropagation is the usual way, which requires a bipolar error signal (difference/signed distance between target and actual output) that is fed back to nudge every mixer vca setting a little bit (i.e. integrated). Basically an informed guess as to how to tune the (virtual) knob to get a better response (lower error). The usual trick is to feed an attenuated copy of the error back using the same mixer weights (more or less), as well as the slope (differential) of the shaper at the current signal level. For a relu shape the slope is either on or off, which makes things easier.

So, bipolar stereo vcas, a waveshaper with dual differentiated shape output (or rectifier with dual binary output, like a comparator), and lots of integrators. Not so easy in Euro, even for one neuron. Even a basic non-leaky integrator is pretty uncommon. Cold Mac has one.

Backpropagation isn't the only way to train a signal network. A really crude method is to try small random deviations and keep them if they improve the result (monte carlo). It is hard to "go backwards" like this with analog, but a general switching method of "keep while improving" might be possible.

(Something about this is reminiscent of how PLL patches work.)

Or go digital ... write some code in a digital module that can handle N inputs and N outputs, including training data inputs and expected result. Not as hands on but certainly much easier to write larger and deeper networks that can discriminate more complex inputs.
oopsy = daisy hardware via max/msp gen~: https://www.modwiggler.com/forum/viewto ... 6&t=242322
modular recordings: https://dokkaebi.bandcamp.com
User avatar
grrrwaaa
Wiggling with Experience
Posts: 436
Joined: Wed Sep 16, 2020 8:00 pm
Location: Toronto
Contact:

Re: Anything comparable to ANN?(logic module?)

Post by grrrwaaa »

I wondered if it was worth fleshing out that PLL comparison for a brief bit of mindwandering.

A PLL structure is usually something like
1. compare the generated output (vco2) to the target output (vco1) to get a signed error function (e.g. via ringmod/xor/etc)
2. send that back to the FM input of vco2, via a slew.

Compared to training a neuron, step 1 is very much like comparing the neuron output with expected results from training to get an error signal.
Step 2 is the feedback backpropagation part. The slew output is basically operating as the neuron weight, adding to / subtracting from the initial vco2 bias. A slower slew means a slower learning rate/better stability, but setting the slew/learning rate too fast can make it too jittery (overoptimizing in the ANN sense).

So the parallels are certinly there. What's different:
- The PLL is generating and comparing continuously oscillating signals, not level signals; and the frequency of oscillation is what we care about. Even the slewed weight will be oscillating slightly.
- There isn't a shaper function involved per se, though there are clearly voltage limits (hard clipping)
- The PLL is constantly learning and forgetting, and is therefore not developing a model of a multidimensional space as such; it only lives for the moment
- The PLL has only one weight (plus an untrained bias)

This just makes me think that a) maybe working with oscillators rather than number levels might have valid applications for an analog neuron, and b) it would be fun to try some PLL patching with multiple feedback paths and multiple oscillators and see what happens!
oopsy = daisy hardware via max/msp gen~: https://www.modwiggler.com/forum/viewto ... 6&t=242322
modular recordings: https://dokkaebi.bandcamp.com
LunaticSound
Ultra Wiggler
Posts: 921
Joined: Tue Dec 25, 2018 3:50 pm
Location: Hamburg

Re: Anything comparable to ANN?(logic module?)

Post by LunaticSound »

vidret wrote: Fri Sep 23, 2022 2:14 pm damn now I want electric motors for my pots for when they receive CV, that'd be insane.
Check out Droid by Der Mann mit der Maschine!
euromorcego
Super Deluxe Wiggler
Posts: 2864
Joined: Sat Jan 18, 2014 6:22 am

Re: Anything comparable to ANN?(logic module?)

Post by euromorcego »

Severed head wrote: Tue Sep 20, 2022 1:46 am What logic modules do you folks find comparable to ANN
(ARC artificial neural network)
Has there been a useful answer yet?

ARC has different sections. Most of these can be readily replicated by other modules. For the more straightforward logic section and inverter, there is always the Doepfer A-166 logic module.

The more interesting part, imho, is the part the 'threshold logic neuron'.

That is basically an attenvuerting mixer with a comparator afterwards. Attenuverting means you cannot just add the signals, but also substract. The comparator acts as a threshold, only if the (summed) signal is above the threshold, the output goes "on" (5 V probably).
So if you have two signal A and B, the output can either be:
"OR" if the level of A or B are sufficient to go past the threshold on their own.
"AND" if A and B are both smaller individually, but summed up they clear the threshold.
plus more interesting functions whose name I do not known, e.g. A is large enough to clear the threshold, but B is substracted, so if B is positive, the output is inhibited.

Since A and B are arbitrary signals, and you can also use more than 2 inputs (or use an offset), there are a lot of possibilities for logic functions.

To re-create it, you need a few attenuverting mixer, plus comparators.

For example Ladik M-120 3Ch Attenuverting Mixer (https://www.modulargrid.net/e/ladik-m-1 ... ting-mixer) and J-120 Comparator (4HP) (http://ladik.ladik.eu/?page_id=1459).
Two of each and you can do basically all logic you want without an explicit ready-to-go logic module. And you can get more 'section', and are thus actually more flexible than the ARC.

Other attenuverting mixer also work (of course), and instead of the comparator anything that works with a threshold should also work (for example a envelope that only triggers when the input is above a certain level ... or a logic module, when you use the OR input with a single signal). Drawback of these is that the treshold is often quite low (<1V) so you do not have a lot room to play with. But definitely also try an envelope.

edit: also on the long never to happen to-do list is also to build a few of the mixer/comparator combos in a single package. 3 input attenuverting mixer with threshold at the output, 3-4hp or less. It is a simple circuit. I wanted to have a few of them as a "logic section":
Post Reply

Return to “1U & 3U Eurorack Modules”