Neurons and synapses are incredibly complex. Think you're emulating over 3,000 heterogeneous cores with over half a million links while communication must be low-latency-ish. A third of the links seem to join other links and we don't even know what that does. If there's computation there we'll need even more cores.
I think you may be overestimating the complexity. The better idea is to set up and experiment with different simulation parameters, and see how far they diverge from actual observed behavior.
Seems two different things are being discussed, sort of top-down, vs bottom-up modelling.
I'm sure we're not far from making a high-level LLM-ish model of behavior based on those extensive studies.
But the topic of discussion is not that, but making a model sufficiently accurate that you "turn the crank" and it yields similar behavior without any priors of what that behavior should be.
To do that, at a minimum, we'd need for each neuron, the profile of responses to each of the neurotransmitters at each synapse, the excitatory/inhibitory effects of each signal, the patterns of how each neuron reacts to those inputs (i.e., receiving a signal from upstream neuron 489327 does not mean that it'll just pass it downstream, but that it'll decide depending on rate of firing, other current excitatory/inhibitory inputs, etc., if and at what rate it'll send the signal downstream), the rate of learning in each of those neurons... and a bunch of other variables, fully modeled.
Then, compute all of those running through the system, and have it take an input like a photo and output the same behavior, from the bottom up, without hints from the behavioral studies.
> But the topic of discussion is not that, but making a model sufficiently accurate that you "turn the crank" and it yields similar behavior without any priors of what that behavior should be.
Change the expression levels of some genes, alter an amino acid here or there, change the input parameters (e.g. make a phenotype that lacks the ability to feel pain), and you can end up with a neuron network that responds subtly, or grossly, differently than another. Put memory into the picture and responses can be learned. Fruit fly behavior permanently changes in response to serious injury, similar to the manner in which humans experience chronic pain. Sure, now this is something that can be modeled, but before the experiment it wouldn't have been modeled.
> After the injury healed, they found the fly's other legs had become hypersensitive. "After the animal is hurt once badly, they are hypersensitive and try to protect themselves for the rest of their lives," said Associate Professor Neely. "That's kind of cool and intuitive."
> "The fly is receiving 'pain' messages from its body that then go through sensory neurons to the ventral nerve cord, the fly's version of our spinal cord. In this nerve cord are inhibitory neurons that act like a 'gate' to allow or block pain perception based on the context," Associate Professor Neely said. "After the injury, the injured nerve dumps all its cargo in the nerve cord and kills all the brakes, forever. Then the rest of the animal doesn't have brakes on its 'pain'. The 'pain' threshold changes and now they are hypervigilant."
Edit: I got confused and didn’t write this comment in the right place.
—-
I somewhat agree with how you described it as top-down vs bottom-up. I think it’s not exactly how I was framing it, but it’s close enough, and it’s a useful way to think of it.
Even in the rest of your comment you’re taking a bit more of a bottom-up approach relative to what I’m saying: you’d be surprised how much we know about how the brain’s gross organization leads to complex phenomena (pick up the latest edition of Blumenfeld’s clinical neuroanatomy if you want the very-high-level summary).
You can, in principle, achieve a “broadly correct” outcome by doing tissue-level modelling of NNs. It’s surprising how much of the brain is macro components, as opposed to micro, cell-level processing. (Of course I’m handwaving a lot here. I’m afraid anything short of a concrete demonstration is bound to be unsatisfying.)
And yes, modeling at the higher functional level can be very useful; knock out the Wernicke's center and speech goes, visual cortex, vision, etc... So, with a more detailed functional description of each level, we may wind up with a model with useful predictive value.
Tho, that said, how does this approach create a truly robust abstraction from the lower level wetware? Would it provide ability to fully reproduce computations? Would it account for lower-level changes in health, hormones, electrolyte levels, neurotransmitter-active drugs...?
This is a little handwavy, but consider that, in the context of modelling a human brain:
- If you model top-down (at the tissue/functional level like you suggest), you'll be recreating the "broadly correct" kind of computation, and you'll be recreating something that looks and feels like a really really bad, quirky and dumb human brain. But it'll have certain human-like qualities that are maybe even difficult to pin down. These would possibly make a ton of mistakes, but there'd be a lot of human-like biases and mistakes.
- If you throw billions of neurons into a bag, you may be able to train them to perform calculations with a high degree of correctness (eg.: ChatGPT, generative art, modern ADAS systems) but when these make mistakes, the mistakes they make will look extraordinarily stupid to a human (eg.: "a human would never have suddenly steered his car into a brick wall like that").
Both approaches can produce extremely stupid results, but you need the top-down architecture if you want to preserve what makes "the human flavour of intelligence" what it is. (I suppose you could emulate the same result with a big enough bag of neurons, but that sounds very inefficient to me, intuitively.)
---
Depending on what you're interested in modelling, you may need to combine multiple approaches, as the brain has multiple layers of emergent properties. I don't think that for most purposes you'd need to go as far as modelling blood contents, but something like it might be required if "embodiment" was an important part of what you'd like to model. There are certain types of things that biological organisms learn particularly fast because they have a physical body that interacts with the real world.
I don't personally believe embodiment is fundamentally required in the model (ie.: I think it's probably possible to emulate the same result if you use a sufficiently large number of neurons), but I think realistically it will be a practical necessity for keeping models and computations as efficient as possible.
What do you actually work on? I'm a synthetic biologist and know a hell of a lot about manipulating pieces of DNA. I know a bit about other branches of lab and academic biology. And I can remember some of the chemistry classes I've taken, usually a bit after the fact.
Are you saying this because you've spent time reading and/or researching fruit fly behavior studies? Or for some other reason?
I currently work as a software/data engineer, but spent 12 years doing fundamental biochem & life science. My MSc was on the characterization of extracellular vesicles in the context of horizontal transfer of information in ccRCC (kidney cancer) cell lines.
I worked closely with one prof (not my PI) who specializes in the study of mitochondrial metabolism in fruit flies, but fruit flies are not my area of expertise. I know just enough to know that there’s an enormous amount of literature on the subject.
To simulate something you need to first know how it works and know all the relevant interactions, this is something that we currently do not understand. Neurons are not equivalent of neural networks we use in computers, they are A LOT more complex with whole groups firing the same time and chemicals regulating neurons all the time and whole topology is plastic, it works in a way we can't model or simulate currently. People hugely overestimate our knowledge about brains.
I'm a biochemist, and I somewhat disagree with this take.
Yes, it's true that modelling just a single cell's interactions with its environments is beyond our capacity.
But here we're talking about simulating how a brain reacts to signals at a higher level of abstraction: we're studying an "emergent" phenomenon. We don't need to model molecular interactions, and we absolutely can model this using artificial neural networks.
This doesn't mean we'll be able to completely accurately model its behaviour, but we should get a lot closer than many HN commenters seem to believe. Biological neurons don't have magical properties, they just have more side-effects.
I specifically used "relevant interactions" to rule out "modelling just a single cell's interactions with its environments" but who knows how deep you need to go to have an accurate model.
> we're studying an "emergent" phenomenon
It's like saying that mapping all cells in human organism to addresses in memory will give us emergent human inside computer.
But still at abstract layer we just do not have a model of a working brain, we only have maps of neurons, not an actual "algorithm" or a model of all relevant environment interactions needed for a working brain or a set of "abstract instructions" needed for it to work.
I think you are over optimistic about current state of our knowledge if we can't even model organism with few thousand neurons accurately. Imagine trying to do that for tens of billions of neurons and trillions of synapses, especially that we know brain doesn't work like an ANN at all[1].
> It's like saying that mapping all cells in human organism to addresses in memory will give us emergent human inside computer.
Sounds more like they're saying: a knee is a knee, it doesn't have any magical properties. Build something that bends, and it will behave like a knee. I don't know how true that is, obviously.
I guess the point is that we might be able to simulate an artificial knee equivalent for neurons. It won’t be as good as a real knee, but it might still be useful.
I’d argue they’re better (in some ways), given that they’re “dead” material as opposed to functional tissue. Human knees don’t last nearly as long when the tissue is dead.
Lol. That's an interesting definition of "better".
If you're active, knee implants have to be replaced after 15-20 years, because they detach from the bone. Knee implants can harbor pockets of infections because they don't have an immune system. They are also more prone to dislocations.
They are objectively inferior to a healthy organic knee.
Yes, my comment was made a bit in jest. I think everyone will agree that after an arthroplasty, your quality of life is not as good as it was before.
But most of the reasons why artificial knees are not as good as the real deal have to do with the fact that they're made of inert (albeit fancy) materials. They don't have the ability to continually heal and do tissue remodelling, which is what real tissue does.
I feel very optimistic when I think about this: we're limited, but I think it's absolutely wonderful what we're able to do.
Wasn't there success in replicating how the neuron modulates a signal with electronics? If we can reproduce the out given any in then that's interchangeable from a system perspective?
Could you please explain why not? 548,000 synapses sounds entirely feasible to me.