Making *outputs* more intelligible. A neural net trained to maximize odds of winning in chess is going to produce more intelligible output than an AI which moves randomly, is what I mean in saying that.

Oh, but you are being directed. It's just that the part of the brain you call consciousness is merely steering the rudder of a gigantic chemical computer.

Expose a man-made computer to the same scale of evolution that your body evolved in and it will start to act and think more like a human (without pesky primates trying to futilely "direct" them).

Moving randomly doesn't sound like very good "AI"! Do you mean "intelligent" rather than "intelligible"?

In my view, really what these highly iterative approaches to classifiers represent is simply evolution on smaller time scales. But of course this is going to be inferior to species which adapted over billions of years, because as I said, the information to adapt to simply isn't accessible.

Also, these classifiers are just tools. They don't threaten to replace humans any more than other tools do. And like most sharp tools, it is easy to hurt yourself on them.

Counterexample: all life except homo.

When it comes to things like vision, isn't it the case that "all life except homo" very often do better than machines? It's just that we don't have access to the (biological) machinery that let them do that in a way that would let us direct that performant hardware.

No, I mean intelligible. An AI that maximizes win rate will make moves that are more intelligible to anyone who understands the rules and gameplay of chess.

Intelligent isn't the right word.

Partly because: the complex history of life on earth makes for a difficult thing to find a data set for. Maybe that's why machine learning algorithms don't always know the difference between a cup of coffee and a baseball?

The moves are "intelligible" because they are good moves, demonstrating intelligence. But the process that allowed them to make those moves? Not intelligible at all.

Anyway, I see you were using the word differently than I took it, but I can't help but point out that deep neural networks seem to work for rather unintelligible reasons.

Also, how is the human brain not also simply automating statistics?

I don't believe anything a computer can do would be appropriately described by the word intelligent, unless it's used as a metaphor or anthropomorphism.



I think it's intelligible insofar as we can make accurate statements about how the algorithm works. We might not be able to peek at the data and pull intelligible data from it, but I don't see why anything else would be impossible to comprehend.



They work because statistics work, no? It's just a decision making algorithm that updates its own datasets and calculates regressions.

The human brain is much more complicated than any sort of algorithm we currently have.

Look at that arxiv paper again. Saying that we "understand" deep neural nets because the math is simple is like saying that we understand how computers work because we know what logical gates do. At best we can simply observe when the classifiers do seem to work (most) of the time, but we good luck trying to come up with a stable interpretation of how and how well it does that.

This definition of intelligence seems slightly mystical to me. I don't see anything particularly special about brains, except that they are more suited to their environment, have had far more time to evolve, and on the whole probably do far more computations than artificial machines do, when you take into account all the information processing stuff going on down to the cellular level.

But we don't even understand all the algorithms that we currently have! That's the whole point of deep neural networks (and genetic algorithms before them), that they emerge classifiers, and we don't know why they work. Or maybe we do, in which case I'd like to hear about it.

That said, I imagine that the brain is far more 'architectural' than current classifers are. Meaning: evolution probably innovated several times in the history of our evolution, and that failing to persist these innovations in order to evolve around them may well prove detrimental to performance. We could be spinning our wheels with a certain class of neural networks and never getting anywhere because we lack some crucial technique.

On the other hand, the counter-example to this idea is that in the limit, state-of-the-art learning algorithms don't differ too greatly in performance once you feed them enough data. But this just begs the question once again: are brains just classifiers? I'm not so sure. I think to say this we might have to vastly expand what we mean by "data", to the point that we begin to model our environment. Of course if I am not mistaken, this is what reinforcement learning does.

It's more that I don't think we're very close to having a good model of how the brain functions, and there are severe limitations with trying to reproduce such a thing on silicon that many bombastic tech guys are quick to hand wave away.



Complex software gets itself into all sorts of unpredictable states. It's not new to ML algorithms to get unexpected outputs. Asking why ML algorithms produce unexpected outputs is like asking why people still find bugs in software. It just so happens that it's more of a feature than a bug of ML.

I don't think asking a ML algorithm why it's doing what it's doing is a good question to ask.

Here's the short version.

1.) Experts know next to nothing about how neurons work. They understand that neurons communicate via neurotransmitters in synapses, they understand there are activation thresholds in synapses, and the neuron uses electrical signals to communicate within itself, but beyond that they don't know anything. Neurons are themselves fully-realized cells containing their own metabolism, and potentially contain significant state or perform significant computation. They also don't work alone. They have companion cells that may very well contribute as well. Even if they are just as simple as neurons in e.g. Tensorflow, though, for whatever reason synapses all seem to secrete and respond to several up to around a hundred different neurotransmitters at different activation thresholds. Nobody knows what significance the different neurotransmitters have, whether the neuron 'understands' that the signal is different or whatever, although we do know synapses switch to different mixes of neurotransmitters over time (e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3209552/). The safe bet IMO is that real neurons should be understood as more like networked computers, rather than labeled nodes on a graph like we currently treat them. Even in the most optimistic case though, we're still incredibly far away from emulating real neurons (not to mention that we need to understand how they work, first). The fact that you can get impressive results from current RNNs speaks to the mathematical significance of networks, not that RNNs are similar to how real brains work.

2.) Experts know next to nothing about the local structure of the brain. fMRI studies seemed super rad but they've pretty much all turned out to be junk science. The remaining reverse engineering techniques are virtually useless outside of brain injury diagnosis (e.g. http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268). Pretty much all we know is that the structure is significant, or in less fancy words, that destroying neurons makes stuff stop working. Even in the most optimistic case from #1, we're still nowhere close to understanding what shape of neural network can admit human-like intelligence, let alone close to being able to engineer one.

3.) Experts know next to nothing about the global structure of the brain. The brain has a changing electromagnetic field, brainwaves. What is the significance of brainwaves? Nobody knows. It could be waste radiation from the operation of the brain. Or it could carry important global state, or maybe even function like neuron 'wifi'. Neurons also respond to chemicals in your blood, neurotransmitters and hormones and antagonists and inhibitors and and and, and certain brain neurons are connected to the glands that excrete these compounds and are therefore also capable of changing global state that way. None of this stuff is possible in current attempts, as far as I know, and even if it were nobody has any clue why it's useful or how you might start usefully introducing it.

4.) Billions of years of evolution aside, it's not even obvious where you should start training a neural network. What conditions make human-like intelligence or consciousness beneficial? It could be a ****ing accident!

This is a highly fatalistic view, and also a bad analogy. We do understand software quite well in principle, and we also know how to make it compose, and how to interpret its source code. The same does not currently apply to deep neural nets. When you train a deep neural net, you are letting it innovate in highly opaque ways, which we would immensely benefit from if we had a systematic way of interpreting.

For all we know our brain is a subspace antenna for a universe-sized Boltzmann brain. Really. You should hope so, because God picking up the slack and giving one of our computers a soul is the most probable way we're going to emulate a human brain within the next thousand years.

i miss watching star trek

Maybe the analogy is bad, but I don't think I'm being fatalistic, I think I'm being realistic, here.

We absolutely in no way understand any of these things in any useful sense of the terms you're using.

We understand how to formally prove software. Similarly, we understand how to assemble useful neural networks by hand. Neither of these things are practical and nobody is doing them.

Point taken about the software analogy. I suppose we do understand neural nets about as well as we do software having a level of complexity that we could still reasonably hope to prove correct (i.e., not very complex at all).

Of course, I believe that simplicity might as well be said to be the most effective tool for writing better understood software (Fred Brooks' notion of fundamental vs. accidental complexity). I do not have experience with neural nets, but I imagine that simplicity is less important in understanding how the classifier ultimately works, because the result couldn't be (edit) less more opaque anyway.

Of course, I believe there is a science designed to better understand classifiers, called statistics....

re: simplicity.

Okay, you've simplified your problem down to one specialty chip and firmware, all formally proven. Oooooops, you get an SEP and it kills a dude anyway. So then you implement fault tolerance with a voting circuit. Tada, now your computer system is 3x more complicated.

Maybe your reasons for anticipating the difficulty of trying to interpret the results of neural nets are spot on, but at any rate, in hindsight, negative results like this would appear to support your skepticism.

I'm actually all for image classifiers in this context, I think what they're doing in terms of diagnosing cancer is a very good use of the technology. Even in its black box state.

I think it's conceivable that the process various ML algorithms go through can be adapted to provide intelligible insights into what it's doing. That seems like a reasonable project for researchers to be working on, actually.

Though when I think about it, I'm not sure what the ML algorithm could be programmed to do. I'm not well versed in how the algorithms process images, but I imagine the software must scan the image at various scales and try to create some invariants which it correlates to invariants from confirmed malignant tumor images. Maybe the algorithms could actually pull out image data and show side by side which data points are closest? Might give something for them to go on.

By the same authors as the arxiv paper, there is this shorter paper whose abstract gives a nice and easy to understand description of why image classifiers based on deep neural nets are vulnerable to adversarial perturbations:



http://web.stanford.edu/~amiratag/deformation.pdf

Medicine might not suffer from this, unless you think cancer will evolve to trick the classifier. Well actually, it just might, if the patient lives long enough.

The problem with introducing machines into medicine might just be the temptation to defer to "a.i." when it might prove personally costly for humans to shoulder the responsibility for their decisions.

"I may not know what you have, but nobody ever got fired for listening to Watson."

Yes, that is the primary concern, in fact, is how lazy humans are and how much we depend on automated systems.

This is what a researcher told me like a whole long time ago.

Nobody is/was planning to use ML for formal diagnosis, but only to propose options and recommend tests. The intention was always to force a human doctor to make every decision.

Traditional ML techniques (cluster analysis?) have been superhuman at medical diagnosis/test proposal for decades. His lab trained and validated on real medical data and performed as well or better than real doctors, without fail. Computer predictive statistical models can account for more information than a human doctor ever could (regardless of which specific method you use to construct that model).

Further trials were blocked by the college of physicians and the faculty of medicine. This was represented to them as a patient safety concern, but no evidence could convince them otherwise. The researcher is convinced that the MDs were primarily concerned with protecting their prestige and monopoly on medical diagnosis, even though they’re, y’know, pretty **** at it.

It wasn’t obvious to him that deep learning/neural network stuff would be any better than whatever they were doing. (No idea if that opinion has changed since.)

Sounds like a mix of fear and lack of understanding about how statistics work. People like reasons, tangible evidence, a doctor wants to point to an X-Ray and say something like, "I don't like the way that spot looks". But to give a picture of the X-ray to a box, and have the box do some confusing, out there things and output "x% sure cancer" feels wrong, no matter how well the math works out.

I think many people would benefit from knowing some more stuff about category theory, without necessarily knowing more category theory.

Many people who haven't studied much don't know the difference between the -jection and -morphism suffixes when describing arrows in math. The answer is actually pretty simple. -morphism is a description of how the maps work in a category, whereas -jection is a description of set-theorhetic properties of the maps.

A map f:A->B is an isomorphism if there exists a g:B->A such that fg=id_B and gf=id_A. That's what isomorphism means in every context you ever read it. The only thing which changes is your category. In the category of topological spaces, "maps" are "continuous functions", and the definition still holds.

It's a really abstract definition, but compound that on layers of math, and it becomes useful to know the most basic idea.

Bijections on the other hand are maps f between sets where f is an injection and a surjection. In any category where your objects are sets, -morphism and -jection coincide with whatever map your category uses.

I mention this because recently I was having a discussion, and it seemed some people didn't understand that "isomorphism" is a more essential and simpler concept than "bijection".

Also, I don't think Reverend Jones is unique in his bias towards constructivist/finitist mathematics. I've started noticing that programmers generally have a bias towards this style of mathematics. I think it's because computers are finitistic by nature and, to get interesting out, computers often construct the answer algorithmically.

I mention that as well because in the same discussion, someone seemed to not understand the difference between showing the existence of something, and constructing it. In programming, existence and construction are near synonyms. In mathematics they are not.

But it led to some weird understanding on the part of my interlocutor. They, for some reason, thought you could argue two vector spaces over the same field and of the same dimension were isomorphic without knowing an isomorphism exists. That sort of blew my mind, because if you understand the definitions above, you'd be showing something without reference to its definition, aka speaking a kind of nonsense. They didn't realize that the proof of that theorem involves creating an isomorphism and showing it's an isomorphism. Understanding what isomorphism means, not just as a map of sets, but what it means in terms of composition of arrows, would have cleared that up for them.

TL;DR the term "isomorphic" doesn't just describe properties shared by two objects, it implies the existence of maps between them which obey certain properties.

The criticism here would be in math education, where I don't believe the difference between terms like "bijective" and "isomorphic" are made clear enough.

Then again, if mathematicians did teach that, sciences would treat it as "obvious" and complain that mathematicians are wasting people's time teaching stuff that doesn't matter, so.. who knows.

isomorphism is a structure preserving map with inverse, bijection is a 1:1 map with inverse.