Taivo Lints Blog

A book by Jeff Hawkins, from 2004, explaining his theory of brain and intelligence.

Jeff takes the view that intelligence is the ability to (successfully) predict. And the view that what brain does, on all levels, can basically be summed up as memorizing patterns (and sequences of patterns) and providing predictions based on those learned patterns.

More specifically, the brain is constantly extracting regularities from the incoming data streams on multiple hierarchial levels: regularities in the "low-level" sensory input, regularities in the occurrences of those low-level regularities, etc. For example, it learns (roughly speaking) certain optical patterns to form certain letters, letters to form certain words, words to form certain sentences, sentences to form certain texts, etc. At the same time there is also a constant flow of information in the opposite direction: the brain associatively recalls, at all levels, based on previous experience, what should be there and what should come next. I.e., it constantly makes predictions. And if something does not match the predictions, an "alarm" is raised that should lead to an (automatic) reconsideration of the situation and possibly to learning something new. For example, if the brain assumes that the text that the person started to read is some particular famous speech it knows, it predicts what sentences, words and letters come next. If these predictions fail, it may need to reconsider the initial assumption that the text is that particular speech. Or if the prediction fails only on a single word, the attention of the person is drawn to that word (which might turn out to be a typo or maybe the word was incorrectly remembered by the person or just misread in the first go at the moment).

And such continuous pattern extraction and prediction is what goes on in almost all parts of brain and on all levels, and underlies everything from the lowest sensory input and motor output to the highest levels of abstract thinking. Jeff emphasizes that "Prediction is so pervasive that what we "perceive" — that is, how the world appears to us — does not come solely from our senses. What we perceive is a combination of what we sense and of our brains' memory-derived predictions.", because what the brain predicts actually affects what and how we sense and notice. Also, while when applied to incoming sensory data the top-down flow has the role of predicting and drawing attention to violations of prediction, the brain can also route the same top-down flow to motor output, in which case it will be executed, if possible: "when your own behavior is involved, your predictions not only precede sensation, they determine sensation". In the given example of text processing, the particular speech as such gets unfolded into sentences, and sentences into words, and words into letters, and letters into hand movements for handwriting or hand movements for typewriting or movements of the vocal apparatus for speaking etc.

As of intelligence in general, Jeff claims that "Intelligence is measured by the predictive ability of a hierarchical memory, not by humanlike behavior." and that there is no reason to be afraid of intelligent machines, because "They will not have personal ambition. They will not desire wealth, social recognition, or sensual gratification. They will not have appetites, addictions, or mood disorders. Intelligent machines will not have anything resembling human emotion unless we painstakingly design them to."

I find the central point of the book very interesting and quite likely to have a strong relevance to how the brain really works (not necessarily in details, as Jeff himself points out, but at least in some of the general principles). However, I am not so sure about the reliability of those parts that touch upon artificial intelligence more generally, outside his memory-prediction framework. I cannot really disprove anything, but I occasionally got the feeling that some things are not fully correct. For example the aforementioned belief that intelligent machines will be just emotionless pattern detectors and predictors unless we painstakingly embed the emotions — some alternative opinions say that such emotions and drives are actually crucial for developing interesting higher level AI (then again, maybe it's partly about the difference of goals — whether we should create emotionless pattern detectors-predictors or systems that behave interestingly). Also, I get the impression that when talking about AI Jeff seems to equate it only with the early classical logic-based AI, and when talking of "robots" he seems to think of inflexible systems without much any feedback... (which isn't exactly the case in general).

Also, even though Jeff emphasizes that he is not interested in building humans, but in understanding intelligence and building intelligent machines, he categorically claims that "We have to extract intelligence from within the brain. No other road will get us there.". I agree that brain is the best example for us to build on, but by no means should we be so categorical and exclude other roads to high-level machine intelligence. Jeff points out early in the book that it was his strong intuition that made him rather sure that the Artificial Intelligence approach will fail to create programs that do what humans can do and will also fail to teach us what is intelligence in general. While I have actually had the very same intuition for a long time as well, and am thus very supportive to nonclassical approaches to AI, I nevertheless exercise caution about getting channeled into the other extreme, and I find it slightly alarming that Jeff seems (in my humble opinion) to fall into the very trap that he, ironically, cautions against just 20 pages later: "However, looking across the history of science, we see our intuition is often the biggest obstacle to discovering the truth".

But these small complaints of mine are mainly about the peripheral / general thoughts in the book, not about the core point which I find very interesting. So, overall I'd still say that the book "On Intelligence" is definitely worth reading for anybody interested in the workings of the brain and (but not necessarily) in how to create thinking machines.

More info about the book at Amazon: http://www.amazon.com/dp/0805078533

A book, the 7th edition (2005) of which I read, by B. R. Hergenhahn and Matthew H. Olson, about the main theories of learning, in a college textbook format.

The book starts by giving a general overview of the concept of learning and the use of scientific method. Then the early approaches are briefly described, starting from Plato and Aristotle and going through the various philosophers (Descartes, Hobbes, Locke, Hume, etc.) up to the early schools of psychology (voluntarism, structuralism, functionalism, early behaviorism). And after that the more detailed overviews of the major theories of learning are presented, which takes up the most of the book:

• Predominantly functionalistic theories: Thorndike, Skinner, Hull.
• Predominantly associationistic theories: Pavlov, Guthrie, Estes.
• Predominantly cognitive theories: Gestalt theory, Piaget, Tolman, Bandura.
• Predominantly neurophysiological theory: Hebb.
• An evolutionary theory: Bolles and evolutionary psychology.

The final chapter of the book rather briefly discusses the current trends and some open questions.

I feel like I got quite a good general overview of the main ideas about the learning process: their historical development, key researchers and key points. The overall flow of the book followed the development of the theories in time, which provided the benefit of understanding why the things that currently seem obvious were not so obvious earlier -- when reading the older theories they seem to fully make sense and match experimental data, but then in the next section / chapter new (later) experiments and ideas are described that partially disprove some of the previous ones and form a new seemingly great theory... until another chapter brings yet another change. This also sustains the desire to keep reading to find out new and new things. On the other hand, the book isn't exactly leisure literature and at some points I really felt like taking a break or pushing myself a bit to keep going.

All in all, it was a very useful book for me: both for my research and for the general understanding of the world.

More info about the book at Amazon: http://www.amazon.com/dp/0131147226/

A book by Ray Kurzweil and Terry Grossman, from 2009, about keeping a good health and extending your expected lifetime.

Modern healthcare is becoming more and more infused and boosted by the information technology, or, in some sense, it IS becoming an information technology itself by moving from the phase of trial-and-error to the phase of data- and simulation-based design of interventions. Ray and Terry believe that this will lead the healthcare to follow the same kind of exponential growth as various areas of IT have enjoyed, which in turn will lead to radically more efficient health maintenance and healing methods already within a few next decades (new efficient drugs, RNA interference, gene addition, pluripotent stem cell based therapies, later also medical nanobots). Their idea is that during the next few decades you could look at your life as being either behind or in front of a moving frontier of extreme longevity -- if you keep yourself in good enough health until the next level of healthcare arrives in around 10-15 years, then your health and expected lifetime will be boosted enough by the new methods of that level to reach the yet another level of healthcare arriving in the 2030-ies, where it will be boosted again, and so on. This book is intended to help you cross the "Bridge One".

Regardless of whether Ray's and Terry's predictions turn out to be correct or not, the book "Transcend" is packed with useful information about living a healthier life. The suggestions are so numerous and detailed that it is difficult to summarize them here, but, broadly speaking, the main topics are:

• Assessing your state frequently and thoroughly enough -- self-assessments, medical examinations, lab tests -- and using this information to adjust your lifestyle, eating, etc.
• Keeping the stress under control.
• Paying attention to what and how much you eat.
• Taking appropriate supplements.
• Exercising regularly: aerobical, strength, and stretching.
• Minimizing toxin contact / intake.

All these are explained with plenty of details and guidelines, based on the very latest scientific knowledge they had at hand. The latter means, however, that not all the suggestions are based on long-term human experiments, and they do acknowledge themselves that some viewpoints might change in time and that some of their suggestions, especially with regard to supplements, are considerably different from the common FDA approved ones. But they do promise to keep interested readers up to date with latest developments and research results via an electronic newsletter (that anybody can subscribe to at http://www.kurzweilai.net/).

A possible conflict of interests can be found in the fact that Ray and Terry also have a supplement-selling business where you can buy the supplements they suggest in the book, but knowing a bit about Ray's background I would rather assume that their idea was not to make a lot of money by suggesting the supplements, but he just wanted to have a reliable source of those at hand, both for himself and for the people they are advising (I haven't bought anything from there so far, though, in case you're wondering).

All in all, I would highly recommend this book to anyone who is interested in maintaining a good health AND who does not freak out when seeing occasional complex-sounding words and phrases like gamma-tocopherol, prostaglandin-E3 or single photon emission computed tomography, AND who has enough education and critical mind to understand that not all suggestions can be taken as the ultimate truth, but just as interpretations of the current state of scientific knowledge.

More info at Amazon: http://www.amazon.com/dp/1605299561
and at Ray's and Terry's site: http://www.rayandterry.com/TRANSCEND/

A book from 2007, about the positive and negative sides of diversity in human groups. The main focus is on the diversity of perspectives and heuristics, NOT identity diversity like race or gender.

The central point of the book is "The Diversity Trumps Ability Theorem" that applies to problem solving by groups. Roughly, it says that given the problem is difficult and the problem solvers are not stupid and no solution except the global optimum is a local optimum simultaneously for every individual in the group and both the initial problem solver population and the selected group are large enough, a randomly selected collection of problem solvers should outperform a collection of the best individual problem solvers. In the model that Page gives, the diversity always trumps ability, given those four conditions hold, but the book also describes how various human factors may make the theorem not work in some cases.

There's also "The Diversity Prediction Theorem": Given a crowd of predictive models, the Collective Error = Average Individual Error – Prediction Diversity. Here a group of randomly selected predictors does not necessarily predict more accurately than a group of the best predictors.

The discussions in the book are well supported with mathematical / computer models (though details are left to referenced papers, keeping the text easy to read), and while it is in general pro-diversity, it is NOT slogan'ishly pro-identity-diversity, but instead specifies what kind of diversity is good in which situations, and what are the accompanying costs that may sometimes cancel out the benefits.

For me, the book was quite an interesting read.

More info at Amazon: http://www.amazon.com/dp/0691128383 and at Princeton University Press: http://press.princeton.edu/titles/8353.html

A book, published in 2006, about the mathematical and physical foundations of music. It covers musical representation, scales, tuning, intonation, physical basis of sound, geometrical basis of sound, psychophysical basis of sound, introduction to acoustics, vibrating systems, composition and methodology (including a glimpse of less usual approaches like using artificial neural nets).

In general, I really like this kind of book about sound and music that has a solid scientific background. Unfortunately in this specific book the background is not as solid as I was hoping. While reading it, especially the physics section, I quite often had problems like: "Er... am I not getting it or what? No, wait, it can't be like that. Damn, it must be wrong in the book!" And sure enough, there ARE errors in abundance. For a list of errors known to Gareth Loy himself, see: http://www.musimathics.com/Errata.html

The other problem was that occasionally the book was not written very clearly and / or captivatingly for me... (plus the feeling that in order to explain those concepts to somebody with a humanities background (who are clearly included in the target audience), the explanations should be significantly different). Though that was not a major problem for me: I still read through the whole book and got a lot of information out of it.

In conclusion, as much as I would have liked to, I cannot suggest this book as a very good one on this topic. There are other books available about the basis of sound and music, and I suspect some of those may have a more solid science and better explanations in them, but I haven't read any others, so I can't say for sure. Also, there's some hope that Musimathics itself will be revised in future editions.

More info at the book's homepage: http://www.musimathics.com/

and at Amazon: http://www.amazon.com/dp/0262122820/