Taivo Lints Blog

To content | To menu | To search

Saturday 28 January 2012

Book: The Quest for Artificial Intelligence

TheQuestForAI.jpg

A book about the history of ideas and achievements in the field of Artificial Intelligence, from the beginnings to today, written by Nils. J. Nilsson and published in 2010.

The book starts with the early dreams and ideas, describes the first important AI-related gatherings in the 1950s and 1960s, and then continues moving gradually through time, describing briefly the main ideas as they came up or became prominent: pattern recognition, heuristics, semantic representations, natural language processing, computer vision, hand-eye coordination, knowledge representation and reasoning, mobile robots, board games, speech recognition, expert systems, machine learning, AI architectures, etc. It also describes the wider historical background of the field: labs, conferences, funding, controversies, most notable achievements so far, and the direction of future research.

The author has been an active AI researcher for most of his life, knows the field quite well, and has managed to pack a lot of useful information into this book. On the downside, the book is somewhat author-centric and particularly US/West-centric. Although a lot of important research was indeed done in the US and some UK labs and thus this focus is not too much of a problem in general, there was certainly a lot of relevant work done also in the rest of the World: wider Europe, Japan, Russia... These are mentioned only a few times in passing. Also, while the classical AI topics are covered quite well, the coverage of less conventional methods has rather serious shortcomings. For example swarm intelligence — one of the most active directions at the moment — is not even mentioned at all.

So I would say that the book is quite informative and very useful to get a bird's-eye view of what has been going on in AI over the time, but it is important to keep in mind that the bird is flying between mountains that somewhat restrict the view: some areas of AI are not covered, both geographically and by subject.

Official web version of the book is available here:
http://ai.stanford.edu/~nilsson/QAI/qai-webpage.html

And more info about the book can be found at Amazon: http://www.amazon.com/dp/0521122937

Monday 15 August 2011

Homepage update

I have updated my webpage taivo.net — I added a downloadable stopwatch application that I recently created (TL MiniTime, tested to work on Mac OS X and Windows, and should also work on Linux), added a list of my publications, added a list of events I have attended, added links to some of my social networking sites, removed photo galleries (I now post my photos mainly to social networking sites) and made various other updates.

I also moved the site from my university server to a commercial provider GigaPros. Among other things this means that the address taivo.net now stays taivo.net in the address bar during surfing the site, instead of being just a redirect to www.dcc.ttu.ee/taivo. As of the latter address, it is still functional and currently displays the previous version of the site. I'll probably just put a sign there that tells people to go to taivo.net.

Sunday 19 June 2011

Book: The Prodigy — Electronic Punks

A "reissue of the very first biography of the world's biggest hard dance / rock act", "with a new introduction by Liam Howlett", published in 2010, written by Martin Roach.

The book gives a close-up and personal view of the early years of The Prodigy — about their coming into existence, about their mindsets of relentless extending of musical scope, of being punks in the sense of doing things their own ways, actively avoiding stardom, being slightly at odds with the authorities, and making music for the sake of feeling excited and making others feel excited. And about fun (and not so fun) incidents along the way.

I really resonate with The Prodigy's music and with their live shows, and I'm also interested in musicmaking myself, and I really enjoyed reading this book and understanding better the background of The Prodigy (especially the first half of the book — the second was also alright, but I got the main insights already from the first half).

So, if you like The Prodigy and would also like to understand them better, then I highly recommend this book.

Btw, the author Martin Roach also seems to be quite an interesting and slightly nonconformist guy who has become friends with The Prodigy (as well as with many other bands) and deeply cares about what he's writing about.

The book is available directly from Martin's publishing company Independent Music Press: http://www.impbooks.com/music-books/The-Prodigy---The-Official-Story/Electronic-Punks.-The-Early-Years-1988-1994/101

Sunday 5 June 2011

Book: Programming Erlang

cover_ProgrammingErlang.jpg

A book by Joe Armstrong, from 2007, giving an introduction to Erlang. Erlang is a programming language and runtime system originally developed in Ericsson and now open source, and Joe Armstrong is the original author of that language.

The reason why I felt very curious about Erlang is that it seems to be a very good fit with my interests in distributed systems composed of relatively autonomous interacting components. It has language-level support for the actor model concurrency. It supports hot code swapping. And it was developed for and has been industrially successfully used for distributed fault-tolerant soft-real-time systems. Yay.

Contrary to some reviewers on Amazon, I really enjoyed reading this book — the text was fun and the code was to the point and induced a very strong urge to go and code some clever stuff :) It, admittedly, is a book for a specific and relatively small audience, both content- and stylewise, but if you're "in the demographics" (i.e., you feel similarly excited about Erlang's features, have some previous programming experience, are ok with slightly informal writing style, and are looking for a practical intro instead of a language reference) then I'd definitely recommend this book.

More info about the book at Amazon: http://www.amazon.com/dp/193435600X and on the publisher's homepage http://pragprog.com/titles/jaerlang/programming-erlang

And thanks a lot to James Chapman for lending me this book!

Thursday 12 May 2011

Book: On Intelligence

Cover_OnIntelligence.png

A book by Jeff Hawkins, from 2004, explaining his theory of brain and intelligence.

Jeff takes the view that intelligence is the ability to (successfully) predict. And the view that what brain does, on all levels, can basically be summed up as memorizing patterns (and sequences of patterns) and providing predictions based on those learned patterns.

More specifically, the brain is constantly extracting regularities from the incoming data streams on multiple hierarchial levels: regularities in the "low-level" sensory input, regularities in the occurrences of those low-level regularities, etc. For example, it learns (roughly speaking) certain optical patterns to form certain letters, letters to form certain words, words to form certain sentences, sentences to form certain texts, etc. At the same time there is also a constant flow of information in the opposite direction: the brain associatively recalls, at all levels, based on previous experience, what should be there and what should come next. I.e., it constantly makes predictions. And if something does not match the predictions, an "alarm" is raised that should lead to an (automatic) reconsideration of the situation and possibly to learning something new. For example, if the brain assumes that the text that the person started to read is some particular famous speech it knows, it predicts what sentences, words and letters come next. If these predictions fail, it may need to reconsider the initial assumption that the text is that particular speech. Or if the prediction fails only on a single word, the attention of the person is drawn to that word (which might turn out to be a typo or maybe the word was incorrectly remembered by the person or just misread in the first go at the moment).

And such continuous pattern extraction and prediction is what goes on in almost all parts of brain and on all levels, and underlies everything from the lowest sensory input and motor output to the highest levels of abstract thinking. Jeff emphasizes that "Prediction is so pervasive that what we "perceive" — that is, how the world appears to us — does not come solely from our senses. What we perceive is a combination of what we sense and of our brains' memory-derived predictions.", because what the brain predicts actually affects what and how we sense and notice. Also, while when applied to incoming sensory data the top-down flow has the role of predicting and drawing attention to violations of prediction, the brain can also route the same top-down flow to motor output, in which case it will be executed, if possible: "when your own behavior is involved, your predictions not only precede sensation, they determine sensation". In the given example of text processing, the particular speech as such gets unfolded into sentences, and sentences into words, and words into letters, and letters into hand movements for handwriting or hand movements for typewriting or movements of the vocal apparatus for speaking etc.

As of intelligence in general, Jeff claims that "Intelligence is measured by the predictive ability of a hierarchical memory, not by humanlike behavior." and that there is no reason to be afraid of intelligent machines, because "They will not have personal ambition. They will not desire wealth, social recognition, or sensual gratification. They will not have appetites, addictions, or mood disorders. Intelligent machines will not have anything resembling human emotion unless we painstakingly design them to."

I find the central point of the book very interesting and quite likely to have a strong relevance to how the brain really works (not necessarily in details, as Jeff himself points out, but at least in some of the general principles). However, I am not so sure about the reliability of those parts that touch upon artificial intelligence more generally, outside his memory-prediction framework. I cannot really disprove anything, but I occasionally got the feeling that some things are not fully correct. For example the aforementioned belief that intelligent machines will be just emotionless pattern detectors and predictors unless we painstakingly embed the emotions — some alternative opinions say that such emotions and drives are actually crucial for developing interesting higher level AI (then again, maybe it's partly about the difference of goals — whether we should create emotionless pattern detectors-predictors or systems that behave interestingly). Also, I get the impression that when talking about AI Jeff seems to equate it only with the early classical logic-based AI, and when talking of "robots" he seems to think of inflexible systems without much any feedback... (which isn't exactly the case in general).

Also, even though Jeff emphasizes that he is not interested in building humans, but in understanding intelligence and building intelligent machines, he categorically claims that "We have to extract intelligence from within the brain. No other road will get us there.". I agree that brain is the best example for us to build on, but by no means should we be so categorical and exclude other roads to high-level machine intelligence. Jeff points out early in the book that it was his strong intuition that made him rather sure that the Artificial Intelligence approach will fail to create programs that do what humans can do and will also fail to teach us what is intelligence in general. While I have actually had the very same intuition for a long time as well, and am thus very supportive to nonclassical approaches to AI, I nevertheless exercise caution about getting channeled into the other extreme, and I find it slightly alarming that Jeff seems (in my humble opinion) to fall into the very trap that he, ironically, cautions against just 20 pages later: "However, looking across the history of science, we see our intuition is often the biggest obstacle to discovering the truth".

But these small complaints of mine are mainly about the peripheral / general thoughts in the book, not about the core point which I find very interesting. So, overall I'd still say that the book "On Intelligence" is definitely worth reading for anybody interested in the workings of the brain and (but not necessarily) in how to create thinking machines.

More info about the book at Amazon: http://www.amazon.com/dp/0805078533

Friday 15 October 2010

Moment: Kiwi

Kitchen. Slowly and thoughtfully I roll a kiwifruit in my hand, feeling the softly rough texture, and carefully start peeling it. "Chile" says the sticker. I've never been to Chile... yet this furry little guy grew up there, happily, obliviously... And then an opportunity arose -- to break free, to see the World, to travel across the ocean towards the rising sun, to hear the plangent church bells of the old Europe... I paid for that trip. And now I'm gonna eat it.

[a little reflection to stimulate thinking about personal and global issues :]

Friday 17 September 2010

Book: Coders at Work

cover_CodersAtWork.jpg

A book by Peter Seibel, from 2009, consisting of interviews with various accomplished programmers:

  • Jamie Zawinski -- Lisp hacker, early Netscape developer.
  • Brad Fitzpatrick -- creator of LiveJournal, memcached, Perlbal.
  • Douglas Crockford -- creator of JSON, involved in the development of the JavaScript language.
  • Brendan Eich -- creator of JavaScript, current CTO of Mozilla Corporation.
  • Joshua Bloch -- has led the design and development of several major features of Java.
  • Joe Armstrong -- creator of Erlang and the Open Telecom Platform (OTP).
  • Simon Peyton Jones -- a major contributor to the Haskell programming language, and lead developer of the Glasgow Haskell Compiler (GHC).
  • Peter Norvig -- a computer scientist with wide-ranging interests, currently the Director of Research at Google.
  • Guy Steele -- has been involved in defining Common Lisp, Fortran, C, ECMAScript, Scheme, Java.
  • Dan Ingalls -- the main initial implementer of Smalltalk, inventor of BitBLT.
  • L. Peter Deutsch -- creator of Ghostscript, the author of some notable implementations of Smalltalk and Lisp.
  • Ken Thompson -- the main developer of Unix, creator of the B programming language, Belle chess computer, and UTF-8 Unicode encoding.
  • Fran Allen -- pioneer in the field of optimizing compilers, the first female IBM Fellow and the first female recipient of the Turing Award.
  • Bernie Cosell -- one of the main software developers for ARPANET routers, Lisp hacker, creator of DOCTOR (a version of ELIZA).
  • Donald Knuth -- the author of "The Art of Computer Programming", creator of TeX and METAFONT, inventor of literate programming.

The interviews are pleasantly lengthy and detailed, and include discussions on how the person got started with programming, their work on various projects, how their views on programming have changed over time, their approach to designing software, how they do debugging, their views on formal proofs of program correctness, how they approach reading code written by somebody else, their feelings about commenting and documenting the code, how to recognize a good programmer, the teamwork aspects of software projects, whether they consider themselves a scientist, an engineer, an artist, a craftsman, or something else, the question whether nowadays' programmers should bother learning what goes on at the low machine level, whether programming is a young person's game or can be done well by older people as well, and much more.

The scope of the discussions is satisfyingly broad, ranging from contemplations about big high-level issues down to the detailed stories and technical explanations without any fear of scaring off some potential readers / buyers of the book. Instead of trying to write a book acceptable to everybody (which, on the flip side, might not be outstandingly exciting to anybody in particular), the author has chosen a specific target audience and caters to it remarkably well.

So, "Coders at Work" is a collection of interviews with great programmers, done by a good programmer, and intended to be read by programmers. And if you ARE a programmer, there is a very high probability that you will love this book.

More info about the book at Amazon: http://www.amazon.com/dp/1430219483

Saturday 4 September 2010

Book: An Introduction to Theories of Learning

cover_TheoriesOfLearning.jpg

A book, the 7th edition (2005) of which I read, by B. R. Hergenhahn and Matthew H. Olson, about the main theories of learning, in a college textbook format.

The book starts by giving a general overview of the concept of learning and the use of scientific method. Then the early approaches are briefly described, starting from Plato and Aristotle and going through the various philosophers (Descartes, Hobbes, Locke, Hume, etc.) up to the early schools of psychology (voluntarism, structuralism, functionalism, early behaviorism). And after that the more detailed overviews of the major theories of learning are presented, which takes up the most of the book:

  • Predominantly functionalistic theories: Thorndike, Skinner, Hull.
  • Predominantly associationistic theories: Pavlov, Guthrie, Estes.
  • Predominantly cognitive theories: Gestalt theory, Piaget, Tolman, Bandura.
  • Predominantly neurophysiological theory: Hebb.
  • An evolutionary theory: Bolles and evolutionary psychology.

The final chapter of the book rather briefly discusses the current trends and some open questions.

I feel like I got quite a good general overview of the main ideas about the learning process: their historical development, key researchers and key points. The overall flow of the book followed the development of the theories in time, which provided the benefit of understanding why the things that currently seem obvious were not so obvious earlier -- when reading the older theories they seem to fully make sense and match experimental data, but then in the next section / chapter new (later) experiments and ideas are described that partially disprove some of the previous ones and form a new seemingly great theory... until another chapter brings yet another change. This also sustains the desire to keep reading to find out new and new things. On the other hand, the book isn't exactly leisure literature and at some points I really felt like taking a break or pushing myself a bit to keep going.

All in all, it was a very useful book for me: both for my research and for the general understanding of the world.

More info about the book at Amazon: http://www.amazon.com/dp/0131147226/

Thursday 26 August 2010

Book: Focus

cover_Focus.jpg

A book by Jurgen Wolff, from 2008, about how to fight procrastination and how to better focus on your most important activities and goals.

The feeling of not living up to your potential and letting the precious time slip by, day by day, year by year, can be extremely frustrating. The opposite -- fulfilling your potential and spending your time on activities that you love and / or that result in great achievements -- makes you feel happy and empowered. The good thing is that being able to focus on the activities that are important and positive for you is a skill and habit like any other -- you can learn it and become better by conscious practice. Thus, spending some of your time on improving your ability to work efficiently (as well as on other so-called "self-help" topics like improving your attitudes and social skills) is just as reasonable as spending some of your time on learning a foreign language, a musical instrument or some scientific theory.

Now, there already exist thousands and thousands of blogs and books about productivity and time management, offering a large number of tips and techniques, so why bother reading yet another one? Because Wolff's "Focus" has a somewhat nontraditional perspective and offers advice that at least I hadn't read anywhere else before. Wolff's view is that the majority of traditional time management methods are best suited for the analytical, well-organized, "left-brain" persons, whereas the more intuition-based, variety-loving "right-brain" people would benefit more from a slightly different approach that he then tries to provide.

Some of the advice Wolff gives is quite well-known (which, of course, does not diminish its importance), such as identifying your behavioral patterns, eliminating-delegating less important activities, setting big goals and dividing them into small subgoals, using visual reminders, focusing on positive things, creating the flow, and many more.

As of the less traditional suggestions, here's an example: be very careful about setting deadlines (especially if some parts of achieving the goal depend on external partially unpredictable factors), because failures to get things done by deadline are demotivating, and deadlines also tend to lead to the exponential increase of workload (and to a decrease in the willingness to look for alternative, more efficient routes, due to being in a hurry) just before the final date. A better approach might be to commit to moving steadily towards the goal in small steps (revising the strategies on the go as necessary) and achieving the goal whenever the steady progress has resulted in the desired state.

Other interesting advice includes the use of hero pattern (switching from "I have a problem / goal" mindset to "I am on a quest"); letting go of the belief that one person has one personality, and instead having a well-developed set of (possibly inconsistent) alter egos and for each situation picking the most suitable of them; taking the dividing of a big task into small steps to the extreme, e.g., making the first starting steps so absurdly tiny that they will not trigger postponing the task, for example if the goal is to start going to the gym regularly, then the first step might be literally stepping out of your front door with the gym bag, with the full permission to then stop and step back in (the trick is that the likelihood of continuing towards the gym instead of stepping back in is actually remarkably high); and much more.

Additionally, there are useful chapters on managing people and meetings, using language more efficiently (mostly NLP-inspired suggestions), dealing with the daily information overload, and so forth.

So, although "Focus" surely is not a magic book of spells that makes you highly efficient and happy overnight, it nevertheless gives very useful advice that is either complementary or alternative to the more traditional methods. If you are willing to make an effort to improve yourself, then this book most certainly can help you move faster towards your goals.

More info about the book at Amazon: http://www.amazon.co.uk/dp/0273715445

Sunday 15 August 2010

Book: Lies and the Lying Liars Who Tell Them

cover_LiesLyingLiars.jpg

A book from 2004 (first version 2003), where Al Franken, a US comedian and liberal, exposes some lies and dubious acts of Republicans and American right-wing media (Fox News and the like). Quite a hilarious book. Though, assuming that Al writes mostly truth, which he claims he does, it is also a rather saddening treatise -- I was kind of hoping that the politics and media in US are not THAT dirty.

So, if you'd like to read a highly satirical and amusing book about real American politics and media, then take a look at "Lies and the Lying Liars Who Tell Them". But, of course, keep in mind that it isn't exactly a balanced analysis.

More info about the book at Amazon: http://www.amazon.com/dp/0452285216/
and Wikipedia: http://en.wikipedia.org/wiki/Lies_and_the_Lying_Liars_Who_Tell_Them

And thanks to Chätrin who gave me this book as a birthday present!

- page 1 of 5