jmora

Artificial intelligence, automation, and effectiveness.

[jmora]

New Year resolution: never program in Java again

Maybe it is a weird New Year resolution, but I can explain it.

For a long while, programming in Java has felt like digging with my own hands when there is a shovel free to use right next to me, Scala. For reasons.

Sure, it is not an excavator, the language is far from perfect and I hope new languages improve the current state of the art, preferably very high-level languages, strongly-typed and with nice tools, including an IDE that does the work for you and a compiler that generates very efficient runnable files. But at least it is better than Java (not that it's a hard thing) and can be used with the same libraries, so there are really no excuses to continue with that Blub language.

BTW: if you feel like discussing Clojure, please do so in the comments.

[jmora]

The 10x programmer is (not) a myth

Recently I have been seeing quite some few references to the 10x developer, whether it is a myth or reality. It is both.

On the one hand, if you think that you can keep your programming language, processes (including meetings) and get some individuals that are 10x (each) more productive than the folks next to them, then it is a myth. You can and should definitively forget about that. Certainly some people are going to be more productive than others, but an order of magnitude is “unlikely”.

On the other hand, you may find that a few guys using Erlang can produce more value than infinitely many startups and companies that eventually go bankrupt (negative worth). You may find that a few lines spent on a language like Haskell get much more done than 10x those lines in Java, or perhaps 100x, or 1000x those lines in assembler. It is not a matter of typing, but of thinking, of having into account the complexity that comes with the considered elements (e.g. state, types, data races, etc.). In short, it is thinking at a higher level, which means getting more done with less. Write less classes, less code, reuse more, and lots of overhead costs (e.g. tests) start shrinking orders of magnitude. It is not about solving 10x more useless problems, it is about avoiding those that can be avoided and solving the right ones.

Certainly the programming language, the libraries and frameworks, and even the tools may have a strong impact. In this scenario, achieving 10x productivity is much more likely. Considering other factors (meetings, clear goals, working on the right problem, etc.) the productivity may increase several (many) orders of magnitude. It is hard to estimate and hard to measure, unless you consider that in a capitalist society all work and value produced can be measured by money. throughout the years I have noticed improvements in myself not only due to expertise, as they would usually happen in specific moments (switching languages, tools, etc.), and it took an active search for improvement for years (I still miss the perfect programming language, though). In any case, if I can be 100x more productive than my former self, then so can one person be with respect to another.

As a conclusion, if you are a programmer and you want to be something else, the post may not be for you, but if you want to be a better programmer, or if you want your programmers to be more effective, then the question is what are you doing about it? It is your turn now.

[jmora]

UX is not UI, UX is AI

User experience (UX) is often associated with user interface (UI), sometimes even mistaken. The consensus is that UX is a broader term that includes the UI and some other aspects of the interaction of the user with the product (web page, web application, mobile app, desktop application or whatever).

There is a trend in the realization of the potential of artificial intelligence (AI) in UX and how they relate. Certainly, the intelligence that some product may feature will change the experience of its users and a usual cause of frustration is the lack of intelligence of some products.

The user experience describes how the interaction should be, but it does not change what the product does. AI, on the other hand, does exactly that. AI roughly consists on making software that does what humans do. Therefore, when successful, humans (users) do not need to do that anymore, and that is good, left aside games and recreational software, which are not means to get something done, but ends by themselves and bugs may be challenging features (in some cases). In general, the more that the user can get done with the least effort, the better.

In this sense, the best user interface, and a different approach to the “singularity” is a huge button with the label: “solve all my problems”. Pressing the button should fulfill this promise without causing other problems, and the user experience will depend on the extent to which this promise is fulfilled, i.e. the intelligence. If this such a product is ever built, it will be the last one. So far this is beyond the realms of possibility, the point is how much our solutions approximate to this goal and how much deviate from it.

What is stated here is also relevant for software engineering and the design of APIs, analogously to GUIs, and generalizable to all interfaces. Ideally, code should be usable as a black box without needing to go into details about its inner workings (abstraction), and the lack of intelligence is again a source of frustration. There are several attempts at solving this issue, one of them being convention over configuration, which is intelligent in the sense that it saves the user (programmer in this case) from having to configure something, by applying a conventional configuration by itself.

[jmora]

[jmora]

AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking.'

Donald Knuth

[jmora]

What I am most fond of are beautiful and simple theoretical ideas that can be translated into something that works. I have very little patience for people who do theory about a particular thing simply because it’s easy—particularly if they dismiss other methods that actually work empirically, just because the theory is too difficult. There is a bit of that in the machine learning community. In fact, to some extent, the “Neural Net Winter” during the late 1990s and early 2000s was a consequence of that philosophy; that you had to have ironclad theory, and the empirical results didn’t count. It’s a very bad way to approach an engineering problem. But there are dangers in the purely empirical approach too. [...] that stifles creativity, because to get to the level where if you want to beat other teams that have been at it for years, you need to go underground for four or five years, building your own infrastructure. That’s very difficult and very risky, and so nobody does it.

Yann LeCun

[jmora]

Yet the irony of artificial intelligence, many point out, is that once it arrives, it no longer seems particularly intelligent. AI is a fuzzy category. If artificial intelligence is what humans can do but computers can’t, once computers can do it, AI disappears.

Geoffrey Hinton

[jmora]

Foreseeability

It is a word, I checked that.

In short:

Big developments are easy to spot in the future, they take a lot of hard work, and approaching to them feels terribly slow as we become aware of the challenges implied and their dimensions. Small developments are not so easy to spot, but they may have big implications and consequences. Finally, educated guesses are possible.

“In long”: 

There is a fallacy to state that humanity can achieve anything (and everything) in the future, and it is likely to happen. It is based on some biased “evidence” in the form of: “nobody saw whatever coming”, where whatever may be the Internet, smartphones or anything else. It can be extended with “look how things were X years ago and how they are now!”

New stuff that was reasonably unpredictable is produced constantly, but silently and inadvertently, hitting the markets by surprise and not at such a scale. Stuff like artificial intelligence, ageing reversal, or curing cancer gets a lot of attention from companies, people and governments and it is unlikely to be surprising.

For example it could be the case that Google’s self-driving car is a delusion, so that everybody believes that the state of the art is different from what Google can do, while some secret lab at Google develops some artificial intelligence (AI) much more intelligent than a whole nation, in raw power and generality. There are strong arguments against that (e.g. the economic argument), so I completely disregard that possibility. If the state of the art is the self-driving car, Watson, BigDog, etc. then the kind of research that is being done on AI does not lead to Skynet, neither in the next decades, nor ever.

Not only that, the fact that non-predicted stuff happens does also defy the argument, because most predictions are also wrong being worse as they are further in the future. When something not predicted happens, it changes the state of the art and with it something that could be predicted before may become pointless or impossible. If something like some AI that will take humanity to hell or heaven was near, we would not be able to say that nobody saw it coming, many people do (mistakenly, IMHO).

IMHO, usually the big changes that happen without being foreseen do so through the implications of small changes. Internet may not seem like a big deal having telegrams, but it enables many things, including the web, which may again not seem a big deal having yellow pages, but it is an enabler of further changes, including the web 2.0 with the forums and everything else, and it may not seem like a big deal when compared with the Pasquino and classified ads, but I think you get the point.

Finally, there may be some laws that prevent something from happening. For example time travel is impossible according to the laws of physics, and if it were not then it would make no sense according to logic and the many paradoxes that would arise in such an event. Scientific (empirical or formal) knowledge can allow some educated guesses about what will happen (or be possible) in the future and they provide more information about what is impossible.

[jmora]

Personal interest on artificial intelligence

I am close to finishing my PhD on Artificial Intelligence (AI) and (IMHO) it is time to look back in time and to look forward.

As a child I found amusing stuff like Shortcircuit and Transformers. Especially in the case of Shortcircuit it looked quite realistic in the movie, the limitations seemed to be in the software (obviously, it was manually operated) and not on the hardware. As I grew, I noticed that having such a kind of AI would allow solving lots of problems, in fact it would solve all the other problems. So I studied Informatics Engineering (that’s the literal translation for the equivalent of Computer Science).

After becoming an engineer I felt my knowledge about AI was falling short, so I decided to study a PhD as well. I did read a lot about strong AI, artificial general intelligence, seed AI, artificial curiosity, the singularity, and whatnot. I found a number of researchers like Jürgen Schmidhuber, Marcus Hutter, Ben Goertzel and Raymond Kurzweil. However I didn’t find much applied research, which I like because in some sense it allows closing the loop with feedback about how what has been done works. I continued at UPM as I could not find any viable alternative. There I asked my supervisor about how could I do something related with the next picture:

The picture is fairly vague and useless, even though it represents to some extent the part of the cybernetics loop that corresponds to the machine. It was my best attempt to put together everything that I had learned about AI so far. Finally, I focused on knowledge representation, as I believed and still believe that it has the best chance to bring some significantly performing AI in a reasonably short term and I consider it to be crucial for future developments. For example, currently there is a lot of attention to machine learning, but the whole area and applications of machine learning would be much more powerful if the knowledge acquired through machine learning could be easily shared among systems.

However, nowadays I feel that such a challenge is too complex to be achieved. I am currently interested in machine learning and data analysis to understand (in a deeper way) how these systems work and perhaps how the knowledge should be modeled to be useful for them. I will probably continue working in the area of knowledge representation, due to my background and experience, but I will try to start a smooth transition towards these areas as part of a longer term strategy for the real goal of seed AI.

Finally, I think that the main aspect to keep in mind is how AI has been fragmented into narrow AI, focusing on domains, applications, technologies and problems, without keeping in mind the integration of these developments into a greater architecture for AGI. This is not the usual approach in software engineering where a system is split into modules but they are always meant to work together and that is kept in mind (ideally at least). IMHO, the key to strong AI (a very hard problem) consists on splitting such a problem into several modules that are easier to build (and reason about) while keeping in mind the broader perspective of the architecture and the system.

[jmora]

ToDo: Artificial intelligence

As commented in the previous post, there has been some hype about artificial intelligence (AI) recently. Some fear a new AI winter, some see in AI an existential risk, and some think that AI will evolve to artificial general intelligence and then artificial superintelligence in the next decades, becoming indeterminately more intelligent than the whole humanity, leading humanity to immortality.

There are two groups of factors to consider in these speculations about the future. One group refers to the economy and the other group refers to the progress of AI.

AI may have a disruptive effect on economy. Regardless of the likelihood of such an event, IMHO, if a job can be properly done by a machine, then people should probably do something more important. The shift in the skills and jobs needed could be a problem to solve, perhaps hard times for many, but not an existential risk. Similarly, economic circumstances may reduce the investment on AI, causing an AI winter. The likelihood of such an event is beyond my current knowledge and the scope of this blog post.

On the side of the AI progress there are no reasons to believe that AI will cause neither our extinction nor our immortality in the foreseeable future. The reason is that the kind of AI and research on it that could lead to those outcomes has not started yet, and not even its beginning is foreseeable at this moment.

Computers can be programmed further than other machines to display some behaviour that is not constant, e.g being a function of the input and some internal state. AI can modify this function automatically, e.g. machine learning, and in some cases, like the again popular neural networks, any function can be approximated (they are universal approximators). This certainly gives a lot of “freedom” to computers, being able to do anything that is computable, with many references to “universal” stuff. However there are strong bounds not in their intelligence per se, but in their universe.

For example, for a self-driving car pedestrians may be labeled as “person”, but they are just moving obstacles to avoid, the knowledge representation used for the car may include everything that can have an address or GPS coordinates, and every object (including people) with which it must not collide. This model could include further features like eye tracking to consider potential changes in the direction of the movement of people. However, this is not an input of the system, considering such fine grained details in such a complex scenario would require a far greater computational power with current techniques, so nobody does it, AFAIK. For the AI in the car there is no difference between people and dummies that move: colliding with them must be avoided. The machine learning in self-driving cars may learn any function, but all the functions that it can learn are enclosed in some space and nothing beyond that space exists for the AI.

There is a second example, also from Google, of an AI that plays games. Old 2D Atari games are fairly straightforward, it is a universe of pixels and it is possible to do reasonably well without some deep knowledge about what is on the screen and its meaning. Technically it should be feasible to move on to 3D games and then to robots, learning to move, including learning to walk like a person, similarly to what BigDog does. These are fairly unconstrained spaces but they still define the boundaries for the AI, and then there are the boundaries on the goals and the means. Moreover, each of these steps is going to take a lot of work and time and even if they are successfully achieved they would mean no progress towards consciousness or a self-improving AI.

I am aware of genetic programming and neural Turing machines. However, in general applications AI does not self-improve by modifying its goals, inputs or means. AI tries to find the best function in some space of functions (potentially infinite) and not beyond. The spaces of functions that could lead to some self-improving AI are very far away from current research. This means that the progress in AI is manual, it has to be crafted, progress may be empowered by a growing population and PhD numbers, but challenges become harder, costs grow, and a winter may be near.

There may be many things to fear or to hope for in the next decades, but the technological singularity is not one of them. This is one thing I would be happy to be wrong about.

[jmora]

Artificial intelligence - hype again?

Artificial Intelligence (AI) is very present in the media and our lives due to recent developments in the field. For example the research done by many people as Geoffrey Hinton and Yann LeCun is getting a lot of attention due to the good results obtained with Deep Learning. In addition to that, big companies are heavily investing on AI, e.g. IBM. 

We have “known” for a while that machines can think, (in fact, perhaps humans cannot) and there is no reason to believe that they cannot be as intelligent as humans, or that machines can be more intelligent than humans. This could make for very interesting games, not just chess and Jeopardy, but also using AI in video games. That is current technology, nevertheless, players and developers often choose multiplayer for a more “intelligent” experience in a trend that looks quite alive (I would personally say that is going to stay at least for a few decades). In any case, AI is really everywhere, from spam filters, to Internet search results, recommendation systems, etc.

However, some fear that we may be very close to an utopian or distopian future. Elon Musk seems to be afraid of AI as to say that it could be more dangerous than nukes, or like summoning the demon. Stephen Hawking warns about the possible end of mankind. For Andrew Ng it is clear that robots won’t kill us, and Deep Learning is a mandate for humans. They may not kill us but robots may make us their pets, Steve Wozniak points. At the same time Jeff Hawkins ensures that the Terminator is not coming.

As with any new technology (or further improvement of not-so-new technology) it is hard to avoid some hype. Personally, I cannot see how the narrow AI that is being researched nowadays could get out of control, how it could lead to some “superintelligence”, to our immortality or extinction. Climbing a tree will not take us to the Moon, unless we climb exceptionally fast, but that takes the metaphor too far (that is why I do not like metaphors).

I think most people like the Roomba and similar robots, and I guess most people would like self-driving cars, except perhaps cab drivers, or lorry drivers, or people working in delivery... Some people may not like AI, however the reasons may not be so much into the end of mankind but economy. IMHO, the chances of creating Skynet accidentally (or purposely) are close to zero, and they will continue that way for centuries.

There seems to be no clear definition about what AI is, which causes great confusion, including unrealistic expectations and some judgement of current developments that may be out of proportion and drawing attention from more important (even urgent) matters. IMHO, every definition will have supporters and detractors. Additionally to that, if definitions are not formal then they are not fully clear, and a formal definition is a hard enough task as to be worth a paper, or a book.

In the next blog post I will try to make some limitations in current research and technology clear because if there is any danger in AI it is in the economic area, and IMHO there are greater risks in that area to worry about at this moment.

[jmora]

[jmora]

Ancient maintainability costs

What if I told you that there is a system, probably one of the biggest ever made, much bigger than Windows, probably one of the oldest systems. A system that cannot be compiled anymore, which has to be manually checked for errors through code inspection and in all likelihood too big for a single human mind. A system that works as an ever increasing library, providing more and more functions that can be used and must be used to avoid code duplication. A system composed nearly entirely of plain fragmented documentation that cannot be put together and whose foundations and most inner workings are not-so-well documented because that's in fact quite hard to explain, everybody has a fairly good idea and anyway you are meant to be working on the highest level functions, leaving the low level work for the greatest among the greatest. 

What if I told you the name of this system is "logic".

You would probably say you were thinking about something else. Well, I was trying to speak about logic, as the analogy with a programming language is more straightforward. The analogy can probably be applied to many other areas. In particular, the maintainability, specially with respect to adding new functions, may be a greater issue in other areas.

For example, in maths, as time passes, the variance of the age of authors of big mathematical discoveries seems to reduce (AFAIK, if someone has the data please let me know), on one side people would be too young to acquire the deep knowledge that is required, on the other side people would be too old and the cognitive decline due to aging would be noticeable for the most demanding tasks.

This could be a problem in the future, if progress becomes nearly impossible and an impasse is reached. I would like to know the solution that is found, assuming that there is a problem, a solution for it and both are found at some point.

[jmora]

Programmer pride

Any potential follower of the blog may be wondering why in the last three weeks there were just quotations of some other texts. There are several reasons for this.

This is a very convenient way to keep the blog alive for weeks with a just little time invested (I'll try not to abuse on it).

I like to keep posts short, so those extracts made for three full posts, and commenting on them is the fourth (this one).

I didn't put the full texts but extracts that I consider particularly relevant, again to keep them short. I however recommend reading them fully, especially Dijkstra's.

There are several points that I would like to mention, but elaborating in them would make this post too long for microblogging. The points are:

Life's short, stay passionate about what you do, do things that matter.

Programming (properly) takes some effort and dedication. Similarly for learning and mastering new programming languages. It all pays off.

Code should be obvious. Perhaps there isn't much to do in assembler, but any reasonable modern language should allow for self-documenting code.

Programming can produce great added value per hour, hence career progression. That depends on the abstraction level and re-usability of the code. In the end it’s always about getting things done.

If you feel the "need for speed" and you like optimizing programs, perhaps you would like to optimize innumerable programs at the same time by working on compilers. (Indeterminable added value per worked hour, depending on the number of programs that benefit from it in the end).

The programming language is important, data structures and algorithms matter more, but think about implementing an algorithm relying on higher-order in Java before Java 8. What about a Map/dictionary (basic data structure) from types to functions to apply?

Abstraction and expressive power lead to precise and concise code, i.e. one-liners, which are fine as long as they are self-documenting (not cryptic!)

“Always” prefer common sense to dogmas.

You may wonder the reason for the title. Be very humble to the unforgiving code, but if you like programming, be proud of being a programmer, regardless of managers, and especially towards people who make no mistakes.

[jmora]

There is a clear need to point out the differences between the typical high-school junior Pac-Man player and a Real Programmer. [...] The easiest way to tell a Real Programmer from the crowd is by the programming language he (or she) uses. Real Programmers use FORTRAN. Quiche Eaters use PASCAL. [...] If you can't do it in FORTRAN, do it in assembly language. If you can't do it in assembly language, it isn't worth doing. My first task in the Real World was to read and understand a 2O0,OO0-line FORTRAN program, then speed it up by a factor of two. Any Real Programmer will tell you that all the Structured Coding in the world won't help you solve a problem like that -- it takes actual talent. [...] Real Programmers can write five-page-long DO loops without getting confused. Real Programmers like Arithmetic IF statements -- they make the code more Interesting. Real Programmers don't need comments -- the code is obvious. [...] As all Real Programmers know, the only useful data structure is the Array. [...] Your typical Real Programmer knew the entire bootstrap loader by memory in hex, and toggled it in whenever it got destroyed by his program. (Back then, memory was memory -- it didn't go away when the power went off. Today, memory either forgets things when you don't want it to, or remembers things long after they're better forgotten.) [...] the Real Programmer wants a `you asked for it, you got it' text editor [...] What sort of environment does the Real Programmer function best in? [...] it's best to put him (or her) in an environment where he can get his work done.

``Real Programmers Don't Use PASCAL'' from Ed Post

[jmora]

It has been suggested that there is some kind of law of nature telling us that the amount of intellectual effort needed [to design a program] grows with the square of program length. But, thank goodness, no one has been able to prove this law. And this is because it need not be true. We all know that the only mental tool by means of which a very finite piece of reasoning can cover a myriad cases is called “abstraction”; as a result the effective exploitation of his powers of abstraction must be regarded as one of the most vital activities of a competent programmer. In this connection it might be worth-while to point out that the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise. Of course I have tried to find a fundamental cause that would prevent our abstraction mechanisms from being sufficiently effective. But no matter how hard I tried, I did not find such a cause. As a result I tend to the assumption —up till now not disproved by experience— that by suitable application of our powers of abstraction, the intellectual effort needed to conceive or to understand a program need not grow more than proportional to program length. [...] Now for the fifth argument. It has to do with the influence of the tool we are trying to use upon our own thinking habits. I observe a cultural tradition, which in all probability has its roots in the Renaissance, to ignore this influence, to regard the human mind as the supreme and autonomous master of its artefacts. But if I start to analyse the thinking habits of myself and of my fellow human beings, I come, whether I like it or not, to a completely different conclusion, viz. that the tools we are trying to use and the language or notation we are using to express or record our thoughts, are the major factors determining what we can think or express at all! The analysis of the influence that programming languages have on the thinking habits of its users, and the recognition that, by now, brainpower is by far our scarcest resource, they together give us a new collection of yardsticks for comparing the relative merits of various programming languages. The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague. In the case of a well-known conversational programming language I have been told from various sides that as soon as a programming community is equipped with a terminal for it, a specific phenomenon occurs that even has a well-established name: it is called “the one-liners”. It takes one of two different forms: one programmer places a one-line program on the desk of another and either he proudly tells what it does and adds the question “Can you code this in less symbols?” —as if this were of any conceptual relevance!— or he just asks “Guess what it does!”. From this observation we must conclude that this language as a tool is an open invitation for clever tricks; and while exactly this may be the explanation for some of its appeal, viz. to those who like to show how clever they are, I am sorry, but I must regard this as one of the most damning things that can be said about a programming language. Another lesson we should have learned from the recent past is that the development of “richer” or “more powerful” programming languages was a mistake in the sense that these baroque monstrosities, these conglomerations of idiosyncrasies, are really unmanageable, both mechanically and mentally. [...]

Edsger W. Dijkstra, extracts from The Humble Programmer.

[jmora]

To some of us, writing computer programs is a fascinating game. A program is a building of thought. It is costless to build, weightless, growing easily under our typing hands. If we get carried away, its size and complexity will grow out of control, confusing even the one who created it. This is the main problem of programming. It is why so much of today’s software tends to crash, fail, screw up. When a program works, it is beautiful. The art of programming is the skill of controlling complexity. The great program is subdued, made simple in its complexity. Today, many programmers believe that this complexity is best managed by using only a small set of well-understood techniques in their programs. They have composed strict rules about the form programs should have, and the more zealous among them will denounce those who break these rules as bad programmers. What hostility to the richness of programming! To try to reduce it to something straightforward and predictable, to place a taboo on all the weird and beautiful programs. The landscape of programming techniques is enormous, fascinating in its diversity, still largely unexplored. It is certainly littered with traps and snares, luring the inexperienced programmer into all kinds of horrible mistakes, but that only means you should proceed with caution, keep your wits about you. As you learn, there will always be new challenges, new territory to explore. The programmer who refuses to keep exploring will surely stagnate, forget his joy, lose the will to program (and become a manager).

E. Hoigaard, Smooth CoffeeScript