Newspeak - First Impressions - Underlying Beliefs

First thing to say, after watching Gilads presentation on Newspeak, is that I feel that I'm not qualified to deliver a rigorous critique. This in itself raises an interesting question. After 18 years software engineering experience my "education" is insufficient to evaluate a new OO language that builds on ideas that are over 20 years old. Why? Well, my professional career (the usual diet of Java APIs and frameworks) hasn't prepared me for Newspeak. In fact it is only my self education which I performed at home in my spare time that allows me to grasp most of what Gilad is speaking about. This telling fact says more about our industry I believe then it does Newspeak.

Beliefs and Motivation
You may find beliefs a strange place to start an evaluation of a programming language, but I think that the beliefs of the protagonist behind a language lay at the core of what a language is about, and whether it will appeal to me. Language design, like any other form of design is about trade offs, and it is your beliefs that determine the relative value you place on competing ideas and principles. Well the first thing to say is that Gilad is still a Smalltalker at heart, and he describes Newspeak as a direct decedent of Smalltalk. The other significant influences he mentions are Self, Beta and Scala in that order of significance. Scala seems like the odd man out here, but when you realise that its influence is much smaller then the others (the formalisation of object constructors, which is something Smalltalk never did), then the idea of Newspeak as a successor to Smalltalk makes sense.

My concern was whether Gilad believed in pandering to the needs of the tool over the beauty of the language from the perspective of the programmer. Well he makes his feeling clear in is preference for:

id = letter, (letter | digit) star. (Newspeak)

in comparison to:

id = letter().seq(letter().or(digit()).star(); (Java)

When attempting to express the BNF Grammar:

id = letter, (letter | digit) *

He refers to the redundant parenthesis in the Java version as "solution space noise". He says that what we should be doing is expressing our problem. Stuff in the code that is there to satisfy the tool (Compiler) is noise and should be reduced to a minimum. This belief sits fine with me. It also fits with my artist analogy. An artist is more interested in his subject then he is in his tools. Languages like C++ and Java just miss this point, and when I think about it, this is the central idea I learned from my first encounter with Smalltalk all those years ago. Just going through Squeak the other day I stumbled on things like this:

(2 + 3i) sqrt.

What do you think this does? Yes, you guessed it. It answers the square root of the complex number 2+3i. Now imagine expressing this in Java?

OK, if Smalltalk is so beautiful what is left to be solved in Newspeak? Well plenty actually. In terms of beauty, Smalltalk has its warts too. Like having to place the word "self" before each message send when sending a message to your self. Newspeak, like Self makes the receiver as "self" implicit.

The previous BNF grammar expressed in Smalltalk is:

id := self letter, ((self letter ) | (self digit)) star.

Which is more pretty then Java but not as pretty as Newspeak.

Smalltalk also lacked the ability to define encapsulated abstractions that were larger than a single Class. So although Smalltalk is uniform and "turtles" all the way down, when it comes to Classes everything is horribly coupled in a single global hashmap. Newspeak solves this lack of modularity problem too. So Newspeak is turtles all the way down and all the way up as well, leading to pluggable libraries etc. It achieves this by borrowing the idea of nested classes from Beta. So a module or a library is a class that contains other nested classes. I'll discuss this feature in more detail in a future post, but it is very significant and Gilad highlights it as the major difference between Newspeak and Smalltalk. It allows you to swap out abstractions at a category or library level with Newspeak without worrying about ugly inter-class dependencies (coupling). So Newspeak can be shrunk as well as extended unlike Smalltalk.

I've said that Smalltalk is turtles all the way down. Well that isn't quite true. All OO abstractions (turtles) should encapsulate their insides from the outside world. Smalltalk does do this for member variables, but in Smalltalk-80 all methods are accessible from outside the class in which they are declared, including methods that are meant to be private. When I first came across this, I thought it was a minor oversight that would be fixed soon, but interestingly the situation is still the same today in Squeak. Gilad intends to fix this oversight in Newspeak. So one of Gilads motives seems to be to finally address the design flaws in Smalltalk.

I find that many of Gilad's throw away ad-lib comments are more informative then much of his precise technical explanations. Throughout the presentation, and especially when he would switch to his Newspeak IDE, Gilad would make reference to the past failures of Smalltalk, and how he hoped to address those issues in Newspeak. In the eyes of many, Smalltalk never really made the transition from a research project to an industry ready software engineering platform. Strongtalk was one attempt to complete that transition. Strongtalk dropped the "many windows" class browser approach of Smalltalk-80 and adopted a collapsible pane approach that would be familiar to users of Eclipse today, and feels more natural to people coming from a "file" based metaphor for organising classes. Newpeak does the same. Thankfully Newspeak drops the type annotations of Strongtalk, with Gilad admitting that many of the late bound scenarios that Newspeak is designed to address would be almost impossible to verify statically. Gilad also plans to have a much improved FFI (foriegn feature interface?) for Newspeak, when compared with a typical Smalltalk dialect like Squeak.

Smalltalk as always been an island. Once you were on it the weather and the scenery was great, but you couldn't bring anything with you from the outside world except your bathing trunks, and you found yourself missing a bunch of stuff you had gotten attached to. In contrast Ruby fits into the Unix ecosystem like a dream, and you can use almost any windowing system, foreign language library and API available under Unix(/Windows/Mac OS X) from within Ruby. You can also "shell out" to a Ruby script from a Unix program too, so their is two way communication. Gilad spoke about a similar ambition for Newspeak with a FFI that allowed callbacks from foriegn functions, although he didn't go into this in detail. He does mention the use of Actors as a means of inter-process communication, so perhaps this is a pointer to how some integration may be done.

So to sum up, Gilad believes in clearly expressing the problem over assisting the tool (Man over Machine) and uniformity and modularity ("turtles") all the way up as well as all the way down. He is motivated by creating a successor to Smalltalk-80 that complies with sound software engineering principles and interfaces well with the current incumbent technology ecosystem. His aim is to create a language which is both esthetically pleasing and software engineering sound. The end goal is an elegant and consistent software engineering platform for use in industry.

I was going to give my impressions on how well Newspeak in its current incarnation actually delivers on these lofty goals, but this post is already long enough. Besides you can watch the presentation for yourself :) I'll be coming back to Newspeak in future posts, where I intend to pick up where I've left off here.

Update 29th April 2008
Gilad has kindly offered the following corrections and clarifications:

Re: your review. Well thanks! A quibble or two:

1. The basic FFI with callbacks is working; the higher level convenience layer isn't there yet. We are just calling to/from C (or Objective C) at the moment. Using actors to interface to the outside world was Alan Kay's original plan, but it was far ahead of its time.

2. Types. I haven't given up yet. It's just our lowest priority and a very hard problem. I would like to support pluggable types in Newspeak, and we currently support and encourage type annotations even though they are not checked.