Sunday, October 14, 2007

Ruby, Smalltalk and Lisp Revisited

In my last post I suggested that Ruby may have some advantages over Smalltalk, especially when it comes to borrowing some of the more powerful features of Lisp. First thing to say is titling the post "Ruby versus Smalltalk" wasn't the cleverest thing to do. The responses tended to contain more heat then light. I hold up may hands and take full responsibility for this. Given the amount of FUD that Smalltalk has suffered over the years, it is hardly surprising that the Smalltalk community are protective.

I've been recently looking closely at Rubinius, a Ruby implementation that uses a Smalltalk/Squeak-like VM. Rubinius borrows many of the architectural ideas of Smalltalk/Squeak and is Ruby implemented mostly in Ruby. Eventually even the VM will be implemented in a C-like variant of Ruby called Cuby, much like how the Squeak VM is implemented in Slang.

I'm pretty taken by Rubinius. The approach taken is to implement Ruby right, and in so doing they've gone back to Smalltalk. The other thing that is very impressive is that all the Ruby libraries in Rubinius are implemented using BDD. So there are Rspec specifications for all library methods. The full Ruby library is not complete yet, but using Rspec is an excellent way to finally specify the Ruby language and in so doing allow lots of people (like myself :^)) to get involved in the Rubinous project.

So back to Lisp. The Rubinius compiler generates a simple Lisp (s-expressions) as an intermediate form on the way to byte code. There are lost of good reasons to do this, many of which are dicussed here:

http://on-ruby.blogspot.com/2007/01/will-rubinius-be-acceptable-lisp.html

Another reason not mentioned is that Lisp can become a common intermediate form allowing several source languages to interoperate. So Ruby or Smalltalk could be compiled to Lisp then Lisp to byte code. Peter Frisk has already implemented a Smalltalk to Lisp parser in Vista Smalltalk so this is possible. It also fits in nicely with the language orientated programming ideas that Martin Folwer has been touting for quite some time now.

Rubinius is gaining a lot of momentum in the Ruby community. So why is Lispyness important? I think the answer is flexibility and allowing change. Something that Alan Kay said seems relevant here. The first Smalltalk implementastions where built with Lisp I believe, and whilst Smalltalk was still within the confines of Xerox Parc the language was being experiemented with and changed all the time. Alan Kay complains that once Smalltalk became public in 1983, the language became fixed. He had hoped that Smalltalk-80 would be redundnant by now, and that it would have ben replaced by something better.

The Ruby community seem up for evolution, whilst also being a very practical bunch. The Rubinius project is cooperating with the JRuby project to specify a common Ruby across all implementations, something that I believe was a problem within the Smalltalk comunity. By considering how best to bring Lisp to Ruby they are also showing an openess to change. Matz seems open to change too, Ruby 2.0 although not revolutionary, does show a willingness to deprecate old features and break backwards compatibility, in the same spirit as pre Smalltalk-80 Smalltalks.

I still think that Ruby is closer to Lisp then Smalltalk, but that doesn't stop Smalltalk inlining Lisp or adoptng more Lisp like features if it wants to. In a sense the two languages share a common root. The difference seems to be history and the two communities. The Ruby community seem to be more willing to experiement and evolve, whilst the Smalltalk community is still holding onto Smalltalk-80 as the common Smalltalk dialect. I guess the question is whether the Ruby community can evolve together and stay as one, avoiding the fragmentation that occurred with Smalltalk.

From what I've seen on the web, there is plenty of reason for optimism. If the Rubinius/JRuby people pull it off we may end up in a situation where we don't need to choose. We can mix Ruby, Smalltalk and Lisp as and when appropriate.

4 comments:

gnupate said...

"Another reason not mentioned is that Lisp can become a common intermediate form allowing several source languages to interoperate. So Ruby or Smalltalk could be compiled to Lisp then Lisp to byte code. Peter Frisk has already implemented a Smalltalk to Lisp parser in Vista Smalltalk so this is possible. It also fits in nicely with the language orientated programming ideas that Martin Folwer has been touting for quite some time now."

Hmm, shades of rms and guile.

Mark Miller said...

Just wanted to make a correction. The first implementation of Smalltalk (72) was actually written in BASIC, of all things. It still translated instructions into bytecodes, and ran a bytecode interpreter. The early history of it is available here. From what I've heard Alan Kay say, after that initial implementation, Smalltalk was written in itself, and was bootstrapped to later implementations. I haven't gotten through the whole "early history" document yet. So I could be mistaken about this part.

They're currently working with a language called COLAs at VPRI, which is a Lisp-like language. I heard a rumor recently that it can be compiled to machine code, and the idea is they're compiling the higher level stuff to this Lisp-like layer, and then compiling that to machine code, so they don't have to use C at all, as they did with Squeak.

You are right that there is a tendency within the Squeak community to not want it to change, but there are those within it as well who have heeded Alan Kay's call for people to change it. This has forked the project, which personally I don't mind. People can use what they like best this way, and people can change it how they like without having to get permission from anybody. IMO this allows new ideas to proliferate. If you don't like change, you can use the standard distribution which doesn't change much. If you want something different you can use one of the forks.

Personally I think the advantage the Ruby community will get if Rubinious succeeds is this same ability of developers to create their own version of Ruby, since it'll be written in itself.

Paul Beckford said...

Hi Pate,

Not familiar with either of those languages. Martin Fowler has been talking about "Domain Specific Languages" and something he calls a "Language Work Benches":


A good read, but You're right, the idea is still unproven.

Mark:

Thanks for the correction. It goes to show you shouldn't believe everything you read on the web :^). I agree with your assessment, and yes, personally I do not see the number of Smalltalk dialects as a problem, in a real sense it is a strength. Unfortunately the "fragmentation" of Smalltalk as perceived by some is commonly seen as a negative thing. It will be interesting to see whether Rubinius can avoid the same "negative" vibe, and whether the "core" Ruby language manages to evolve whilst taking the majority of the Ruby community with it.

Paul Beckford said...

Hi Pate,

Forgot to paste the link to Martin Fowler's article opn language workbenches and DSLs:

http://martinfowler.com/articles/languageWorkbench.html