rubydbc

Design by Contract for Ruby is now a Rubyforge project. Version 1.0.0 is available as a gem:

gem install rubydbc

Atom 1.0 parser for Ruby

I've been looking for a Ruby library for parsing Atom 1.0. The only one I could find was FeedTools. Unfortunately, it's intended to be a universal feed-parsing library and not quite what I need.

As a result, I decided to build my own. The project is hosted at RubyForge and the source code is available under the MIT license.

It is still in its early stages, so don't expect much at this point. So far, it understands most atom:* elements and attributes but doesn't perform validation.

The interface is fairly simple. For example, you can do this:

require 'atom'
require 'net/http'
require 'uri'

str = Net::HTTP::get(URI::parse('http://blog.ning.com/atom.xml'))
feed = Atom::Feed.new(str)

feed.entries.each { |entry|
puts "'#{entry.title}' by #{entry.authors.first.name} on #{entry.published.strftime('%m/%d/%Y')}"
}

Give it a try, play with it. Comments are welcome!

Where am I?

Need to get the name of the current method in Ruby?

module Kernel
def current_method
caller[0].sub(/.*`([^']+)'/, '\1').to_sym
end
end

irb(main):006:0> def test
irb(main):007:1>    p current_method
irb(main):008:1> end
=> nil
irb(main):009:0> test
:test
=> nil
irb(main):010:0>

If you want to get fancier, you can get get a reference to the method itself:

module Kernel
def current_method
method(caller[0].sub(/.*`([^']+)'/, '\1').to_sym)
end
end

Design by contract for Ruby

Here's a nice little module Brian and I hacked together a few weeks ago. It adds basic design by contract capabilities to classes with a fairly clean syntax (yay for Ruby's flexibility!)

Take a look for yourself:

class Foo
   include DesignByContract

   pre(:sqrt, "value must be >= 0") { |v| v >= 0 }

   pre("divisor must be != 0") { |dividend, divisor| divisor != 0 }
   post { |result, a, b| a - b * result < 1e-3 }
   def div(dividend, divisor)
      dividend / divisor
   end

   def sqrt(value)
      Math.sqrt value
   end

   post(:sqrt, "error is greater than expected") { |result, value|
         result * result - value < 1e-3
   }
end

When the DesignByContract module is included in a class, the pre and post methods become available. The parameters to these methods are a symbol representing the method to add the pre or post condition to, the message to print if the condition fails and a block that implements the condition.

The method name and message are optional, though. If the method name is missing, the condition will applied to the first method definition following the pre/post declaration. Note that if the method name is present, the condition can be defined anywhere in the class definition, as shown in the example above.

The condition block receives all the arguments passed to the method it applies to. They must return a boolean value indicating whether the condition succeeded. A minor difference between pre and post blocks is that the post block receives the result returned by the method as the first argument. This makes it possible to validate the results against the inputs such as in:

post { |result, dividend, divisor| dividend - divisor * result < 1e-3 }

Get it here!

How it works

In a nutshell, the pre and post methods intercept the existing method using the technique I described a few weeks ago (i.e., no poluting the class' namespace with aliased methods). The new methods test the provided condition before or after delegating the call to the original method and raise an exception if the condition fails.

There is a small caveat, though. If pre/post are called before the method is first created, it cannot be intercepted. To get around this the module uses an alternative trick. Whenever the module is included, it hooks into the method_added callback of class it's included into. If pre/post are called and the corresponding method does not exist, the interception is scheduled until the method is added.

Replacing methods

Let's say we want to replace a method instead of just throwing it away. But we want the new method to be able to call the old implementation.

The standard way of doing this is by aliasing the old method and making the new one call the aliased version:

class Foo
  def method
    ...
  end

...

class Foo
  alias :method :old_method

  def method
    ...
    old_method
  end
end

This solution works perfectly for most situations, but what if we want to remove all traces of the old method?

Here's a simple trick to do exactly that:

class Foo
   old_method = instance_method(:method)

   define_method(:method) { |*args|
       # do something here ...
       old_method.bind(self).call(*args)
   }
end

It shouldn't take much to understand how it works, but let's analyze it in more detail. The first line gets a hold of the old implementation of method. instance_method returns an UnboundMethod object that wraps it.

Next, we redefine method and pass it a block with the new implementation. One thing worth mentioning is that the old_method variable inside the block is bound to the variable assigned to outside of the block. The binding is maintained even after the execution reaches the end of the class definition. As a result, the new method has a direct reference to the old one... and nobody else does.

After doing whatever it has to do, the new method calls the old one after binding the UnboundMethod to the current object (i.e., the object on which method was invoked).

If you have a keen eye, you'll notice that there's something special about self. At first glance, it looks like a standard variable that just happens to hold a reference to the current object. If that were true, it would follow that the reference to self inside the block would be bound to the self representing class Foo. That is not the case, though. The binding of self is established when the block gets called.

Removing methods from a module

How do we remove a method from one of Ruby's built-in modules (Kernel, etc)? It should be easy and obvious, right? The following snippet should be enough, right?

#remove system from module Kernel
Kernel.send :remove_method, :system

Well, not quite. While that prevents you from calling system() in the context of the current object, you can still invoke it like this:

Kernel::system('date')

What's going on?

It turns out that they way Kernel (and other built-in modules) are defined is analogous to doing the following:

module Kernel
  def system (...)
    ...
  end

  module_function :system
end

Thus, system() is a method that will be added to instances of classes that include the given module (in our case, instances of any object, since Kernel is included by Object), but it is also a method in Kernel's singleton class (bear with me for a moment), or a module_method as the PickAxe book calls them. In Java parlance, they are simply static methods.

Looking at Ruby's source code, we can see that all of Kernel's methods are defined by way of the rb_define_module_function function, whose definition is:

void
rb_define_module_function(module, name, func, argc)
  VALUE module;
  const char *name;
  VALUE (*func)();
  int argc;
{
  rb_define_private_method(module, name, func, argc);
  rb_define_singleton_method(module, name, func, argc);
}

So, what does it mean that "system" is a method of Kernel's singleton class? Let's recap what happens when we create a class and add a method to the singleton class of one of it's instances.

class A
end

V1 = A.new    # creates an instance of A and assign it to constant V1
V2 = A.new    # ... and V2

# now we add method x() to V1's singleton class
class << V1
   def x
     puts "hello"
   end
end

V1.x    # will print "hello"
V2.x    # will fail ... V2 does not have a method x()

Clear?

Ok, now, remember that all Ruby modules are instances of a class called Module? Say we replace A by Module, V1 by Kernel and x() by system(). What we get is the following:

Kernel = Module.new

class << Kernel
   def system
     ...
   end
end

And that's essentially what's going on behind the scenes.

Going back to the question that started this post, how do we remove a method from one of the built-in modules, anyway? The answer is easy, if not as obvious as we might have expected:

module Kernel
   remove_method :system
end

class << Kernel
   remove_method :system
end

Or, more concisely,

Kernel.send :remove_method, :system
class << Kernel; self; end.send :remove_method, :system

Ruby's executable class definitions

If you're coming to Ruby from Java, remember that class definitions are executable code. I can think of at least one situation where this has practical implications and if you're not clear about it you'll spend a couple of hours banging your head against the wall.

Consider this simple Java class:

class A
{
  int a = 10;

  public int getA()
  {
    return a;
  }
}

If you create an object of type A and call it's method getA(), you'll get 10. No surprises here.

Now, knowing that member variables in Ruby are prefixed with @ we go ahead and translate our little java class to Ruby like so:

class A
  @a = 10

  def a
    return @a
  end
end

We then create an object of class A and call its a() method:

irb(main):001:0> class A
irb(main):002:1>   @a = 10
irb(main):003:1>   def a
irb(main):004:2>     return @a
irb(main):005:2>   end
irb(main):006:1> end
=> nil
irb(main):007:0> A.new.a
=> nil

"WTF!!" you say. Well, if you remembered what I said, class definitions are executable code. In our case, this means that the @a = 10 is executed when the class is defined, not when an instance of the class is created. As a result, this statement defines a member variable of the instance of class Class that represents class A.

To prove the last statment is true, let's make a() a class method:

class A
  @a = 10

  def A.b
    return @a
  end
end

When we call it we get what we expected:

irb(main):001:0> class A
irb(main):002:1>   @a = 10
irb(main):003:1>   def A.a
irb(main):004:2>     return @a
irb(main):005:2>   end
irb(main):006:1> end
=> nil
irb(main):007:0> A.a
=> 10

ANTLR for Ruby

The latest release of ANTLR v3 adds support for Ruby as a target language, courtesy of yours truly. Let's look at an example to see what's possible.

The following grammar generates a parser that can evaluate simple arithmetic expressions involving integer numbers, +, -, *, /, ( and ).

grammar Calculator;
options {
  language = Ruby;
}

@members {
  @stack = []

  def result
    @stack[0]
  end
}

parse: expression;

expression: mult (
    '+' mult {
      @stack.push(@stack.pop + @stack.pop)
    }
  | '-' mult {
      a = @stack.pop
      b = @stack.pop
      @stack.push(b - a)
    }
  )* ;

mult: atom (
    '*' atom {
      @stack.push(@stack.pop * @stack.pop)
    }
  | '/' atom {
      a = @stack.pop
      b = @stack.pop
      @stack.push(b / a.to_f)
    }
  )* ;


atom: n=NUMBER { @stack.push($n.text.to_i) }
  | '(' expression ')';

NUMBER: ('0'..'9')+;

WS: (' ' | '\n' | '\t')+ { channel = 99 };

First, we need to process the grammar with ANTLR. You'll need to have Java installed in order to do this. ANTLR will generate two files: Calculator.rb and CalculatorLexer.rb.

You'll also need the ANTLR ruby runtime library (just a small .rb file) which you can get here. Just drop into the same directory as the two .rb files created above (in the future I may package it as a gem).

We got our parser, so let's try it out. Create another ruby file with the following:

require 'Calculator'
require 'CalculatorLexer'

lexer = CalculatorLexer.new(ANTLR::CharStream.new(STDIN))
parser = Calculator.new(ANTLR::TokenStream.new(lexer))
parser.parse

puts parser.result

Now, run the program, type an expression and press CTRL-D (or CTRL-Z if in Windows) to see the results, like so:

(5 + 4) * (10 - 2) / 3
^Z
24.0

Simple... but cool, eh?

ANTLR v3 is still in pre-alpha stage, and the Ruby backend is a work in progress. So far, it can handle syntactic and semantic predicates, token labels, etc. and I'm planning to add support for scopes and token/rule parameters and return values next. AST creation will come after that.

ANTLR-based Ruby grammar

It looks like other people are showing interest in replacing the YACC-based Ruby parser by one based on ANTLR. This hits close to a project I've been working on in my spare time.

When I looked at JRuby's source code a few weeks ago, I thought I'd start trying to understand how the parser works. I had a few ideas I wanted to experiment with, but...

The YACC-based parser is too hard to fathom. There are too many scattered pieces with no clear dependencies. Making changes without breaking things should be even harder.

So I decided to start working on ANTLR-based grammar to replace the current implementation -- pretty much what MenTaLguY is trying to do. I'm still in experimentation mode and trying to figure out how I'll deal with some of the features of Ruby that are not usually found in other languages.

One of the hardest aspects, I think, is how to parse %Q-delimited strings. Here's how they work (from the Pickaxe book):

Following the type character is a delimiter, which can be any nonalphabetic or nonmultibyte
character. If the delimiter is one of the characters (, [, {, or <, the literal
consists of the characters up to the matching closing delimiter, taking account of nested
delimiter pairs. For all other delimiters, the literal comprises the characters up to the
next occurrence of the delimiter character.

Additionally, %Q strings can contain #{} constructs, whose contents should be parsed as Ruby expressions. These can probably be handled in a manner similar to how this grammar for the E language implements what they call quasi-literals.

I haven't yet decided on how to deal with delimiters but I have the feeling it may require writing a custom lexer for string contexts.

const_missing

There's an ongoing weekly quiz in the Ruby mailing list. This week's theme is roman numerals -- writing a program that can read roman numerals from the standard input and print out their arabic representation. A few solutions have been posted, standard stuff. But Dave Burt complemented his with a very neat hack. He makes it possible to treat integers and roman numerals interchangeably, and to perform arithmetic operations with them:

VII + 3 -> X
XIV / 2 -> VII
VIII.divmod(III) -> [II, II]
etc.

Part of this is achieved by redefining the const_missing method of the Object class, which kicks in whenever the interpreter finds reference to an unknown constant.

def Object.const_missing sym
    # verify that the constant is a roman numeral
    raise NameError.new("uninitialized constant: #{sym}") unless RomanNumerals::REGEXP === sym.to_s
    # define the constant and return it
    const_set(sym, RomanNumeral.get(sym))
end

The arithmetic operations are made possible by redefining RomanNumeral's method_missing method with code that delegates unknown operations to the it's integer representation.

Take a peek at the complete solution.

building blocks

One of the features I like best in Ruby is the support for blocks. Blocks are, essentially, chunks of code that can be passed around as objects and invoked at a later point in the program (also known as lambdas in other languages). But they are more than just code. When a block is defined, the context in which it is defined (variables, constants, etc) goes with it.

You define a block like this:

b = lambda { |value| puts value }

c = lambda do |value|
          puts value
      end

Either {} or do...end can be used as delimiters. The names between | are the arguments to the block. Any number of arguments can be specified by separating them with comma: |value, i, j|

To call the block:

b.call("hello")

The need for call may go away, though.

A little syntactic sugar allows us to pass blocks to methods implicitly. As far as I know, this syntax originated as a way to abstract looping behavior, one of the key applications of blocks in Ruby.

This is how it works:

methodThatTakesBlock { |value| puts value }

Sweet, eh?

A method that takes a block can refer to it implicitly or explicitly. Implicitly:

def methodThatTakesBlock
     yield "hello"
end

The key here is the yield keyword, which calls the provided block with one or more values as arguments. yield can be called any number of times, for example:

def methodThatTakesBlock
     yield "hello"
     yield "world"
end

But what if you need to keep a reference to the block, or pass the block around from within that method? Easy. If the last parameter in a method declaration is prefixed with &, the provided block will be assigned to the parameter. Take a look for yourself:

class BlockContainer
     def initialize
         @blocks = {}
     end

     def addBlock(name, &block)
         @blocks[name] = block
     end
end

container = BlockContainer.new
container.addBlock("a") { puts "a" }
container.addBlock("b") { puts "b" }

Going back to what I said about blocks taking the context they were defined in with them... it is true! I swear!

Ok, to illustrate the idea, look a this code:

def methodThatTakesBlock
     yield
end

a = 1
methodThatTakesBlock { puts a }

Your first reaction might be "that won't work, a will not be available from within methodThatTakesBlock". In fact, it will, simply because a is part of the context where the block is defined.

Default initializer for Hash entries

zenspider made an interesting observation in a comment to my conditional initialization post. He suggests to use the following construct instead:

def initialize
     @map = Hash.new { |hash, key| hash[key] = [] }
end

def addToList(key, val)
     @map[key] << val
end

Yes, this is definitely more readable. This goes to show, as I mentioned in my reply to his comment, that when switching between languages we need to *think* in the new language. Performing direct "syntax translation" results the kind of code I posted earlier. The ability to think in a new language comes with practice, just like it does when switching "speakable" languages such as Spanish and English.

In the code above, we're passing a block to the new method of Hash. This block is invoked by the hash object whenever an entry that doesn't exist is requested, and the item returned is the value returned by the block (in this case, the result of evaluating hash[key] = [], that is, []).

The only disadvantage here is that any time we try to get the value associated with a key, an entry is created in the hashmap. So it really depends on how the hash is going to be used in the larger context of the program.

Anyway, it's a cool feature that I'm adding to my bag of tricks.

Quack, quack

If it walks like a duck and quacks like a duck, then it must be a duck.

The type systems of Java and Ruby are very different. First and foremost, Java is more statically-typed than Ruby. Then there or other features that pull them appart, such as the fact that in Ruby everything (or almost) is an object, while in Java there are some things that don't qualify as such (primitive types, functions). But there's a more more fundamental, philosophical, difference; namely, the intention behind the type systems.

First of all, let me make a subtle but important distinction that will help understand what comes next. When I refer to the class of an object, I'm talking about the class it was created from. For instance, in Java, new String() creates an object of class String, just like String.new does in Ruby. The type of an object is the set of roles the object can play. Thus, the type of a class is not the same as the class, but typically includes it.

Java uses inheritance (loosely speaking) as the mechanism for defining the type of an object. An object "is a" X if it implements X. While Ruby also supports inheritance, establishing "is a" relationships is not its main purpose. In Ruby an object is of a certain type if it behaves as that type. Thus the saying "if it walks like a duck and quacks like a duck, then it must be a duck" or as it is commonly called, Duck typing. It must be noted, however, that while Ruby has classes, objects are not explicitly declared of to have a certain type.

So, what, concretely, defines what type an object is in Ruby? Well, that's the wrong question to ask. But if you'd still like to hear an answer, an object, conceptually, is of N! types (made of the subsets of all the combinations of methods exposed by the object), where N is the number of methods exposed by the the object,

In a sense, Ruby shifts the notion of type to the client (the client being the code that uses a given object). Thus, an object is (or not) of the type required by client if it implements the operations required by the client.

Ruby types are more granular as well, with the method being the atomic component of the type. In contrast, Java types are defined class by class, interface by interface.

Of course, this is all possible because Ruby is dynamically typed. That, together with the executable definitions concept I posted about before also allows type to vary by object. In Java, all objects created from the same class have the same type, and that type includes the class. In Ruby that may not be so. Look at this piece of code:

class SomeClass
    def method
        puts "within method"
    end
end

value = SomeClass.new
value.method # prints out "within method"

class << value
    private :method
end

value.method # fails with "NoMethodError: private method `method' called for ...

Definition-time, execution-time, anytime!

From "Programming Ruby", by David Thomas and Andrew Hunt:

The important thing to remember about Ruby is that there isn't a big difference between “compile time” and “runtime.” It's all the same. You can add code to a running process. You can redefine methods on the fly, change their scope from public to private, and so on. You can even alter basic types, such as Class and Object.

To this, I would like to add that Ruby blurs the distinction between definition-time and runtime. While definition-time may resemble, and frequently overlap, compile-time, there's a subtle difference.

In Java and other compiled languages, it is the compiler who interprets the class definition and produces an internal representation that the runtime portions will use. In contrast, in Ruby, it is the interpreter the one who does this. But more importantly, the class and method definitions are executed. As opposed to being parsed to extract the definitions before the code starts to execute.

In that sense, a class and method definition is part of the executable code, and the line between definition and actual code begins to fade.

It is this simple concept that enables Ruby to programs to create classes, add methods and attributes to classes and objects at runtime.

To illustrate these points, consider this piece of code:

c =
     class SomeClass
         attr :value
         self
     end

Before analyzing the code, remember that in Ruby every statement returns a value. In the case of a class definition, the value returned is the result of the last evaluated line.

When the Ruby interpreter sees this above code, it defines a class named SomeClass, creates an getter and setter for an attribute named value and assigns the result of evaluating self(i.e., a reference to SomeClass itself) to c.

And then we can also do the following:

obj = c.new
def obj.aMethod
     puts "Custom-made method"
end

This example reinforces the notion that definitions are executable code, and can intermingled arbitrarily with regular code(emphasized because in Ruby there's no such thing as regular code).

Executable definitions is a simple, but powerful concept. I'm still trying to understand all its implications and possibilities it opens.

Conditional initialization in Ruby

Exploring Ruby further, I came accross the || operator. It behaves like the boolean or operator in other languages (Java, C++) but with a catch: it returns the value of the last evaluated term. Since evaluation is lazy (from left to right), the last evaluated term is either the first non-false (I'll qualify this later on, bear with me) term or the last term in the expression.

More concretely, the expression d = a || b || c will assign the value of a to d if a is non-false, the value of b if a is false and b is non-false, or the value of c otherwise.

A non-false value in Ruby is anything that's neither nil, nor false. Thus, values such as true, 1, 0, "a", [] are all non-false.

The || operator can be combined with the = operator in a self-assignment statement: a ||= expression, which is a shorthand for a = a || expression.

A statement of that kind is essentially a conditional initializer: if a is nil (i.e., not initialized), initialize it with the value of expression. Otherwise, leave it untouched.

The conditional initialization idiom has at least one interesting usage. Before we get to it, it is important to note that in Ruby every statement returns a value, which means that any statement can be used in place of an expression. For example, one could conceivably do:

(if true then 8 end) + 3

Ok, on to the example now...

It is common to have to keep track of lists of items associated with certain key values. The obvious way to implement that is with a map of keys to lists of values. In Java, the code for adding an element to such structure would typically look like this:

void addToList(Object key, Object value)
{
    List list = (List) this.map.get(key);
    if (list == null) {
         list = new ArrayList();
         this.map.put(key, list);
     }

     list.add(value);
}

In Ruby, using the conditional initialization idiom, it could be written more compactly as:

def addToList(key, value)
     (@map[key] ||= []) << value
end

The @map[key] ||= [] piece performs the conditional initialization of the list associated with key and the (...) << value part adds the value to the existing or newly created list.

Ok, ok... you might be thinking, while the Java syntax is more verbose, is certainly more readable. That may be true, but in that case you could still write:

def addToList(key, value)
     list = (@map[key] ||= [])
     list << value
end

Which is still more compact than in Java but in my opinion does not sacrifice readability.

Pretty neat, eh?