November 26, 2005

ANTLR-based Ruby grammar

It looks like other people are showing interest in replacing the YACC-based Ruby parser by one based on ANTLR. This hits close to a project I've been working on in my spare time.

When I looked at JRuby's source code a few weeks ago, I thought I'd start trying to understand how the parser works. I had a few ideas I wanted to experiment with, but...

The YACC-based parser is too hard to fathom. There are too many scattered pieces with no clear dependencies. Making changes without breaking things should be even harder.

So I decided to start working on ANTLR-based grammar to replace the current implementation -- pretty much what MenTaLguY is trying to do. I'm still in experimentation mode and trying to figure out how I'll deal with some of the features of Ruby that are not usually found in other languages.

One of the hardest aspects, I think, is how to parse %Q-delimited strings. Here's how they work (from the Pickaxe book):


Following the type character is a delimiter, which can be any nonalphabetic or nonmultibyte
character. If the delimiter is one of the characters (, [, {, or <, the literal
consists of the characters up to the matching closing delimiter, taking account of nested
delimiter pairs. For all other delimiters, the literal comprises the characters up to the
next occurrence of the delimiter character.


Additionally, %Q strings can contain #{} constructs, whose contents should be parsed as Ruby expressions. These can probably be handled in a manner similar to how this grammar for the E language implements what they call quasi-literals.

I haven't yet decided on how to deal with delimiters but I have the feeling it may require writing a custom lexer for string contexts.

Labels:

14 Comments:

Blogger yawl said...

This comment has been removed by a blog administrator.

May 11, 2006 2:33 PM  
Blogger yawl said...

The expression substitution requires the parser to set state for the lexer. In rubyfront, I tried hard to make the lexer independent of the parser, but it is not possible in this cases:

http://seclib.blogspot.com/2006/04/rubyfront-020-released.html

May 11, 2006 2:34 PM  
Blogger nasha said...

Why was there no follow on bankruptcy then? The bailout of AIG FP went to (wow power leveling) hedge funds that bound credit swaps on Lehman failing or others betting on rating (wow power leveling) declines. AIG has drained over 100 billion from the government. Which had to go to (wow power leveling) those who bet on failures and downgrades. Many of whom (power leveling)were hedge funds. I-banks that had offsetting swaps needed the money from the AIG bailout or they would have been caught. Its an (wow powerleveling) insiders game and it takes just a little bit too much time for most people to think (wow gold) through where the AIG 100 billion bailout money went to, hedge funds and players, many of whom hire from the top ranks of DOJ, Fed, Treasury, etc. ZHANG XIAO CHEN

April 19, 2009 10:02 PM  
Blogger Kayla said...

Present-day society, the rapid development of science and technology makes cell phones wholesale is no longer a simple phone communications equipment only. Wholesale cell phones have many features, entertainment, scalability, and so have a good expression of the discount cell phones will be to maximize the effectiveness of multimedia technology.

September 24, 2009 9:23 PM  
Blogger Kayla said...

Cell phones from simple communication became a full-featured entertainment terminal . Can be said that cell phone china new milestone in the industry. This fully demonstrates that, cheap cell phones entertainment has become the most sought after consumer applications and has become the focus of the mobil phones market.

September 24, 2009 9:24 PM  
Blogger Kayla said...

www.crazypurchase.com Web site for the majority of the china wholesalers, and buy products wholesale to provide information and buy wholesale from china information exchange platform dedicated to becoming China Wholesale.

September 24, 2009 9:25 PM  
Blogger Adi said...

Oes Tsetnoc | Mengembalikan Jati Diri Bangsa | Kenali dan Kunjungi Objek Wisata di Pandeglang | Oes tsetnoc | Online Marketing | Electronics Gadgets | etips solution | Travel Guide

October 27, 2009 12:40 AM  
Blogger The Jack said...

Thanks ever so much, very useful article. If you do not mind, please visit my article related to pandeglang district in Banten, Indonesia at Kenali dan Kunjungi Objek Wisata di Pandeglang or Kenali dan Kunjungi Objek Wisata di Pandeglang second and also Kenali dan Kunjungi Objek Wisata di Pandeglang Objek Wisata Air Terjun Curug Gendang or related to a leadership at Mengembalikan Jati Diri Bangsa and Oes Tsetnoc or Oes Tsetnoc the second and our hard work at Kerja Keras Adalah Energi Kita that's right Kerja Keras Adalah Energi Kita, and Kenali dan Kunjungi Objek Wisata di Pandeglang Memasuki Babak Akhir also Objek Wisata Taman Wisata Alam Carita, Kenali dan Kunjungi Objek Wisata di Pandeglang, or Kenali dan Kunjungi Objek Wisata di Pandeglang, also Kenali dan Kunjungi Objek Wisata di Pandeglang, or Kenali dan Kunjungi Objek Wisata di Pandeglang, also Kenali dan Kunjungi Objek Wisata di Pandeglang, or Kenali dan Kunjungi Objek Wisata di Pandeglang, also Kenali dan Kunjungi Objek Wisata di Pandeglang, or Kenali dan Kunjungi Objek Wisata di Pandeglang, very smart thank you!

October 28, 2009 11:01 PM  
Blogger Daniela said...

Find your great Travel News and sing the songs at Free Song Lyric or you can watch the drama at Korea Drama Online one of great korea drama is A Love to Kill if you go to travel to Indonesia learn Learn Indonesia Language first! And find your home cari rumah or make a blog Belajar membuat Blog find your home again rumah dijual and again at jual rumah then if you want buy a new laptop see the Laptop Price List or you can buy a New Blackberry and then take care your Health & Jewerly.

November 14, 2009 7:48 PM  
Blogger bitheads said...

I don’t know If I said it already, but this so good stuff keep up the good work. I read a lot of blogs on a daily basis and for the most part Belajar seo blogspot just wanted to make a quick comment to say I’m glad I found your blog. Thanks.

Best regards
Seo Motivation Oes tsetnoc faq

November 25, 2009 8:41 AM  
Blogger nike said...

Clothing has always been a thing that has been given a great importance by human beings. It displays the attitude that the people exhibit.
ed hardy clothes
ed hardy shirts
Lots of brands have been existent in the clothing industry and a famous one among them is the Ed hardy clothing brand.
ed hardy jackets
ed hardy hoodies
The brand got its name from the famous American tattoo artist Ed Hardy. He was a very famous tattoo artist and has published many books on tattooing techniques.
ed hardy boots
ed hardy polo shirts
But his tattooing turned into a brand by the efforts of a company called Christian Audiger. This company was a very famous and very powerful company in the field of clothing.
ed hardy shoes
ed hardy jeans
They felt that it would be appropriate to create a brand called Ed hardy and use Hardy's art as the main selling point for the brand. This venture had turned out to be a very successful one and Ed hardy clothing is one of the most famous brands in the clothing industry.
ed hardy outerwear
ed hardy long sleeve shirts
The brand became very famous because it was worn by many famous celebrities like Madonna, Britney spears and also Sylvester Stallone.
ed hardy bags
ed hardy winter boots
The brand has clothes for men, women, and kids. They have also diversified their business by having a lot of accessories to support their clothing business.
ed hardy love kills slowly shirts
ed hardy love kills slowly shoes
ed hardy love kills slowly boots
for men they have a variety of products such as active wear, denim, outwear, swim trucks, sweaters, t-shirts, tops etc.
ed hardy trousers
ed hardy mens
the accessories include things such as belts, caps, scarves, shoes, socks, jeweler, sunglasses, ties and even wallets.
The art works is a unique combination of American and Japanese cultures. The unique selling proposition of the brand is the way in which the company has used the art works of Ed hardy.
ed hardy womens
ed hardy t shirts
ed hardy sunglasses

December 09, 2009 5:24 PM  
Blogger nike said...

Nice work and thanks!
Running
Adidas currently manufactures several running shoesNike shoes, including the adiStar Control 5, the adiStar Ride
Cheap nike shoes
Discount nike shoes
the Supernova Sequence and the Supernova Cushion 7, among others.
Nike shox r4
nike shox torch
nike shox shoes
Adidas also uses kangaroo leather to make their more expensive shoes.
Association football
One of the main focuses of Adidas is football kit and associated equipment.
puma cat
cheap sport shoes
Adidas also provides apparel and equipment for all teams in Major League Soccer. Adidas remain a major company in the supply of team kits for international football teams.
cheap nike shox
cheap nike max
Adidas also makes referee kits that are used in international competition and by many countries and leagues in the world. In the United States, referees wear the Adidas kits in MLS matches even though the primary referee supplier is Official Sports.
nike tn dollar
nike running shoes
The company has been an innovator in the area of footwear for the sport with notable examples including development of the Copa Mondial moulded boot on firm dry pitches for forty years.
nike air max tn
puma shoes
Adidas became renowned for advancing the "Predator" boot design.This design featured a ribbed rubber structure for the upper leather of the shoe, used to accent the movement of the ball.
discount puma shoes
puma mens shoes
The Predator also features the Craig Johnston invented "Traxion" sole. As the development and popularity of Football continued Adidas played a leading role in shaping the style of the play itself.
puma running shoes
puma shoes
FIFA, the sports governing body, commissioned specially designed footballs for use in its own World Cup tournaments to favour more attacking play.
ghd hair straighteners mk4
hair straightners
ghd iv styler hair straightener
ghd hair straightners
cheap ghd hair straighteners

December 09, 2009 5:25 PM  
Blogger ally said...

Apart from these military fashion tops and ripped and torn jeans are also hip and happening this season.2010 fashion trends for men would be dominated this season by denim. By denim we don't need to just stick to jeans but also sport denim jackets and casual pants as well. wholesale LV handbags
monclerdiscount moncler jacketsmoncler coatsmoncler vestmoncler outletmoncler t-shirtmonclermoncler jacketsnew moncler coats
moncler vestmoncler outletmoncler polo t-shirtCoach handbags outletCoach TotesCheap Coach handbag 2010Discount Coach hand bagAuthentic Coach handbagNewest Coach handbags outletLouis Vuitton TotesLouis Vuitton handbagsLV handbags 2010Discount LV handbagsCheap Louis Vuitton Outletnewest Louis Vuitton handbagscheap rain weardiscount rainweardog rain jacketscolorful rain bootsrainboots outletCheap Ture Religion Jeans outletDiesel JeansLevis JeansWholesale Ed Hardy JeansDiscount Dior Jeans outletJerseys and sneakers are never out of fashion. It is very urban and very hip hop.ust as leather is in trend for women it is also true for men. Leather jackets are a must in the wardrobe this season.

April 19, 2010 12:46 AM  
Blogger ally said...

It is very stylish and sleek. In the casual attire category tracksuits and tank tops rule for men. The most popular t-shirts this season would be the ones with funny sayings on them.cheap abercrombie fitch clothingdiscount abercrombie fitch T-shirtsdiscount abercrombie and fitch hoodiesabercrombie fitch outletwholesale abercrombie fitched hardy wholesale
cheap ed hardy wholesalediscount ed hardy wholesale
wholesale ed hardyed hardy outletdiscount abercrombie and fitch outletdiscount abercrombie outletdiscount abercrombie clothingdiscount abercrombie jacketdiscount abercrombie shirtdiscount abercrombie and fitch outletdiscount bercrombie and fitch clothesdiscount abercrombie and fitch hoodiediscount abercrombie and fitch shirtsdiscount abercrombie fitch jacketBesides, the urban trend, vintage clothing will also rule this season. This includes tight jeans and leather jackets and embroidered clothing.This season it seems like we are going to have a handful to choose from because the trend is a nice mix match of clothes from different seasons. So go ahead flaunt your style your own way.

April 19, 2010 12:47 AM  

Post a Comment

Links to this post:

Create a Link

<< Home