Initial Project Plan – Learning Processing and ANTLR v3

My project for this semester is creating an ANTLR based Processing to JavaScript converter.

Basically, I’m using ANTLR v3 to create a tool that translates Processing code into JavaScript.

The Processing.js project is working on porting Processing to JavaScript. Currently, it utilizes a series of regular expressions in order to parse the code into JavaScript. However, as Dave puts it, this current method is brittle.

There is an ANTLR v2 based Java grammar that translates Processing to Java. When ANTLR v3 came out, it had JavaScript added as a new target language (a language it could translate to), enabling ANTLR to translate languages to JavaScript. The JavaScript target is still in beta state. It has a few bugs, but is passing many test suites. Short-term, the regular expression parser will work out. Long-term, a stand-alone converter based on ANTLR would be nicer. What I’ll be doing this semester is writing the grammar (the set of rules) needed to translate Processing to JavaScript, thus creating the converter.

If you’d like a peek, here is THE BUG Dave filed.

I have chosen this project because I want to do something for the Processing for the Web project. I am interested in where Processing.js is going. I find that I like the Processing language on a personal level (it programs graphics!), and it has found a little space in my heart. I am also interested in learning about grammars, dusting of my JavaScript skills, and I like the word ANTLR (Both Jeff and Mike saw that one coming). I have decided that if Processing wants an ANTLR based JavaScript grammar, then I want it to have one.

I am about to spend this semester figuring out what it will take to produce this grammar.

The Process:

My work can be tracked via this fantabulous project page on the CDOT wiki.
As things progress, comments may appear on lighthouse, where Dave filed THE BUG.
I will also be updating this blog as I proceed.

The Timeline:

0.1 Release – Feb 19th, 2010
0.2 Release – Mar 19th, 2010
0.3 Release – Apr 9th, 2010

All of these days are Fridays. 11:59pm is the latest I can submit each release before it’s counted as late. Check out my blog. When things are going well, expect earlier releases.

0.1 Release

For the first release I intend to have the ANTLR grammar able to translate variables from Processing into JavaScript. The reason this is so basic is that I have approximately 20 days between today and the 0.1 release to learn ANTLR, familiarize myself with ANTLRWorks (ANTLR’s grammar development environment), dust off my Java and JavaScript skills and write the first piece of this grammar. It is possible that I could finish early. If this is the case, I will start pulling tasks out of the 0.2 release and including them with 0.1.

Translating variables would make an excellent starting point for this project. Once I succeed in getting a piece of the ANTLR grammar working, the rest should come along with a little time and effort.

0.2 Release

(To be updated while I learn ANTLR during the beginning of the 0.1 release)

I believe I’ll be writing 2 grammars. One for a lexer and one for a parser. Here’s the big problem. Until I get more into ANTLR and writing grammars … this section is going to be a little blank.

I’m in the middle of the ANTLR Translator Generator FAQ From jguru. I’m reading ‘an overview of ANTLR and how to approach building things with it’. I’m not totally hopeless.

Any update to this section will be posted on the Project Page.

0.3 Release

This release will have that nice little ANTLR based JavaScript grammar for Processing.
PDE (Processing Development Environment), Bespin, and Eclipse should be able to “compile Processing for the web”. =)

Things I’ll Be Learning:

ANTLR and ANTLRWorks

• ANTLR may take a bit of time and the best place to practice it is probably by using it’s grammar development environment, ANTLRWorks. At least for the purpose of practicing examples from Terence Parr’s ANTLR reference books (more on those in a sec), and examples from the wiki. Unfortunately, ANTLR’s JavaScriptTarget isn’t integrated into ANTLRWorks yet.

Processing and the PDE

• I know nothing of Processing, but I like what I’ve seen from it. I’ll learn it so I can translate it to JavaScript. Excited.

Refreshing my Java and JavaScript

• I need the JavaScript because it’s my target language. I must know how it works. I need Java because Terence’s reference books, and a large majority of the examples, were written for ANTLR’s Java target. I need it to learn ANTLR.

I Plan to Learn ‘em with these Resources:

I’m going to be all over antlr.org (especially the FAQ and documentation) and Processing.org/learning/.

Besides a few class readings I need to catch up on,

this blog is going to be about learning Processing and ANTLR v3.

The best way to learn ANTLR is from Terence Parr himself. He has 2 books out:

1. The Definitive ANTLR Reference: Building Domain-Specific Languages
2. Language Implementation Patterns: Create Your Own Domain-Specific and General Programming Languages

The second book came out Dec 31, 2009, just in time! Apparently, the first book (noted as being an excellent resource) is easier to grasp once parts of the second book is read.

Supplement the books with examples from ANTLR’s wiki (for examples in target languages other than Java) and ANTLR’s two mailing lists: ANTLR-[interest] and ANTLR-[dev], and I should be good to go. I hope.

I’ve been looking high and low for ANTLR’s IRC channel. Does it exist? If anyone knows where the ANTLR developers are on IRC, I’d love to do some lurking.

Contacts:

None. Contacting someone now would only waste their time.

Since I am completely new to ANTLR and its dev environment, I need to familiarize myself with it before I can ask any feasible questions. My 0.1 release requires a lot of learning. Shortly before its release, I should have a much clearer picture of what I am dealing with, and I will be able to ask some good questions. Until then, I will be sitting down with ANTLR, learning in small steps.

First, I must RTFM.

Any update to this section will be posted to the Project Page.

Collaboration:

My code will need to be tested on PDE (Processing Development Environment), Bespin and Eclipse once I’ve got the grammar going.

The grammar needs to be tested to ensure it translates functions properly. Anyone who would like to try breaking it is welcome.

Contribution:

Since this project will require me to brush up on my Java and JavaScript, as well as learn some Processing, I would be up for testing or debugging in these languages. I’m also good for any tasks that may be deemed ‘boring’. I’m not easy to bore.

Talk to me.

Barriers and Risks:

Learning ANTLR3JavaScriptTarget, which is still in beta state.

• It has poor documentation, isn’t integrated with ANTLRWorks yet, the -debug option is unavailable, and I’m told to expect bugs and problems.
  Fix: the Java target API Documentation will suffice for now, the rest  I have to figure out as I go.

My learning style, which is very systematic.

• I will be hesitant to start on any code until I have thoroughly researched my topics (ANTLR, ANTLRWorks, Java, JavasScript, grammars). I like (and will be tempted) to read entire books/webpages on the subject before starting. I know I cannot approach the project in this way.
  Fix: I need to tweak the way I usually learn. I only need to learn enough code to solve the problem. Use indexes, and tables of contents. (This is the only risk I could control).

ANTLR3JavaScriptTarget isn’t integrated with ANTLRWorks.

• I’ll need ANTLRWorks to help me work my way through examples, etc. However, when it actually comes to my grammar, ANTLRWorks won’t be of much use.
  Fix: … hum. I guess ANTLRWorks will be temporarily reduced to being a regular text editor.

Wk 3 – On Processing

Yesterday’s class introduced me to the Processing language and the Processing.js project. I’ve seen this project around, but I didn’t look into it. I really like how Processing.js is making it possible to program graphics and render them in the browser. I also took an interest in ANTLR once Dave mentioned ‘grammar’. A part of INT522 (a voice-driven web app programming course at Seneca) studies VoiceXML grammars. I love when different courses start to touch on the same subject, so I’ve grown curious about ANTLR and would like to learn a little more. I joined the ANTLR-interest mailing list looking for this info.

I think I’d be interested in working on the ANTLR based JavaScript grammar for Processing that Dave was discussing in this post.

Wk 3 – On Blogging

Class was rather enlightening for me yesterday. It’s been over 2 weeks since my first blog post.

I find blogging difficult. I’m learning a lot of new things, and I’m trying to draft a concise and cohesive post while I’ve got a bunch of thoughts flying off in so many different directions. Trying to sit down and focus long enough to write them down is proving difficult for me.

Dave’s discussion on blogging helped a lot. Gusti mentions the difficulties of making ideas in larger blog posts flow, and Scott suggests breaking the post into smaller paragraphs. Breaking the post up is so much better than what I was trying to do – force myself to write a nice paragraph before I’d publish anything to the blog (ty, Scott!). That, and hearing someone else describe blogging as ‘painful’, probably got me to sit down and write this post. ‘Painful’ hit it right on the head.

While trying to motivate myself to blog (before the discussion) I started with Dave’s post Why Blother? and ended up at Matt Jadud’s the busy student’s guide to project blogging. Using the blog to capture thoughts that are still in-progress and blogging a brainstorm are things I’ll need to get used to. I’m too accustomed to keeping something hidden until it’s complete.

First Post! + Revolution OS

Hi, I’m Crystal!

I had my first Open Source Dev. (OSD600) class on Tuesday. Besides using FireFox and a brief stint with SUSE Linux and OpenOffice, I did not know much about Open Source – I thought I had the gist of it, but I now know that I did not. I related free software only to money, as in, I could download this, not pay, not get in trouble, and not feel guilty.

The first thing I did off the To Do list was watch Revolution OS. I came away with a few things:

1. The real meaning of ‘free’ – freedom. Freedom to fix an error. Improve an application. Study code. It did not only refer to money.
2. Open Source is also a philosophy, not just a methodology.
3. One guy wanted to change society for the better.

One day later, and all I remember is how passionate Richard Stallman is about the free software movement. Dave’s talk in class about the open source community was really inspiring, yet, made me suspicious. A community where everyone is working together, and are passionate about what they are doing. Everything seems too nice. It’s one of those things you want to believe but know it’s probably too good to be true. It’s the way communities should work, but all too often don’t. I am glad I watched Revolution OS, mostly because I got to see Stallman and many others talk. Putting faces to names does not usually happen on the web. It personifies the community for me, and I could see how passionate many of the speakers are. When Stallman says that he only wants to change society for the better, I can see and I believe that he genuinely means. It all helps me understand how the open source community may actually be what Dave describes it to be.

This is it for my first post. Thanks for reading!