MC97 Lab 2: Lexical Analysis (Spring 1999)

Due: February 24, 1999

For this lab, you are to do the lexical analysis assignment from pages 34-35 of the textbook.

First, copy the chap2 directory of files that Appel provides into the directory you are going to work in and look at them. You can access the chap2 directory off of this web page or copy it from ~max/www-docs/courses/S1999/MC97/tiger/chap2. The simplest way to do this would be using the command

cp -pr ~max/www-docs/courses/S1999/MC97/tiger/chap2 .

which will give you your very own chap2 directory tree, just like mine. Note that I have fixed a couple bugs in Appel's code; if you for some reason want to use the code from his web site instead of mine, you'll need to make those same fixes.

Next, compile the program as it stands. To do this, change directory to the chap2 directory, set your CLASSPATH environment variable so that java can find JLex and java_cup, and then run the make command:

cd chap2
setenv CLASSPATH ~max/JavaLib:.
make

To test this initial minimal version of the lexical analyzer, make a file (called commas.tig, for instance) containing nothing but commas, spaces, and newlines. Then give the command

java Parse.Main commas.tig

This should report the position in your file of each comma and of the end of file.

Here are some tips for extending this lexical analyzer to the full Tiger language:

Note that the way error messages are printed in Program 2.9 on page 31 doesn't correspond to how it is done in the provided code. Instead you should do something like
```
err("illegal character");
```
Of course, it would be preferable to be more specific.
Some token names are inscrutable to the uninitiated, so don't be afraid to ask. In particular, the relations are EQ for =, NEQ for <>, LT for <, LE for <=, GT for >, and GE for >=.
Remember, you need ample testing, including errors as well as correct input. The test cases provided by Appel, which I have in ~max/www-docs/courses/S1999/MC97/tiger/testcases, may not be adequate. Also, stop and test incrementally rather than waiting until the end. You should do easy cases early and add fancy ones later.
The control character corresponding to a character c, where c is an @ sign or an upper- or lower-case letter, has an integer character code equal to c's reduced to a remainder mod 32.
Make sure to call newline() or errorMsg.newline() when a newline character is seen, even if it is in a comment or string literal. Note that the newline procedure in Tiger.lex assumes the newline is at the position given by yychar. If this assumption is incorrect, then you need to call errorMsg.newline() directly.
In order to find out how to get backslashes and quotes into a JLex pattern, see the JLex documentation.
Testing tokens that have attributes will require changing Main.
Program 2.9 on page 31 is incorrect to show quotes around \n and \t.
What if a newline character is encountered while processing a string literal, and the newline is not prefixed with a backslash (i.e., is not in one of the sequences Appel describes as \f___f\); how should this be handled?
Note that the tok procedure assumes the entire token was matched by the current regular expression. (This is implicit in its use of yychar as the starting position.) If this assumption is wrong (because you have used the start state feature to match a token in multiple pieces), you will need to do something a little fancier than just calling Appel's tok procedure.
Appel's makefile is not perfect, and sometimes won't rebuild a file that it should. To get around this, you can do
```
make clean
make
```
This deletes all the output files and then reproduces them all from scratch, rather than trying to only make those that need it.

Your lab report should include documentation of design and testing as well as your code.

Instructor: Max Hailperin