Part 1: The Problem
1. Problem summary:
Code documentation. Writing up documentation for code is long and tedious. Javadocs provide a convenient way to imbed documentation in the source code and create documentation from the source code. So, we’d like to implement a baby version of Javadocs for the 164 language… we’ll call it 164doc.
A development scenario that will be less tedious:
Without 164doc: You just finished coding a massive project. Now you have to write up the documentation for your project. You can either write it up yourself or hire someone to write it for you. You majored in computer science, not English, so this could be painful.
With 164doc: While you’re coding your project, take a few moments to add a comment above all the methods you’d like documented. In this comment, you can describe the parameters, return values, authors, etc. You should probably be writing these comments while you’re coding anyways, just to keep track of what you’re doing. Then, once you’ve finished, run your code through the 164doc creator and it will output documentation for you in the form of HTML. No extra writing yay!
2. Why this problem is hard:
Documenting code can be tedious and very detail-oriented. It’s very easy to forget to write about a small detail. Hiring a documentation writer means you have to spend more money.
3. Why this problem is worth solving:
164docs will save programmers a lot of time because they can spend more time coding and less time writing up documentation. It also provides a clean, uniform way of writing comments to make your code more readable.
Part 2: Studying Javadocs
1. Simple code example
/**
* Returns an Image object that can then be painted on the screen.
* The url argument must specify an absolute {@link URL}. The name
* argument is a specifier that is relative to the url argument.
* <p>
* This method always returns immediately, whether or not the
* image exists. When this applet attempts to draw the image on
* the screen, the data will be loaded. The graphics primitives
* that draw the image will incrementally paint on the screen.
*
* @param url an absolute URL giving the base location of the image
* @param name the location of the image, relative to the url argument
* @return the image at the specified URL
* @see Image
*/
public Image getImage(URL url, String name) {
try {
return getImage(new URL(url, name));
} catch (MalformedURLException e) {
return null;
}
}
The first line contains the begin-comment delimiter, /**. The first sentence in the comments is a short summary of the method, which is placed in the method summary table in the resulting HTML. The inline tag {@link URL} is converted to an HTML hyperlink pointing to the documentation for the URL class. Multiple paragraphs in the description should be separated with a
tag. A blank line separates the description from the list of tags. The first line that begins with an “@” ends the description. An @param tag is required by convention for every parameter, even when the description is obvious. The @param tag is followed by the name of the parameter, followed by a description of the parameter. The @return tag is required for every method that returns something other than void, even if it is redundant with the method description. The @return tag is followed by a description of the return value, including special cases whenever possible. The @see tag is followed by a reference. In this case, it is a reference to the java Image class. Many other tags are also supported. The last line contains the end-comment delimiter */. Javadoc takes the doc comment for each class, constructor, field, interface, and method and generates the API for each in HTML.
2. Javadoc Implementation
Javadoc requires and relies on the java compiler. It calls part of javac to compile the code, keeping the declarations and doc comments but discarding the implementation. It builds a parse tree and then generates the HTML from that. Relying on the compiler ensures that the HTML output corresponds exactly with the actual implementation. For example, the compiler creates default versions of constructors that are not present in the source code. If Javadoc did not use the java compiler, it would be difficult to handle these special cases.
for each class, constructor, field, interface, or method:
parse description looking for inlined tags
parse each tag
generate html
Part 3: Features
1. The domain:
The “programs” we intend to write are descriptions of methods coded in the 164 language. For example, we should be able to specify a description (written in plaintext and HTML), authors and versions for classes, and parameters and return values for functions. Our parser will then parse these fields and generate a HTML document.
2. The programming model:
Our programming model will be declarative programming for parsing (grammars and syntax-directed translation). We wrote the grammar to parse the 164doc tags into an abstract syntax tree, and then translated the ast into HTML using python.
3. Example programs!
Sample Program #1: Documentation for a factorial function
/**
* Returns the result of computing factorial on the argument.
* <p>
* This function is borrowed from test-advanced.164. It tests
* to make sure the while loop works.
*
* @param d the integer to compute factorial on
* @return the result of d!
*/
def fact2(d) {
def r = 1
while (1 < d) {
r = r * d
d = d - 1
}
r
}
HTML Output #1:
fact2
This function is borrowed from test- advanced.164. It tests to make sure thewhile loop works. |
Sample Program #2: Documentation for a fibonacci function
/**
* Returns the nth Fibonacci number.
* <p>
* This function is borrowed from test-fibonacci.164.
*
* @param n the index in the Fibonacci sequence
* @return the nth number in the Fibonacci sequence
*/
def fib(n) {
if (n < 2) {
n
} else {
fib(n - 1) + fib(n - 2)
}
}
/**
* Prints the first n numbers of the Fibonacci sequence.
* <p>
* This function is borrowed from test-fibonacci.164.
*
* @param n how many numbers in the Fibonacci sequence to print
* @see fib
*/
def printfib(n) {
def i = 0
while (i < n) {
print fib(i)
i = i + 1
}
}
printfib(input())
HTML Output #2:
fib
This function is borrowed from test- fibonacci.164. |
printfib
This function is borrowed from test- fibonacci.164. |
Sample Program #3: Documentation for a Shape Object hierarchy and Shape-list generation
/**
* A Shape prototype. Represents an abstract polygon.
* <p>
* Shape is the parent class of Circle and Rectangle.
*
* @author Joel
* @version 1.0, 12/06/09
* @method draw
* @see Circle
* @see Rectangle
*/
def Shape = {}
Shape.__index = Shape
Shape.draw = lambda(self) { print "I'm a shape." }
/**
* A Circle prototype. Represents a polygon with no vertices.
* <p>
* Inherits from the Shape prototype.
*
* @author Joel
* @version 1.0, 12/06/09
* @method draw
* @see Shape
*/
def Circle = {}
setmetatable(Circle, Shape)
Circle.__index = Circle
Circle.draw = lambda(self) { print "Let's draw a circle!" }
/**
* A Rectangle prototype. Represents a polygon with four vertices
* and four right angles.
* <p>
* Inherits from the Shape prototype, and is the parent class
* of Square.
*
* @author Joel
* @version 1.0, 12/06/09
* @method draw
* @see Shape
*/
def Rectangle = {}
setmetatable(Rectangle, Shape)
Rectangle.__index = Rectangle
Rectangle.draw = lambda(self) { print "I have four sides." }
/**
* A Square prototype. Represents a polygon with four vertices,
* four right angles, and four sides of equal length.
* <p>
* Inherits from the Rectangle prototype.
*
* @author Joel
* @version 1.0, 12/06/09
* @method draw
* @see Rectangle
*/
def Square = {}
setmetatable(Square, Rectangle)
Square.__index = Square
Square.draw = lambda(self) { print "I'm a special Rectangle that's, well, square." }
/**
* Returns an instance of the argument prototype.
*
* @param parent the prototype to create an instance of
* @return the instance
*/
def makeObject(parent) {
def o = {}
setmetatable(o, parent)
o
}
/**
* Returns a list of user-specified shapes.
* <p>
* Typing Shape adds a Shape instance to the list.
* <p>
* Typing Circle adds a Circle instance to the list.
* <p>
* Typing Rectangle adds a Rectangle instance to the list.
* <p>
* Typing Square adds a Square instance to the list.
* <p>
* Typing 0 returns the list.
* <p>
* Typing anything besides 0 will allow the user to continue adding
* shapes to the list.
*
* @return a list of Shape or Shape-subclass instances
*/
def makeList() {
print "Let's make a list."
def list = {}
def i = 0
def continue = 1
while (continue) {
print "Do you want to add another element to the list? Enter 0 for no; anything else is considered yes."
def result = input()
if (result == 0) {
continue = 0
} else {
print "Please enter the next element to add ('Shape', 'Circle', 'Rectangle', or 'Square')."
def elem = input()
if (elem == "Shape") {
list[i] = makeObject(Shape)
i = i + 1
}
if (elem == "Circle") {
list[i] = makeObject(Circle)
i = i + 1
}
if (elem == "Rectangle") {
list[i] = makeObject(Rectangle)
i = i + 1
}
if (elem == "Square") {
list[i] = makeObject(Square)
i = i + 1
}
}
}
list
}
# No 164docs needed here since we're not defining classes or functions
def objects = makeList()
print "Let's do some drawing."
for o in objects {
o:draw()
}
HTML Output #3:
ShapeShape is the parent class of Circle and Rectangle. |
CircleInherits from the Shape prototype.
|
RectangleInherits from the Shape prototype, and is the parent class of Square.
|
SquareInherits from the Rectangle prototype.
|
makeObject
|
makeList
Typing Shape adds a Shape instance to the list. Typing Circle adds a Circle instance to the list. Typing Rectangle adds a Rectangle instance to the list. Typing Square adds a Square instance to the list. Typing0 returns the list. Typing anything besides0 will allow the user to continue adding shapes to the list. |
Part 4: Implementation decisions
General idea: We wrote a 164doc grammar that extends the 164-language grammar that also accepts the doc comments. In the grammar, we use syntax-directed translation to parse an input source code file (paying attention to only the 164doc comments and the function declarations) into an abstract syntax tree. Then, we translate the AST into an HTML document using an interpreter written in python. The interpreter creates one file per doc comment and links them together (if the comment specified relationships).
1. Frontend
A 164doc comment must go above a function definition or class definition. For example, in the second sample program, the comment goes above “def makeObject(parent)” (function definition) and “def Square = {}” (object definition).
The comment must begin with “/**” and end with “*/”, each on its own line. Each line of the comment must begin with “*”.
The first part of the comment is the description of the object/function. To differentiate between paragraphs, use the HTML paragraph tag on its own line.
Next comes a blank line, i.e. just “*”.
Lastly the user can specify tags. There are different tags depending on if the user is describing an object or function.
The object tags:
@author <authorname>
@version <version#> <date>
@method <methodname>
@see <another method or class to link to>
There can be multiple author, method, and see tags. For example, if you need to declare more than one author, use this syntax:
@author Adam
@author Jillian
… as opposed to putting the names on one line. The tags may be in any order, but the order they are listed above is preferred because that is how it will be displayed in the HTML document.
The function tags:
@author <authorname>
@version <version#> <date>
@param <argument name> <argument description>
@return <description of the return value>
@see <another method or class to link to>
There can be multiple author, param, and see tags. The tags may be in any order, but the order they are listed above is preferred because that is how it will be displayed in the HTML document.
2. The core language
There is no sugar on top of a simpler-to-implement language.
3. Internal representation
We use an AST as an intermediate structure between the base language and the HTML-generator. As an example, the AST for the first sample program looks like this:
[('doclist',
[('description', 'Returns the result of computing factorial on the argument. <p>This function is borrowed from test- advanced.164. It tests to make sure thewhile loop works.'),
('tags',
[('param', 'd', 'the integer to compute factorial on'), ('return', 'the result of d!')])]),
('def-function', 'fact2')]
The root list contains tuples of the form (‘doclist’, …) and (‘def-function’, [function-name]) or (‘def-object’, [object-name]). The second element of the ‘doclist’ tuple is a list containing two tuples; (‘description’, [description-string]), (‘tags’, [tag-list]). The tag-list contains multiple tuples of the form ([tag-name], [id (if necessary)], [description (if necessary)]).
4. Interpreter/compiler
For each document-definition pair in the AST, we generate an HTML document by stepping through the AST and building the HTML incrementally. We create a folder to hold all these files, and use a consistent naming convention so that the “see” tags can be used to create links between the files.
5. Debugging
To debug, we would modify the grammar little by little and tested it on very small programs. We would print out the AST and make sure it looked reasonable. Adam’s AST-to-HTML interpreter worked on the first try, so we didn’t need to debug that.
:)
