ANTLR

I was looking around for better ways to do my lexer. Writing a lexer by hand is/was educational and fun. But it's also a royal pain, which is the biggest reason that you have products like ANTLR in the first place.

So I pulled it down, created a grammar, and bamm I had a nice little scanner in place. The problem? The scanner itself is now 4 times the size of the rest of the application.

Is this really an issue? It depends. I was rather excited by the fact that I had created something so small and I know exactly what it's doing. The ANTLR scanner is nice and a better design. But there's a lot of magic going on there that leaves me uncomfortable.

For right now, I'm, sticking to my scanner and working on enhancing it.

An extensive example of code

function ? (predicate result1 result2) {
 if predicate {
  ! result1
 } {
  ! result2
 }
}

function <= (value1 value2) {
 if [< value1 value2] { true } {
  = value1 value2
 }
}

function incr(x) {
 + x 1
}

function decr(x) {
 - x 1
}

/*
* inclusive range
*/
function .. (begin end) {
 set func [? [<= begin end] incr decr ]
 if [= begin end] {
  append () begin
 } {
  append [.. [func begin] end] begin
 }
}

/**
 *  iterates through a list and executes 
 *  the passed in block
 */

function foreach(list block) {
    if [not [empty? list]] {
        set item [head list]
        eval block
        foreach [rest list] block
    }
}

All of the above code is needed to prepare for the following

foreach [ .. 1 12] {
    println "hi! number " item
}

The reasons why things are the way they are

One of the best things about creating a programming language is that sudden realization on why people do the things they do.

One of the most common elements you see in a language is the "end of statement" the semi-colon ';'

In posl, there's no need for one, it's one line is one statement.

That sounds alright at first (if slightly restrictive, in the form of expression it's enlarged when you see that you can include a multi-line block of code within that single line.

if [<= 2 3] {
    println "low!"
    push () 4
} {
    println "high!"
    push () 8
}

In this contrived example, the parser sees a single line which equates to

command predicate multi-line multi-line

The problem comes when you typo

if [<= 2 3] {
    println "low!"
    push () 4
 {
    println "high!"
    push () 8
}

This bad code, we forgot a right brace. However the parser isn't going to complain. It's going to assume that the brace will eventually come at some point. Currently I have an interactive parser (going to need to change that in some sort of switch mode) the interactive parser, when it runs out of code to parse, just assumes that it's going to show up at some point.

Now if we included a semi-colon

if [<= 2 3] {
    println "low!"
    push () 4
 {
    println "high!"
    push () 8
};

The parser at this point damn well knows that there's something wrong and would throw a flag.

It's not enough to change my vision but it does provide insight into why it was done in the first place.

Changes in the grammar

My original vision for posl is changing. I tried hard to look at the various constructs that compose language and provide a unique charachter set that would always mean the same thing.

That would provide us with a function definition like so

function foo|bar|{
    doSomething bar
}

The 'pipe' was used to delimit a list of atomic values that would be used as variable assignments in the context of the function.

But what was it? a list? if it was a list could I use it in other places to define a list?

Then I had braces '(' and ')' this originally defined a predicate, a boolean result. so if you saw

set x (foo bar)

you knew the result of foo bar bar would(or should) be equivalent to true.

I like this, and it makes sense at a high level. Yet underneath, it's all a list. It's conceptually the same thing, over and over again, represented in different ways.

I then ran into an issue of "lack of characters" One of the defining problems when designing a language is the sheer lack of representational characters we have on a keyboard. We've already over ridden and multi-purposed all the special characters already so you have to be cautious over what you are attempting to do.

I finally said "screw it" and removed the '|' as special characters and reverted to a list. So that the demo function in question now becomes

function foo(bar){
 doSomething
}

which, of course, makes it look closer to other languages and that, in itself, provides some value.

Problems with Posl

It's interesting. S0 far as I can determine there really is two core types of structures in programming. The sequence(list, array, stack, etc..) and the Mapping (i.e. x is equivalent to y)

Everything else flows from this.

Take object, all an object is is a map. In some instances the mapped value is accessible external from the cope of the map, sometimes it's only internal. Sometimes the mapped value is a core datatype, other times it represents code.

It's how to represent this that I find problems with.