The entry I just wrote in the past 2.5 hours is lost again -- this time due to a disconnected wireless network -- so here is some very brief recaps of what we've accomplished:
Quasiquoting and Macros
Previously, Pugs only supports macros that return strings (aka source filters), because the macro-returning-AST form was not specified. We populated the Macro section of the Subroutine Spec (S06), and bsb implemented them in Pugs (note that the :COMPILING flag makes free variables in the quasiquote bind to the macro user's site):
macro hi () { q:code(:COMPILING) { "hello $s" } }
my $s="world";
is(hi(),"hello world","macros can bind in caller's lexical env");
$s="paradise";
is(hi(),"hello paradise","macros but it's a binding only");
We can even use a closure's body as a macro:
macro hey () { { "hello $^s" }.body }
my $s= "paradise";
is(hey(),"hello paradise","macros but it's a binding only");
This paves the way of desugaring require, use and other special form into regular macros. Also I came up with a pretty cute way of interpolating an expression inside a quasiquote ("unquoting") -- just repeat the quote delimiters three times:
return q:code { say $a + {{{ $ast }}} }
return q:code [ say $a + [[[ $ast ]]] ]
return q:code < say $a + <<< $ast >>> >
return q:code ( say $a + ((( $ast )))
where the $ast is a string ("code snippet"), or another AST object constructed by quasiquotes. Also, because q:code is just sugar for q:code<perl6> (which is just sugar for q:code(:lang<perl6>)), it leaves room for q:code<perl5> or even q:code<TT2>...
Core Speedups
Test.pm is now precompiled to Test.pm.yml at the beginning of "make smoke" (and installed along with "make install"), thanks to gaal, lumi and bsb's work. On Larry's computer, the smoke cycle now takes 79 minutes, instead of ~100 minutes as before.
Nothingmuch has proposed a per-user cache system that will allow us to reuse parse trees for individual test files as well, and gaal thinks it can be implemented in Perl 6 itself, but it's not currently implemented.
The compile time is also much reduced, by splitting the two bottlenecks (Pugs.AST.Internals and Pugs.Parser) into further submodules, and move all DrIFTed instances into separate files. To automate this effort, anatoly, gaal and lumi worked on a call graph visualizer and refactoring splitter for Haskell programs -- implemented in Perl 5.
Result Objects from Matches
Matches can now return arbitrary object as its "result" (think $& in Perl 5), instead of just a simple string, as specified in v11 of the Rules spec (S05). These are accessed by dereferencing the match object as a closure as $match() -- the default match object $/ has a result object called $(). This allows Parsec-style rules:
my $moose = m:{
<antler> <body>
{ return Moose.new( body => $<body>().attach(:$<antler>) ) }
}.();
With rule interpolation (<{...}>) and lookaround (<(...)>), this allows a rule to match some subrules, and conditionally invoke more subrules based on the result objects of the earlier ones. This is critical for writing a Perl 6 parser in rules, as it allows a
rule to construct AST objects, using result objects from its submatches
as material. It allows us to port the existing Parsec subparsers in a
direct style, rather than using separate tree-transformation rules that
operates on a huge Match object.
Perl 6 on Perl 5
The pX subproject, initiated by putter and contributed by tewk and fglock, is an attempt in implementing a self-hosting Perl 6 implementation using Perl 5 as a virtual machine. In addition to tewk's OpTable port, tonight fglock committed a working Perl 6 Rules Parser, which uses itself to parse the rules syntax.
The engine is interesting as it can be extended at runtime, and allows dynamic interpolation of rules, as well as calling back to Perl 5 code -- something that's quite difficult to do with Pugs's native PCRE bindings.
GetOpt
Ran and migo started porting Getopt::Std to Perl 6. They then worked with nothingmuch to parse for even more style of options -- long, mixed, stacked and whatnot -- and generalized into Getopt::Process, which also serves as a nice showcase of the state of the art of Perl 6 syntax.
The eventual goal is to massage the command line arguments into argument lists, so you can use subroutine signature syntax as option specification, including plurality, types, validation, and it'd even work for positional parameters. Stay tuned!
Syntax Generalizations
Lots of corners in Perl 6 syntax that made Pugs's Parser slow and/or difficult to implement have been normalized in the Synopses. For example, braces at the end of a line can now terminate statements without using semicolons:
try { ... }
try { ... }
It also makes control structures and macro declarators very easy to parse, because this is now invalid:
sub f {...} sub g {...}
There needs to be a semicolon between the two named-sub expressions, though the end-of-line semicolon is still optional.
Perhaps more importantly, thanks to Robrt's help, all these Synopses changes now gets posted to the perl6-language mailing list, for all to review and discuss. Hopefully this will alleviate the Perl 6 is changing under us and we can't see where it's going problem, and encourage more constructive feedback loop. Have fun! :-)
How useful is the Perl6 Rules Parser in a strictly Perl5 environment? Can I use it in place of Damian's merely-proof-of-concept "Perl6::Rules" module?
Posted by: Randal L. Schwartz | 2006.02.25 at 12:08 PM
merlyn: I'd like to use it in production as well in the near future -- it's far more robust, as it doesn't rely on Perl 5's regex engine for backtracking.
When fglock gets back to #perl6 next week, I'll see if we can package it up as Perl6::Rules::Iterative or something like that, and put it on CPAN.
Posted by: Audrey T | 2006.02.25 at 06:45 PM