Pugs

Declarative schema landed to Jifty trunk.

Shiny new Jifty::Param::Schema and Jifty::Manual::Actions landed; thanks to the brainstorming session with obra++, Object::Declare has grown to be a very flexible swiss army katamari, with support for negations, synonyms, callbacks, and copula-specific prefixes. Now I'll have to take tomorrow off to write about them for my OSCON talk. :-)

Gaal++ worked on a generic DrIFT-based transformer rule to get all Haskell structures used in the new AST into Perl 6 land, so we can ensure that the internal representations in v6 and Pugs/Haskell stays closely in sync, and even ship serialized objects across Haskell/Perl5 boundaries.

Following the recent S02 cleanups, rodi++ showed up on #perl6 and committed as test case for the then-unimplemented q<<< repeated bracket delimiters >>> syntax, which passed shortly afterwards.

TimToady++ pondered the semantic dilemma posed yesterday, and decided it's fine for f().g() always evaluate f() in scalar context, even though it has no .g method and the currently declared g subroutine demands a list context. However, g(f()) will always evaluate f() in list context, and only invokes the same .g method if f() returns the same value as it would in scalar context. Now we need to somehow write this down in spec language...

Jifty Defer/Declare/Moosified.

Today has been a really long day -- Jifty hacking by the day and Parrot hacking (IMCC lexer reentrancy) by the night. Happily, both proved to be fruitful.

The Jifty hacking part involves putting a declarative layer around Jifty::Action, so we can use the same Model-side Jifty::DBI::Schema syntax for Controller-side action parameters. It's now realized at the moose branch: the sample actions in t/ has been updated to reflect the new syntax:

The best part of this is that the declaration sugar is generally applicable: I've abstracted out all the build-your-own-DSL-kit sugar into a new CPAN module Object::Declare: It's not Jifty-dependent nor Moose-dependent, and there is no source filter or symbol table pollution involved.

However, changing from a custom-method interface:

sub arguments {
  return { foo => { label => _("bar"), default => time } };
}

to a static declaration interface that occurs at compile time:

param foo =>
  label is _("bar"),
  default is time;

means that we need to find a way to express runtime-calculated values. For example, the _("bar") call above for localization needs to use the current user's language handle, and the time call should be different each time the action is requested.

To solve that problem, I uploaded another new CPAN module, Scalar::Defer, so one can write:

my $t = defer { time };
warn $t; # 1153209784
sleep 10;
warn $t; # 1153209794

In addition to the call-by-name defer {...}, that module also provides the call-by-need lazy {...}, which works the same way as in Pugs -- i.e. it memoizes the first result value and reuse it on subsequent calls. By using deferred values, we get to aggressively memoize all statically-known parameter values, yet retain full flexibility for runtime-calculated fields. Yay for hybrid evaluation strategies!

State of v6-alpha: 499 tests pass.

As of v6-alpha 0.009 and Sub::Multi 0.002, we now pass 499 tests from our shared test suite with more than 11k tests. In other words, about 12% of tests are parsed, of which 5% are passing.

Still a long, long way to go (see the HACKING file for how to jump in and help), but I'm really excited -- this resembles iblech++'s multi-month PIL2JS journey on getting a few more tests passing each day, and this time there's even more CPAN folks helping out. So the fun is definitely optimizing rapidly... :-)

Moosifying Jifty.

I've allocated this weekend's time away from the Pugs repository, to concentrate on merging my day $job and Perl 6 -- namely, by converting Jifty from Class::Accessor into Moose, so I can use Class::MOP to implement the long-overdue support for API reflection on Action parameters, as well as use Moose's TypeConstraint for introspection on validators.

Not only this make Jifty coding much more declarative, it also makes it far easier to create custom views -- for example, a RESTish API with an endpoint for each Action, or translating validators as Web Forms 2.0 attributes.

As Web Applications 1.0 gradually matures, we can translate even more of this Object Model to run on the client side via JavaScript, and invent creative embeddings for browser with less native support for client-side continuations.  (See slipwave.info for some nice hackery toward that direction.)

Those techniques will form half of my upcoming OSCON talk; the other half involves put-closures-into-embedded-SQLite, a trick I've used in $job to create stored procedures that are aware of the current Jifty session/continuation context.

This way it's easier to pretend SQLite speaks perl, so the surface experience can begin to resemble DLINQ -- though the inner working is very much different.  More on that later... :-)

v6.pm now runs Test.pm!

Due to clkao++ and fglock++'s work, the CPAN version of v6.pm now passes all Pugs sanity tests, up and including the Perl 6 Test.pm.

That makes it the second implementation (after Pugs) and the fourth runtime (after Pugs's Haskell/JavaScript/PIR backends) that has access to the 11,000+ subtests in the test suite. Once the Parrot/Perl6 implementation support for subroutines, arrays, hashes and use statements, it will join as the third implementation that can run the test suite.

Development of v6.pm, the new AST, a concrete definition of multiple dispatch, etc. continues apace in #perl6; we are seeing more than 50 commits per day in the past few days. (gaal++ says it's just like the good old days of early Pugs development, but it's actually faster now. :-)) For example, v6.pm can now parse regex/token/rule declarations natively.

I'm very excited about this new cleanly-partitioned task space of Perl 6 implementation:

(Non-Perl 5 Specific)

• Descriptions of semantics in the test space;
• Analytical summaries and stories in the spec space;
• Algorithmic expression of effects and structures in the new AST space;

(Perl 5 Specific)

• Extensions in XS to Perl 5 to make Perl6 semantics possible;
• Implementation APIs to embed those semantics as Perl 5 modules;
• Idiomatic Perl 5 sugar that makes the APIs accessible;
• Translation from Perl 6 surface syntax into those new perl 5 idioms;

And the best thing is that, instead of a cyclic dependency as we had before, each layer are independent from the ones after it. It means that CPAN people can use Class::MOP and Data::Bind to improve their own frameworks; or to use the new idioms enabled by Moose.pm et all; all without necessarily switching to the Perl 6 surface syntax.

Moreover, it also makes v6.pm's output idiomatic -- we just need to incrementally build a new Perl 5 idiom, which, as Bjorn Freeman-Benson noted, is a Very Good Thing.

Steps toward P6-on-P5.

Some quick notes from today's hackathon about how to get perl5 into a first-class virtual machine for Perl 6, by gradually introducing CPAN modules that are useful by themselves, even without the surface Perl 6 syntax. Names in parens are #perl6 people who work on this:

1. Unified Perl 6 AST, covering all of P6 as specified (audreyt, gaal)
2. Emitter from #1 into Perl 5 (fglock, nothingmuch)
3. Perl 6 grammar that parses Perl 6 with MiniPerl6 inside rules to create ASTs (scw)
4. A bridge from Perl 6 calling conventions to Perl 5 subroutines (clkao)
5. A bridge between the Perl 6 object model to Perl 5 objects as well as native Perl 5 values (stevan)
6. Installation and co-existence of perl5:DBI and perl6:DBI (robkinyon)

More on those items later...

Multiversioning and qualified packages

Following is an implementation of "qualified packages", which means that each module (Test1 and Test2) can depend on different variant ("Foo1" and "Foo2") of certain package ("Foo").

It's posted here because I couldn't remember just how many people I agreed to send it to. :-) In any case... enjoy, and see you tomorrow!

use v6-pugs;

now invokes the new Perl5-based Perl6 compiler:

use v6-pugs;
"hello, world".say;

$ perl -Ilib hello_world.pl

It can even be called from the command line:

$ perl -Ilib lib/v6.pm -e ' "hello, world".say; '

It still fails many of the sanity tests, as well as nearly all of the other tests under t/.

Parrot and Perl5 Embedding Repaired.

Now with extra import sugar for Perl 5:

use perl5:Digest::MD5 <md5_hex>;

# 09189568a8ee4d0addf53d2f6e4847cd
'pugs'.md5_hex.say;

Also, through some evil contortions, embedded Parrot now won't crash due to invalid PGE expressions. Hopefully some day Parrot can be embedded with multiple non-interfering instances, or maybe raise a proper exception instead of forcibly calling Parrot_exit on a PIR syntax error... But for now, evil contortions would have to do.

Next in sight: PGE adverb support, repair the JavaScript backend, README, and preflight!

Compiling Rules and Modules.

After much socialtechnical discussions and hackings, the shiny Pugs::Compiler::Rule and Module::Compile are finally  upon us! fglock++ and ingy++ for tirelessly working with #perl6 people, almost non-stop, for the past 48 hours.

There is still a long way to go for Pugs::Compiler::Rule, but it's shaping up to be a promising start. The Perl5-based Perl 6 compiler in the pX/ tree is already moving forward after the release of the Rule engine -- in particular, ruoso++ is is refactoring .pl files into proper .pm modules in the lrep-compiler subtree, similar to how iterator_engine.pl was refactored out to Pugs::Compiler::Rule.

As for Module::Compile... for a general overview of what it is good for, and some of the socialtechnical issues around it, please see my post on perl5-porters and Juerd's excellent overview on perlmonks. I'm really excited about a saner drop-in replacement to source filters so we can cleanly extending the Perl 5 language in a composable fashion.

During the discussion of Module::Compile on #p5p, certain very crazy folks suggested a possible "goto *FH" form for Perl 5.10 to simplify the control transition between .pm/.pmc, or anything that needs to transfer control over to other files -- i.e. eval_exec. It is certainly a very perlish, very golfy, very wacky suggestion... Not sure whether it will make into the core or not.

On the Parrot front, today I continue to discover segfaults for Leo to fix, learning quite a bit about gdb and valgrind along the way. Happily, we discovered that all the Perl 6 semantics required by the object space bootstrapping code can be encoded into PIR; I've scribbed down some notes to be translated into a much improved Emit.PIR tree tomorrow.

There is some interest underway from gaal++ and mugwump++ to machine-translate Haskell structures into Perl 6, so we can export all the Pugs/Haskell data definitions (such as Emit.PIR), then encode the same logic into Perl 6, with its rather powerful structural pattern-match syntax for writing compilers and emitters.  That may prove to be very useful for the gradual transition toward full self-hosting.

That's it for today... See you tomorrow! :-)