Pugs

The night before the Intel talk.

Landed safely to Portland last night.  Predictably, I was hit hard by jet lag today, although I did manage to finish the slides for DLS, which will get a pre-run at Intel tomorrow:

• Pugs: An implementation of Perl 6 (PDF).
• The source slides in Keynote.app format.
• A Flash version is also available.

As usual, feedback are most welcome!

Meanwhile in Pugsland:

• seano++ implemented non-strict-vars for one liners: "pugs -e '$x = 1'" is no longer an error.  An impressive first commit for a new committer! :-)
• raptor++, another new committer to Pugs, asked on #perl6 if there's a way to find out which parts of specs are not yet covered by tests.  Thanks to smartlinks, we now have a standard answer for that: spec.pugscode.org.  Shortly afterwards, raptor++ followed up with a test for "my $x = 1 if 1".
• On the airplane, I implemented this recent  ruling from TimToady++: Closing curly of hash composers may not end a statement by itself:
  

  my $c = sub { say 123 }  # this is valid -- semicolon is optional
  my $f = sub foo { say 456 } # valid -- named subs are expressions
  my $h = {
      a => { x => 1, y => 2 }, # valid, with a comma
      b => { x => 1, y => 2 }  # ERROR, because it's ambiguous
  };

  The ambiguity arises on the "b=>" line, because if there's an implicit comma after it, then $h will be Hash; if there's an implicit semicolon, then $h will be Code. We can't intuit either way, so it's ruled into an error.
• As usual, the ruling turns out to have interesting repercussions when actually implemented (e.g. statement-level blocks shouldn't compose as hashes at all), so xinming++, TimToady++ and me took some time to repair the build. It's all good now, though.

Oh, and I'd like to thank Schwern++ for being an extraordinary localhost, even offering to drive me to Intel tomorrow and attend my talk.  That's good, because the number of registrations for DLS apparently doubled the expectation, so walk-in may not be possible for that maxed-out room. :-)

Check smoke results while reading Synopses

Our new smoke server now has some extra links with the name of SYN in the right margin of the page, where one SYN link corresponds to one smoke report:

http://m19s28.vlinux.de/cgi-bin/pugs-smokeserv.pl

Click on one of these links and you will be redirected to a list of Synopses. Enter one synopsis, say, ``02 Syntax'', and you will find out the magic.

As you can see, these synopses contain test passing/failing marks in the code snippets. (This feature was originally suggested by Christopher++ and Gaal++.) For example:

√	is($foo, "blah", "lone block actually executes it's content");
	my $foo2;
	{$foo2 = "blah"};
√	is($foo2, "blah", "lone block w/out a semicolon actually executes it's content");
	my $foo3;
	({$foo3 = "blah"});
×	ok(!defined($foo3), "block enclosed by parentheses should not auto-execute (1)", :todo<bug>);

Note that TODO tests are marked as passing only if they're actually passing (i.e., unexpected passing from the perspective of implementers) and they're marked as failing only if they're actually failing (i.e. expected failing). I don't introduce special marks for these tests because added complexity will lead to unnecessary confusion.

Skipped tests have no marks at all and the intention is straightforward. Just don't mistake blank tests for smartlinks.pl's bugs or something like that. ;)

There's also brief stats info in the link name so that you can see how many sub-tests are failing and how many are not without actually expanding the code snippet. (Thanks Juerd++ for suggesting this one.) For instance:

   - Show the snippet from t/var/var.t (line 13 ~ line 132  —  30 √, 0 ×) -

On top of the web page, there's a header like this:

  This page was generated at 2006-09-17 05:58:59 GMT.
  (syn r12136, pugs r13398)

where the revision number for pugs is extracted from the corresponding smoker's report. Christopher has ensured the version of the Pugs test suite used by the smoke server matches exactly the smoker's local Pugs. The timestamp was proposed by Juerd while the revision numbers for both syn and pugs were proposed by [particle]++.

All the Synopses on the smoke server are updated periodically by cron, even including old smoke reports that had been processed. We're trying to offer you the latest Perl 6 Spec everywhere. :)

For more details on the original motivation and Christopher's design decisions, please read the p6c threads below:

Integrating the Pugs test suite into the Synopses

synopses on smoke server

Thanks Gaal and Larry's tests.yml while I was implementing smoke result rendering feature in smartlinks.pl many weeks ago. At last but not least, I really appreciate Christopher's work on the smoke server/client mechanism to make this feature truly useful. (We don't need auto-smoking on feather any more, whee!) And I also thank iblech++ (yes, iblech's alive!) for helping Christopher through the whole process.

The Synopses on feather will still go without smoke results, since smoke results are always specific to a certain Perl 6 implementation or a Pugs backend. We hope you will still like it. :)

Audrey once said on #perl6 that she was biased to fix tests with smartlinks. And [particle] recently said that he had started a port of smartlinks.pl to PIR so as to cross-reference PGE's tests with the Synopses and parrot PDDs. Furthermore, I've seen more and more people are adding both tests and smartlinks to the Pugs test suite, which is really really wonderful. :D

Back at home.

I'm happy to report that I survived jet lag, a couple days of $job catchup, and am now ready to push forward with the long overdue merge-6.28.0-AST-to-trunk change, updating the Haskell runtime of Pugs to support the new Capture/Signature/MOP based AST.

Mad props to gaal++ for getting the new code to compile -- it's not compiled with "make pugs" by default, but that'll change as soon as I wake up.

Oh, and I'm happy to report that Hiveminder, the 0th Jifty application and my todo-list-keeper of choice, went live today!  There's already a quick review for what it has to offer.

I'd very much like to use Hiveminder as Pugs's task tracker, as soon as the public group feature is implemented. Give it a try -- you might like it too. :-)

The night before the "Jifty RHOX" talk.

I took the evening off from making slides, and went to see three great talks from Larry, Kathy and Damian. I very much did not regret going there, but that does mean I'm woefully behind on slides again:

Be as it may, here is what little I have so far -- I'll try to sleep a bit and finish them in the morning.  Wish me luck! :-)

Pre-OSCON intermission.

Still working on my slides (modulo the occasional perlmonks post), so updates in this space will be slower than usual.

Sadly, our IRC logger is currently down.  Here is a short recap from #perl6 logs:

• Sage++ is using Parsec to parse the Perl 5 regex syntax and  got most of them translated to Perl 6 regex in his  P5toP6 translator,
• scw++  meanwhile worked on translating Perl 6 regex and embedded MiniPerl6 into Haskell's Parsec syntax.  (Mmm circularity.)
• gaal++ worked on getting the new AST to compile (as Haskell and emit as Perl6/Moose classes), cutting down hundreds of type errors  to a couple of them.
• TimToady++ hacked the spec to introduce repeat while loops, resolving an issue raised by a recent p6l thread.  Then he hacked the test suite to reflect the new syntax so they fail. Then he hacked Pugs.Parser so it now supports the spec... and the tests pass again.  Yay!

Now I'll go back to make slides for the "Jifty RHOX my $Job" talk.  (i.e., "How Jifty tamed the Relational/Hypertext/Object/XML mismatch for my day job.")  See you at OSCON!

Random recaps.

Quick lists of some things happened today:

• Very productive afternoon Dockathon with Uri Guttman. We proof-edited S02 over two passes (#1, #2) to optimize for readability. Uri has moved to S06 after this, and we are also considering breaking up the specs by independent sections, to e.g. Perl6::Spec::Syntax::Quoting and put it on CPAN.
• Stevan++ promptly integrated Object::Declare with Moose, which makes its class declaration syntax even neater than before.
• fglock++ started the v6/ directory, the namespace designated for a Perl 6 compiler implemented in Perl 6, to maximize reuse from his, pmichaud++'s and my work. We also considered a plan to fly pmichaud++ to Brazil for YAPC::SA, so we can meet in person to make the p6-on-p6 "Christmas compiler" happen.
• Thanks to cdpruden++'s prodding, I finally fixed the long-standing chained-method-call-fails-lookup bug. However, it revealed a dark corner of unspecified semantics, namely the weird interaction between the method-call-falls-back-to-sub rule and the invocant-is-always-in-scalar-context rule. The commit message for r11640 has a detailed writeup.
• Scalar::Defer now works for objects as well, and fakes non-blessedness successfully by blessing values to package "0" with &0::AUTOLOAD and overloading. As seen on #perl6:

  <TimToady> or maybe it's overloaded somehow...
  <kolibrie> its overloaded
  <TimToady> which overloading, I wonder...all of them?
  <TimToady> seems like the only way it'd work.
  <kolibrie> use overload fallback => 1, map { $_ => \&force } qw( bool "" 0+ ${} @{} %{} &{} *{} ); sub force (&) { &{$_defer{ id $_[0] }} }
  <TimToady> Perl is a scary language.

Oh, and tsee++'s PAR::Dist::FromCPAN is really cute. Combined with Safe::World (the less-advertised but more-powerful Perl equivalent to Why's Ruby sandbox hack), this could make multiversioning and separate linking in Perl 5 happen without breaking everyone's existing code...

Boston.pm talk.

This afternoon I gave a revised Deploying Perl 6 talk to Boston.pm; thanks to putter++ for inviting me there! It was very nice to report the new progress that happened during the two hackathons since YAPC::NA, and people seems to like the v6-alpha roadmap very well. The slides are available as PDF and Flash as usual.

Was going to recap gaal++'s work the new AST and metaobject protocol today, but got myself sidetracked from #parrotsketch into a very long discussion on the possibility to deprecate an obscure feature (:immediate) from PIR.

Thanks to chip++ and allison++, a speedy consensus was reached within a day: We'll code up a saner alternative for the only thing people are currently (ab)using it for, but keep the :immediate syntax alive because there may be some unexpected use for it in the future.

It's wonderful to see #parrot and its RT queue becoming very active and helpful these days, due to Andy's successful call for janitors. Yay!

On the summaries front, kudra++ redid the Parrot section to fix the off-by-three-years error, and nothingmuch++ produced some scripts to make URI shortening and text formating easier, while preserving full links to Google Groups, as suggested by feedback comments on use.perl. It'll be a more smooth process next week... :-)

Hackathon recap: Multi Dispatch

Now that the week-long Boston Pugs hackathon reaches a closure, I'm getting some slices of time to look back at what the lambdacamels did over the past week.

First up is the draft multi-dispatch specification meant for inclusion in S06, building upon luqui++'s previous work. putter++ and spinclad++ spent many hours charting out in the design space, with inputs from stevan++ and later nothingmuch++.

Here's some syntactic background: In Chicago, I lobbied TimToady++ and dconway++ to distinguish between these two cases:

multi method f ($x: $y) {} # $x is self, $y is arg
multi method f ($x; $y) {} # implicit self, $x and $y both arg

It used to be that colon is used to separate both invocant/params and long names, causing painful ambiguities in the parser. TimToady++ came up with the idea of using semicolons to split up long names, and it was happily accepted. putter++ also suggested the double-semicolon notation to distinguish between these two cases:

multi f (Int $x; Int $y) {} # $x is more important than $y
multi f (Int $x;; Int $y) {} # $y does not participate in tie-breaking at all

instead of the currently specified:

multi f (Int $x; Int $y;) {} # $x is more important than $y
multi f (Int $x; Int $y) {} # $y does not participate in tie-breaking at all

Regardless of the surface notation, this new draft is important to me, as it de-handwaved several points left unanswered from luqui's previous draft, namely:

• How multiple long names, separated by semicolons, interact with each other.
• How implicit coercions affect tiebreaking.
• How named subset types, as well as anonymous where clauses, compare with their base classes.
• How Roles and Classes differ in the tiebreaking process.

clkao++ paired with me to produce Sub::Multi, a way to realize the dispatch algorithm on top of Data::Bind signatures. We still need some help on hacking Moose::Meta::TypeConstraint to express union types and partial ordering among constraints, but it's a good start already. Furthermore, we also need a sugar layer, perhaps based on Sub::MicroSig, that makes declaration simple (and filtering-free); this way we can rebuild Perl6::Subs on top of them, to support multi dispatch cleanly and efficiently. Stay tuned!

Release announcement.

[ANNOUNCE] Pugs 6.2.12 and v6.pm released!

(Due to non-stop Hackathon since YAPC::NA, this announcement has been delayed for a week. :-))

I'm glad to announce that Pugs 6.2.12 is now available from CPAN:

    http://search.cpan.org/dist/Perl6-Pugs-6.2.12/
    SIZE: 2693459
    SHA1: c9731da8e597591ca7e279766481ce0bece8cfa4

This release features much better performance: Building Pugs is 3 times faster; compiling Perl 6 programs becomes 10 times faster; running Perl 6 programs is now up to 2 times faster.

We also support various Perl 6 syntax changes since the last release, as well as exciting new features, such as atomic code blocks with Software Transactional Memory (STM) semantics.

6.2.12 marks the last release with the 6.0.x abstract syntax tree (AST) to represent Perl 6 programs.  We are currently switching to a new AST that supports the new Signature/Capture calling convention, multi-dispatch with constraints, and a full Meta-Object Protocol (MOP).  We are developing this new runtime simultaneously as Haskell modules and Perl 5 CPAN modules, to ensure that they have identical semantics.

To this end, I'm happy to announce that "v6.pm", a prototype Perl 6 Compiler implemented entirely in Perl 5, is also available from CPAN:

    http://search.cpan.org/dist/v6-pugs/

All Perl 5 components are released as separate CPAN modules.  One can use them as pure-perl5 modules, and get fully-conformant Perl 6 features, without the need of using Perl 6 syntax provided by "v6.pm".

(These CPAN modules maintain their own release cycles; we will release more components on CPAN as they are abstracted out from the Pugs runtime.)

The .meta API for Object/Class/Method reflection is supported by the "Moose" and "Class::MOP" modules:

    http://search.cpan.org/dist/Moose/
    http://search.cpan.org/dist/Class-MOP/

The compiler and runtime for Perl 6 Grammars (top-down) and operator precedence (bottom-up) is available as the "Pugs::Gramamr::Rule" and "Pugs::Grammar::Precedence" modules:

    http://search.cpan.org/dist/Pugs-Compiler-Rule/

The calling convention with named, optional, and slurpy arguemnts is supported by the "Data::Bind" module:

    http://search.cpan.org/dist/Data-Bind/

The precompile-Perl6-to-Perl5 mechanism is based on "Module::Compile", a safe and composable replacement to source filtering:

    http://search.cpan.org/dist/Module-Compile/
    http://search.cpan.org/dist/Filter-Simple-Compile/

In summary: Perl 5 is now a first-class virtual machine for Pugs, and in this journey toward self-hosting, we will share as much common structure as possible between the Perl 5, Haskell, and the Parrot runtimes.

With a prototype end-to-end implementation written in pure Perl 5, we are entering the "Hack, Hack, Hack" phase of the (imaginary) Perl 6 timeline from nearly one year ago:

    http://pugscode.org/images/timeline.png

I'd like to thank Flavio Glock for initiating and leading the v6.pm effort, and all lambdacamel and lambdamoose on irc.freenode.net #perl6 for their relentless enthusiasm in getting Perl 6 deployed to the Real World.

See you on IRC!

Audrey

Chicago Hackathon Day 1.

Today's multitasking-collaboration was highly, extremely, productive, on many fronts:

• I finally sat down with gaal++ to do the long-awaited P6AST redesign.
• uri++ proofread much of S02 and found a huge number of typographical/forward-reference-pedagogical errors.
• TimToady++ and I finally worked out the multi dispatch and implicit coercion semantics (not yet committed to S06).
• clkao++ implemented the Capture/Signature semantics for methods on Data::Bind (soon to be on CPAN -- after SVK 1.08's release I'm told).
• cwest++ started coding trypugs, a tryruby clone harnessing Pugs's interactive shell -- coded in Jifty, no less! (We'll probably deploy it on feather.perl6.nl soon.)
• With pmichaud++'s help, sanug++ (George Wood) made some amazing Perl 6 architecture diagrams, re-expressing the Pugs compilation flow under Parrot/PGE and Perl5/v6.pm's context. I've made some draft PDFs available by exporting the OmniGraffle Pro source file.
• kolibrie++ worked on (ab)using the <ws> rule to allow whitespace inrules to match arbitrary junks (an idea reminiscent of Template::Extract). With pmichaud++'s help, his proof of concept now worked flawlessly with PGE.
• stevan++ worked on Moose to push more non-sugary protocols into Class::MOP, and re-implement the two missing Perl 6 metaobjects (Package for adding, removing and testing existence of symbols, and Module for version and author metadata, as well as importing/exporting of symbols.)
• Ingy++ clarified the YAML situation on Perl tags, ruling that !!perl is correct, but both YAML::Syck and YAML.pm should recognize the !perl form.

There were far more happening than this -- everybody was hacking and discussing and triaging acively in the Common room -- but I'd need to sleep now, and resume the continuation tomorrow. See you!