Pugs

Today's release-a-thon.

Today the release-a-thon continued, with gaal joining the bug triaging. We made good progress, but sadly $job days begins from tomorrow until Friday, so the release schedule continues to slip toward this weekend. :-(

Leo implemented the nested subroutine scoping in Parrot, so closures is now always closed at compile time first, and then recloses itself at runtime as neccessary:

sub f ($x) {
    sub g { $x }
}
g();     # undef -- pad not initialized
my $g = f(10);
$g();    # 10 -- sub{$x} is returned at runtime
g();     # 10 -- also &g is rebound
f(20);
$g();   # 10 -- the old closure is kept around
g();     # 20 -- the new rebound &g takes the new $x

This behaviour is dynamic, nice, fun, and sufficiently perlish. :-)

Now that the fallout from the lexical import change is mostly cleared, I can resurrect evalbot to #perl6 and rest in peace for tonight. See you tomorrow!

Vienna Hackathon.

Leo just confirmed that we'll gather together in Vienna after GPW (March 1-3), for another week-long Hackathon on Perl6/Parrot integration.

Last time Chip and I stayed at his place, we made Pugs 6.2.0 (procedural interface without objects) target PIR through a then-newly-specified PIL language.

I hope we can repeat the same performance for the 6.28.0 (object model) milestone, after the Perl 5 targetting in the Israel Hackathon. We can approach this by porting the PIL^N representation types into PMC types, following the step of Lua and Tcl. It'd be a lot of fun. :-)

Oh, and it seems that Larry and Damian may come to YAPC::Asia as well.  The Tokyo hackathon is looking more impressive by the day; I wonder if we can get Ruby people into the mix...

 

Bug triaging continues.

I lost most of today's productivity, thanks to an allergic reaction to hair colorants (sigh). So while we brought the test fail count down from 2933 to a more manageable 1396 (and implemented "env" scope and $+outer variables along the way), there was not sufficient time to triage the rest of t/pugsbugs/.  There is always tomorrow, though...

DrIFTing YAML.

Three more backends joined Pugs today:

• "./pugs -CPIL1-YAML" dumps the PIL1 tree using gaal's Syck emitter binding.
• "./pugs -CPIR-YAML" serializes the PIR syntax tree generated from PIL1.
• "./pugs -CParse-YAML" does the same for the parse tree itself, upon theorbtwo's request.

They are made possible thanks to the DrIFT technology. It lets us specify how to convert any Haskell structure into YAML nodes, with a simple rule file and some base instances.  From there on, say if we'd like to add various serialization for PIL^N native types, all it takes is a single pseudo-comment in the .hs-drift file:

{-! global : YAML, Perl5, JSON, HaskellXml, ... !-}

Debugging PIL1, PIR and the parse tree is made much simpler with this effective visualization tool. I've also released a new YAML::Syck that can load Haskell structures into blessed Perl5 objects, although Ingy's YAML.pm 0.50 also does this in a more elegant fashion, without involving Class::Rebless.

Loading from YAML to arbitrary Haskell structures should also be straightforward; now we just need a Syck emitter binding at Parrot's side, and we can shuffle Parrot objects (e.g. PGE Match objects or TGE trees) with Pugs's compiler backend across separate compilation units. That should be fun...

PIR compilation lives again!

After a 18-hours hacking run, I fixed all three blocking issues today.

Thanks to generous help from Leo and #parrot folks, Pugs now generates correct PIR; the parrotBrokenXXX global flag we used to dodge around register alligators is finally gone. Hurray for a new dawn of non-scheduled Parrot!

The lexical pad spec proved to be quite sane and easy to target, except for the somewhat irritating demand of having to manually initialize lambda lifted closures, even for package-scoped subroutines. For example, this Perl 6 code:

my $x;
sub f { $x++ }
f();

becomes this PIR code (you can try it out with "./pugs -CPIR test.p6"):

.sub "main" :anon
    .local pmc s__x
    .lex "$x", s__x
    $P8 = find_name "&f"
    $P8 = newclosure $P8
    store_global "main", "&f", $P8
    # ...
.end
.sub "&f" :outer("main")
    # ...
.end

Running newclosure for each subroutine at runtime is quite silly, because its outer subroutine "main" cannot be entered again. Moreover, it prevents code in BEGIN and other packages from calling &f using fully qualified syntax.

It would be nice if the newclosure can be performed at compile time, e.g.:

.sub "main" :anon
    .local pmc s__x
    .lex "$x", s__x
.end
.sub "&f" :outer("main") :closure
    # ...&f is initialized  automatically to see "$x"
.end

 

Leo suggested me to write this request up and post it to p6i, so here it is. :-)

Releng for 6.2.11 starts.

I slept for 16+ hours today, and woke up to find gaal had made a first cut at changelogging, and the freshly-returned putter starting to check in the new prelude -- an implementation of builtin functions in Perl 6.

Presently I'm working on the three remaining items blocking 6.2.11:

• Emit.PIR
• Pugs.Embed.Parrot
• Lexical export of multisubs

I'll continue working on release engineering with a full (and less sleepy) day tomorrow, as well as the day after if neccessary.  Stay tuned...

lighttpd++

Today is again a full $job day.  Thanks to Jesse, I discovered the amazing lighttpd web server (the  equivalent of SQLite of web servers), and promptly hacked Jifty::Script::FastCGI to support it.  These two lines of config worked wonders on a FreeBSD setup, providing such an instaneous feedback that I simply wasn't used to:

server.event-handler = "freebsd-kqueue"
server.stat-cache-engine = "fam"

On another 350mhz AIX machine (on which I can't manage to get mod_perl2 or mod_fastcgi built), the response speed is easily twice that of Jifty's native web server, and the memory footprint is a mere 3mb. Authentication is also made much easier with the builtin FastCGI authorizer.

Much kudos to Jan Kneschke and the lighttpd team for creating this lovely lighty creature. :-)

A bit more YAML::Syck.

I'm still neck deep in Jifty dispatcher for $job, so there is nothing much to report today, except that I've released two more versions of YAML::Syck, with support for loading/dumping of Unicode strings, scalar references, as well as dumping self-recursive data structures.

As syck doesn't have native Unicode support, I had to do some syck hacking to get these features implemented... I'll  find some time to package it up properly and send back to Why, after this weekend's Pugs 6.2.11 release.

Chupei #3: JSON::Syck and YAML.pm released.

Last night, Ingy döt Net changed his Chinese name to 應吉大聶, thus gaining the unique status among Pugs committers to have two non-ASCII names.  And you thought I was weird...

On a more serious note, miyagawa and I continued hacking on JSON::Syck,
running into weird Perl 5 XS issues left and right.  But our persistence
prevailed: with a simple C-based pre-/post-processor to handle JSON's lack
of whitespace after comma and colons, JSON::Syck now supports full JSON
syntax.

Here is some benchmark results with pure-perl5 JSON.pm.  As you can see, we are 3-5x faster:

[Dump] JSON.pm: 44 wallclock secs (38.20 usr +  0.00 sys = 38.20 CPU) @ 2618.12/s (n=100000)
[Dump] JSON::Syck: 14 wallclock secs (13.12 usr +  0.00 sys = 13.12 CPU) @ 7619.05/s (n=100000)

[Load] JSON.pm: 66 wallclock secs (46.11 usr +  0.01 sys = 46.12 CPU) @ 2168.39/s (n=100000)
[Load] JSON::Syck: 12 wallclock secs ( 9.63 usr +  0.03 sys =  9.66 CPU) @ 10347.62/s (n=100000)

Ingy finally released YAML.pm 0.50, the first major release in almost four years to the pure-perl YAML implementation.  Aside from a huge set of functionality improvements, it also comes with a comprehensive test suite to eliminate corner cases that the earlier YAML.pm versions were notorious of.

However, the rigorous refactoring did run against Perl 5's method call overhead (which I'd like to eliminate with Sixxy.pm), so the benchmark numbers differed  by 20-40x this time:

[Dump] YAML.pm: 308 wallclock secs (285.05 usr +  0.06 sys = 285.12 CPU) @ 350.73/s (n=100000)
[Dump] YAML::Syck: 13 wallclock secs (13.29 usr +  0.00 sys = 13.29 CPU) @ 7524.99/s (n=100000)

[Load] YAML.pm: 474 wallclock secs (422.63 usr +  0.05 sys = 422.69 CPU) @ 236.58/s (n=100000)
[Load] YAML::Syck: 12 wallclock secs (10.95 usr +  0.04 sys = 10.98 CPU) @ 9103.84/s (n=100000)

Gaal and clkao worked on Pugs release engineering, by repairing "make smoke" and ensuring that the JSPerl5 backend did pass its usual 80%+ share of subtests.  I've also briefed Ingy about Pugs's OpTable/Rules implementation, though I wonder if he'll find sufficient cycle to read through Programming in Haskell...

Unfortunately, I had to leave the hackathon early tonight, as I have three consecutive $job days of Jifty hacking from tomorrow on.  As of this writing, the hackathoners are still going strong in Chupei... Happy hacking, folks!

Chupei #2: Crazy web magicks.

Thanks to miyagawa, you can take a look of some of the hackathoners from the photos he took.

Gugod hacked on Kwiki::Theme::Ajax, a JSAN-enabled Kwiki theme that provides a smoother wiki experience.  See kx.gugod.org for a demo -- note the very clever way to mark page links using #PageName fragment identifiers. I wonder if Wifty can take a page or two from it...

A new hackathoner hlb joined our ranks today. As miyagawa brought me a book on Takahashi method, we chatted about my XUL-based slides, which works really well but doesn't downgrade to w3m, KHTML, or anything else than Gecko.

To address this issue, hlb started from Eric Meyer's S5 and added resize-to-the-max JavaScript controls to it (here is a work-in-progress demo). Once Gugod's Spork::S5 is ported to support this new S5+Takahashi theme (with maybe some presentacular), I'll promptly switch over all my talks to use it.

Ingy continued hacking on YAML.pm. Now that Gaal has finished object/yaml roundtrip in Pugs (see his livejournal entry on this), I'm delighted to learn that YAML.pm now supports marshalling non-perl5 nodes into Perl 5 objects; now we can really serialize all the Object Space and PIL tree into YAML via Syck. Which would need a DrIFT rule to dump/load all Haskell structures.

I made three more YAML::Syck releases, adding the capability to toggle implicit typing using a package variable $YAML::Syck::ImplicitTyping. Ingy noted that it should default to false (and handle only the '~' => undef case), and handle things correctly when it's set to true. I've delivered both, thanks to Why's rubyext.c code, and now YAML::Syck parsing should be even faster. :-)

Miyagawa hacked on a lot of CPAN modules, such as XML::Liberal, Catalyst::View::JSON, WWW::OpenSearch, Kwiki::TypeKey, WWW::Blog::Metadata, Kwiki::OpenSearch::Service, and
various other CoolTechX::CoolTechY bridge modules.  Also see miyagawa's own account about the hackathon in his bulknews blog...

Now it's way past 5am, and I should sleep. Ingy, gugod and miyagawa are all still hacking on; Ingy seems determined to unleash YAML.pm 0.50 to the world, with its thousands of Test::Base based tests.  Happy hacking!