Pugs

Day 118: <punt>

After some 21 hours of non-stop hacking (on $work, Pugs, and the Haskell Workshop paper), I'm afraid I need to punt on today's journal, especially because there's some 80 commits today (largely driven by Catalyst6, Test::Builder and Locale::KeyedText). It was a lot of fun, though; the OO core of today is an entirely different beast than it was yesterday. :-)

Day 117: evalbot; Maypole and Catalyst for Perl 6.

I'm surprised and delighted to see lathos porting his Maypole framework to Perl 6:

<lathos> I'm actually *really* enjoying porting Maypole to Perl 6. SO much cleaner.

I relayed this news to #catalyst, only to discover that sri has already beginning a Perl 6 of Catalyst. I happily sent commitership invitation mails to both of them; it is an honour to have two such illustrious hackers on board.

Another big news today is iblech's evalbot.p6, an IRC bot providing ?eval command for all #perl6 folks to try out Perl 6 code with. To protect against system(), a safe mode is implemented to disable all IO primitives, when Pugs is started with a true value in the PUGS_SAFEMODE environment variable. The evalbot has already proven to be a very useful quick-checking tool during language discussions; iblech++ for getting it written!

iblech also added the $?PUGS_VERSION special variable to represent the revision number of Pugs.

eric256 and his wizard.p6 continued to drive more development:

• .attr = 1 is no longer parsed as $_.attr(=1).
• use lib '/path'; now works correctly by calling ::lib.import('/path').
• when .does(Foo), when $_.does(Foo) and when Foo are all implemented now.

Some more work on the Pugs/Perl5 embedding front:

• Perl 5 can now invoke Pugs objects as well as closures.
• Chained method application like DBI.connect($x).do($y) now works properly.
• Tweaks on C compiler flags handling to make it compile on even more platforms -- as long as GHC and Perl was built with the same C compiler.

broquaint wondered why $str.split(//) was parsed as split($str, //). I fixed that specific case, but $x.split($y) still does not mean split($y, $x) yet; broquaint added various tests for them. Also at broquaint's request, iblech implemented &getc.

iblech added some information about using the Compose key under X11 to enter special characters. I followed the instruction with the Menu key, but it had no effect at all. Strange...

Aankhen ported HTTP::Headers::Util from Perl 5, along with its unit tests.

SamB started working on Pugs.Compile.Pugs to produce more readable output code.

hcchien, our Taipei.pm leader and one the first committers, returned to Pugs today and implemented a input method driver for lukhnos's OpenVanilla framework. He remarked that Pugs is not yet fast enough for a real-time imput method. This will likely change next week when I started working with Leo...

There have been a flurry of commits on util/perl6.vim syntax file from iblech, ninereasons, scw and me. With the improved highlighting, it's now much easier to edit OO modules in Perl 6.

That's it for today. See you tomorrow!

Day 116: Massive backlogging.

Stevan's life as the ChangeLog scribe would probably be much miserable, if I actually skipped all daily summaries for this week. So I'm back now to summarise the developmeng since Pugs 6.2.5's release:

• Multi-invocant calls like foo($x, $y: $z) is no longer legal, according to Damian's new revelation. That broke &sub.goto for a while (since you can't re-appoint invocants anymore), but it's fixed now.
• Class methods now works; Int.isa(Int) now returns true.
• ./SUPER::method() implemented at chromatic's request.
• pugs -MFoo now means use Foo instead of require Foo. That makes pugs -MDBI--perl5 -e '...' work correctly.
• chromatic hacked in DESTROYALL/DESTROY support with help from scook0 and SamB, taking advantage of GHC's fast generational GC system and System.Mem.Weak's support for adding finalisers. I also forced a GC run to always happen on program termination.
• luqui implemented exclusive-range operators: ^.., ..^ and ^..^.
• integral (aka bsmith -- I'm going to switch to IRC nicks as primary handles) continued to factor away AST, Parser and other modules into smaller submodules.
• gaal added quickref/unicode to document how to enter those Unicode operators in editors, upon dragonchild's request on p6l; Juerd also improved layout on many quickref documents.
• ninereasons worked on the cookbook, in particular 01-02default-variables.t, adding idioms like the use of [//] @foo to find the first defined value in @foo.
• iblech adjusted the scaling of svnlog2graph.pl again, because we have more 4000 commits now. Parrot has ~8000, so there's still some way to go...
• mugwump played with the new coro support, adding a (not fully working) iterator interface to his Tree.pm module.
• theorbtwo and scook0 tweaked PA02 to make it more readable.
• mugwump rearranged metamodel-related modules under ext/Perl-MetaModel to be more consistent; the beginning of a Perl 6 port of Pugs's source code is interesting.
• Stevan updated perl6_meta_meta_class_hierarchy.pod; I asked chromatic to audit them before I implement them in Pugs.
• rgs, integral and clkao worked on Makefile.PL's support for Perl 5 embedding; it should now work on most *nix platforms.
• Limbic_Region needed hyper-dereferencing operators, so he wrote test for them.
• iblech made &reduce and &map accept n-ary functions, in a similar way as the for block.
• dakkar, our new committer, added a JAPH written for the t-shirt of the Second Italian Perl Workshop:

  say [~] (-> @c is copy {gather { while @c[0] { for @c -> {take(.shift)} } }
  }(['Joec','utrk','shle','te6r',' r .','a h.','nPa.'].map:{[split "",$_]}));

• Aankhen added &escapeHTML and &unescapeHTML to CGI.pm, refactored CGI::Header to behave more like the Perl 5 counterpart, implemented full support for CGI::redirect, and ported over Perl 5's CGI::Util module.
• cdpruden, our new committer, found that our &eval_haskell is broken with newer hs-plugins. Old versions with non-hierarchical namespace is no longer supported.
• Corion declared that &times now works on Win32 for real.
• iblech made die "abc\ndef\nghi" correctly align vertically.
• gaal started working on the standard Prelude.pm, a set of not-really-primitve functions that can be implemented in Perl6. The first example is &Pipe::open (but known as &openpipe for now.
• Hyper-reduction prefix operators like [+]« are now parsed, thanks to iblech.
• iblech also made &readline handle UTF-8 correctly, so svnbot6 can finally announce the commit messages as UTF-8.
• scook0 added more Haddock, and renamed the enterLoop internal function to the more correct enterWhile.
• fireartist, our new committer, added many tests for corner cases in array and hash slicing.
• iblech unbroke "$hash<key>" in string interpolation.
• nothingmuch added a draft module, Class::Events, a publish-subscribe event framework much like Class::Publisher. With luck (and handles delegation support), hopefully he will finish it before the 6.2.6 release and move to ext/ where it belongs...
• iblech contributed another useful real-world Perl 6 script, screen-nodestatus.p6, that monitors host aliveness for GNU screen users. The bottom-right corner of this screenshot has this in action.
• Aankhen moved ext/LWP-Simple/ to ext/libwww-perl/ after some discussions in #perl6.
• ninereasons improved support for inline documentations in perl6.vim. I wonder when will we see a perl6-mode for Emacs...
• SamB, our new committer from lambdaland, added export lists Pugs.Types, while investigating various ways to speed up the compilation process.
• iblech implemented implemented a couple Control::Basic::* functions from S29: &nothing and &evalfile.
• ninereasons clarified a pair.t test that reflects Larry's ruling that => now assumes assignment precedence, so 1 => 2 ?? 3 :: 4 will construct a pair. I promptly implemented it.
• Aankhen improved porting_howto to keep it up to date with recent practices.
• eric256 continued to work on wizard.p6, uncovering many issues with given/when and wrote tests for them. Thanks to his report, when .method() {...} now works.

Whew. I think that's all. See you tomorrow!

Day 115: More Pugs/Perl5 fun

See clkao's journal. Also note that we can pass coro {...} to Perl 5 land too.

This also works:

#!/usr/bin/pugs

use v6;
use DBI--perl5;
# use perl5:DBI; # in Pugs 6.2.8+, write this instead

unlink 'test.db';

my $dbh = DBI.connect('dbi:SQLite:dbname=test.db');
$dbh.do("CREATE TABLE Test (Project, Pumpking)");

my $sth = $dbh.prepare("INSERT INTO Test VALUES (?, ?)");
$sth.execute(<Pugs Autrijus>);
$sth.execute(<Ponie Nicholas>);
$sth.execute(<Parrot Leo>);

my $res = $dbh.selectall_hashref('SELECT * FROM Test', 'Pumpking');

# Just another Pugs hacker
say "Just another $res<Autrijus><Project> hacker";

Still lots of things left to do -- I want to share runloop as much as possible, so $CALLER::x can see into Perl 5 lexicals, and maybe reify the control stack both ways via Coro.pm if it's installed. But this is already quite usable for real-world scripting... I'm so excited. :-)

Day 114: Feather -- aka perlcabal.org -- flies!

Thanks to Juerd's generous offer, we now have a dedicated box for Perl 6 development. Much kudos to Twistspace for sponsoring the rackspace and connectivity. If you wished to join the Pugs/Parrot/Ponie teams, but found setting up GCC/GHC/SVK/HsPlugins/PerlEmbed/IRC difficult, please ask for an account from Juerd and join us!

The canonical domain name for the machine is feather.perl6.nl, but I have also pointed perlcabal.org and dev.pugscode.org to it. The read-only darcs mirror has been moved there; we also have a shared SVK mirror in the new box.

I worked some more on Perl 5 embedding; the context from Perl 6 is now propagated to Perl 5, and AUTOLOAD etc are handled as well. I plan to give the existing eval_haskell() and eval_parrot() similar treatment, so you can say:

use DBI--perl5;
use SDL--parrot;
use Test::QuickCheck--haskell;

# ... do lots of interesting things using those modules ...

I'm glad to see that clkao finally checked in his Benchmark.pm. He also gave me plenty of good ideas of representing Pugs thunks in embedded Perl 5, which will make coroutines and continuations available in Perl 5 land.

Oh, and clkao also threatened to port svk to Perl 6, after taking a look at Pugs's internals -- as seen on #svk:

<clkao> bad news. i like haskell

...and so I skipped my summarising duty for another day. Sorry. :-)

Day 113: Pugs runs CPAN modules!!

Earlier today, I've posted my Perl 6 (wishlist|todo list) as a survey. The responses were very constructive. My #1 wishlist item is apparently shared by many:

• An improved Inline::Pugs so I can call Perl 5's DBI.pm from Perl 6 code, and manipulate Perl 6 objects from Perl 5.

So, wish help from clkao, we just did it in the last 3 hours. There's many rough edges that will be tested and fixed soon, but this already works:

use Digest::SHA1--perl5;

my $cxt = Digest::SHA1.new;
$cxt.add('Pugs!');

# This prints: 66db83c4c3953949a30563141f08a848c4202f7f
say $cxt.hexdigest;

My full wishlist, ordered by descending importance, is reproduced below:

1. An improved Inline::Pugs so I can call Perl 5's DBI.pm from Perl 6 code, and manipulate Perl 6 objects from Perl 5.
2. A fully working Perl 6 compilation backend to Parrot or C, so Perl 6 can run in a very competitive speed.
3. A more stable core language definition, with both online and offline documentation.
4. Unit tests for the language and the "core" modules/pragmas I'd use.
5. A few really neat Perl 6 modules that simply can't be done in Perl 5. A robust Seaside/Halo implementation that makes Rails/Catalyst pale in comparison; a combinator library that can be used for formatting, parsing and template generation; et cetera.
6. Shared introspection, threading and debugging support for Perl 5 and Perl 6, as offered by Ponie.
7. A translator from Perl 5 code to Perl 6, to ease migration efforts.
8. ...after those, a fully bootstrapped Perl 6 compiler written in Perl 6, and for Larry to call it Perl 6.0.0. :-)

I think I can be excused of not doing the usual summary today. :-)

Day 112: Pugs 6.2.5 released!

On behalf of the Pugs team, I am elated to announce the release of Pugs 6.2.5, with much more comprehensive OO support, hyper and reduction metaoperators on user-defined operators, as well as experimental coroutine support.

Also of note is pugs.pm and Inline::Pugs, two Perl 5 modules that lets you inline Perl 6 code in Perl 5 programs. The docs/quickref/ series of documents is also helpful as a companion to the Synopses.

You can download Pugs from a nearby CPAN mirror, or from pugscode.org:

Perl6-Pugs-6.2.5.tar.gz
SIZE = 1012358
SHA1 = a4442eb156b919ec139ab2b64e3ea9db47ed371c

A Pugs/Parrot live CD, contributed by iblech, is available as well:

pugs-livecd-6.2.5.iso

Amazingly, the project's velocity is still increasing, with 680 new commits, compared to 507 from 6.2.3. The graph below shows the growth of number of commits and committers:

pugs-svngraph-6.2.5.png

Again, my sincere gratitude to all lambdacamels for making this release as exciting and enjoyable as it is. See you next week!

Thanks,
/Autrijus/

Changes for 6.2.5 (r3794) - May 24, 2005

Bundled Modules

• Fix one broken test from the Set module.

Changes for 6.2.4 (r3790) - May 24, 2005

Pugs Internals

• All infix operators now receive reduction forms, such as [+]
• All operators now receive hyperised forms, such as >>+<< and ~<<
• Dereferencers: @{...} and @$var
• Experimental support for coro { ... }, coro name { ... } and yield()
• Experimental support for lazy {...}
• External Parrot for Rules is now kept in a single process
• Inheritance: class Foo is Bar and class Foo does Bar
• Interactive shell commands normalised to always begin with :
• MMD handling is now more sophisticated, and $.chained.attr.methods works
• Much better error messages, with cascading stack trace
• New ./method syntax implemented
• Objects numify to a unique value accessible with $obj.id()
• Parrot compiler backend now handles namespaces and method calls
• Parsing of hierarchical return types: sub foo returns Hash of Str
• Private attributes: has $:foo now generates private accessors
• Private methods: method :foo () and $obj.:foo
• Switch to sum-of-inheritance-level distance for MMD dispatch on invocants
• Symbolic references: $::(...) and $::some::("var")::($bar)
• User-defined symbolic infix, postfix and prefix unary functions
• $?CLASS and $?PACKAGE works; $?ROLE currently works as $?CLASS
• &code.name and &code.arity added
• FIRST {...} and my $x = FIRST {...} support
• INIT {...} and CHECK {...} blocks in void context and as rvalues
• OUTER:: scope implemented
• do {...} literal added
• gather { ... } and take() implemented
• submethod BUILD is called for each parent class and class itself
• time() now returns a fractional number
• try {...} literal allowed at expression level
• warn(), uniq(), fail(), times() implemented

Bundled Modules

• Inline::Pugs and pugs module to inline Perl 6 into Perl 5 programs
• Locale::KeyedText re-added (this was our first contributed module)
• Net::IRC, OO version added
• Perl::MetaModel, prototype of Perl 6 OO meta-model in Perl 6 OO
• Set::Junction and Set::Hash added as implementation backends to Set
• Set now has many overloaded operators
• Test::Builder, with Perl 6 objects (parses, and mostly works -- see tests)
• Tree::Simple, renamed to Tree and converted to OO
• Test now gives better diagnostics to cmp_ok()
• fp module added for functional programming

Tests, Examples and Documentations

• Many new test and several tests refactored, we now have 5600+ tests
• Hangman IRC bot created from hangman.p6
• IRC logfile to HTML converter added
• Initial sketch of Pugs Apocryphon 2 as docs/02Internals.pod
• Much work on internal Haddock Haskell documentation
• OO Wizard RPG game added in examples/games/
• Parrot is now included in the Pugs Live CD
• Perl 6 quick reference documents added to docs/quickref
• Perl6::Rules test suite incorporated into t/rules/
• Removed usage of force_todo() in favor of :todo

Bug Fixes

• Bare blocks containing $_ is now executed correctly
• Correct parsing for user-defined nullary functions
• Hash and array sub parameters are read-only by default, same as scalars
• Post-term invocation in interpolation no longer eat trailing whitespace
• Slurpy hash parameters no longer count as nonslurpy during arity matching
• Type-to-type smartmatch, e.g. Int ~~ Num, now works
• $obj.method($arg1, $arg2) can now MMD dispatch over all arguments
• foo 3 and foo 4 is now parsed as two separate function calls
• loop (;0;) {...} will no longer execute the loop body
• map({...} @list) is no longer valid syntax
• next now re-evaluates condition in loop constructs
• returns Foo::Bar from subs/methods now works
• split// now attaches the submatches to the resulting list
• state $x = 42 now only assigns $x once
• system() returns proper exit codes

Day 111: Hyper operators and named blocks.

I mentioned yesterday that hyperoperators are coming soon. As expected, today iblech and I hacked them in. Now all prefix, infix and postfix operators have hyper forms, regardless of whether they are user-defined or builtin:

pugs> (1,2,3) >>-<< (4,5,6)
(-3, -3, -3)
pugs> -<< (4,5,6)
(-4, -5, -6)
pugs> sub postfix:<!> { [*] 1..$_ }; (4, 5, 6) >>!
(24, 120, 720)

But it gets scarier when you mix hyper metaoperator with reduce metaoperators:

pugs> [+]<< ([1,2], [3,4]);
(3, 7)
pugs> [+]<<<< ([[1,2], [3,4]], [[5,6], [7,8]])
((3, 7), (11, 15))

iblech implemented FIRST {...} with trailing undef, INIT {...}, and CHECK {...}. Together with the existing support for BEGIN {...} and END {...}, we have a pretty good coverage of global closure traits now. Next in sight would be things like PRE and POST...

clkao is working on Benchmark.pm, which really wants times() support. I walked him through the implementation and testing in 30 minutes; Corion then ported the underlying getProcessTimes Haskell function to Win32.

clkao tweaked cmp_ok() from Test.pm to show the name of the comparison operator used. I hacked in &code.name to support this, as well as &code.arity.

Juerd added quickref/reduction for reduction metaoperators, and iblech contributed some more examples to it.

Backtrace in error messages is now much more legible:

$ ./pugs -e 'sub{ sub {die "Error!"}.() }.()'
*** Error!
    at -e line 1, column 1-32
       -e line 1, column 6-28
       -e line 1, column 11-23

iblech added the ability of dumping raw AST structure to the interactive shell. Following that, I reorganised all commands in the shell to being with :. Hopefully this will eliminate the confusing situation of !1 evaluating to 1 in the shell.

Stevan continued to study metaclass koans, aiming for grokking the inner working of Perl 6's OO model. He recorded his current understanding in the Perl 6 Meta Meta Class Hierarchy document.

Also in the metamodel front, mugwump updated Perl 6 Meta Model with the latest insights.

asavige added some precedence tests, more todo tests to assign, and reported that 1..Inf as parsed as &.Inf(1). I fixed it a while later.

I also implemented the three "yada" literals: ??? that warns, !!! that dies, and ... that fails -- i.e. dies when use fatal is in effect, otherwise return a delayed error that dies when actually evaluated.

gaal added missing signatures for the Win32-specific parts of source code. With this, Haddock will finally recognise all our toplevel functions.

knewt, our new committer, made the backend part in command line parsing case-insensitive, so pugs -Bparrot will also work.

Aankhen added support for stringifying cookies to CGI.pm, un-evalled passing OO tests, and cleaned up operator_overloading.t.

adehohum added many tests, ranging from OO, syntax, operators to rules. scook0 also added tests for exclusive-range operators.

eric256 improved the wizard.p6 game by adding rooms and more enemies -- it seems that the player doesn't stand a chance against the Skeleton -- but curiously, fleeing from the room made the enemy disappear. This bug would probably not last for long...

Day 110: Beginning of PA02.

Today saw nearly 100 commits, doubling the usual amount. Sadly, I've already spent my tuits writing two more Euro OSCON proposals -- "Perl and Internationalisation" and "Introduction to Pugs" -- and now I'm way past my bedtime. So, only short entries today.

nothingmuch checked in the first part of Pugs Apocryphon 2 - Overview of Pugs Internals. There will be many parts (and many more interviews) to come.

iblech hacked a lot today:

• Implementation of lazily-evaluated fail primitive, temporarily known as fail_ to avoid clash with Test.pm.
• FIRST {...} works.
• state $x = 42 works.
• &::("infix:<+>") works.
• $::some::("var")::($bar) works.
• Unbreak <=> etc caused by yesterday's symbolic ref code.
• A new obfu: overloading random operators.

clkao was biten by index (1,2) not being parsed as index(1,2), so I added a more helpful error message at his request.

New committer chuck added a check to Makefile.PL to disable Parrot embedding with mixed MSVC/MinGW which is known to not work.

New committer japhy discussed his plans to port his Regexp::Parser to Perl 6, and make it understand Perl 6 Rules.

ninereasons revived cookbook and cleaned up strings docs in 01-00introduction.p6.

putter made time() into returning a fractional number. He and Juerd also worked on quickref/namespace.

Again, sorry for the omissions. :-)

Day 109: Coroutines, gather/take, symbolic deref, use pugs!

The Inline::Pugs hack inspired Ingy, so he implemented pugs.pm. To wit:

#!/usr/bin/perl
use pugs; # Here is some Perl 6 code...
sub postfix:<!> { [*] 1..$_ }
sub sum_factorial { [+] 0..$_! }
no pugs; # Here is some Perl 5 code...
print sum_factorial(3); # 21

A highly experimental implementation of coroutines has landed to Pugs, with semantics loosely borrowed from Perl 5's Coro::Cont:

coro flip_flop { yield 1; yield 0 }
print flip_flop() while 1;

# Output:
# 101010101010....

Here is a more interesting example, courtesy of integral:

my @generators = map -> $N { coro { yield $_ for 0..$N } } 0..3;
say "{ map { $_() }, @generators }" for 0..3;

# Output:
# 0 0 0 0
# 0 1 1 1
# 0 0 2 2
# 0 1 0 3

As the coroutine design in Perl 6 had not been finalised, this is merely an experimental feature for people to play with; suggestions are most welcome -- see my p6c post for details.

Limbic_Region asks whether gather/take has been implemented. I said no, then promptly corrected myself 15 minutes later by implementing it. :-)

The Haskell-fu is really strong with iblech nowadays:

<iblech> Yeah, I start to grok Haskell :)
<autrijus> iblech++ # demonstrating that the productivity is really not me, it's really Haskell :)

Among the usual influx of tests and work on the p6explain script, he also implemented:

• min, max and uniq as described by Damian.
• [.[]] and [.{}] for walking down a nested data structure by a list of keys.
• [or], [err] and the rest of logical reduction metaoperators
• Symbolic dereference: $::("foo").
• [,] -- with two lines of code.

Colin Paul Adams has successfully embedded Parrot in his XSLT processor, although he ran into some problems more-or-less the same with Pugs/Parrot embedding; however, this time around I have the committer bit, so with Leo's guidance, I've checked in an implementation for the missing string_to_cstring API.

Colin also asked on p6c about how to embed Pugs into the XSLT processor; I regretfully told him that Pugs does not yet handle Perl 5 programs, but gave a few pointers that are hopefully useful.

• mugwump and Stevan started porting Pugs to Perl 6, by translating its core types into its MetaModel counterparts.
• duff and Aankhen worked on quickref/rules.
• eric256 worked on examples demonstrating bitwise operators and .pick.
• nothingmuch continued his work on PA02, and interviewed me about closure support in Pugs internals.

There's much more, but I need to sleep (a lot) now. :-)