Pugs

Day 28: 6.0.9.

I am happy to announce that on the final day of February, Pugs 6.0.9 has been released! Compared to 6.0.8 of one week ago, we have made these improvements:

• First Perl 6 module: Test.pm.
• First Pugs Apocryphon: docs/01Overview.html.
• More than 600 unit tests, with a comprehensive coverage.
• Relicensed under GPLv2 and Artistic2.0b5.
• New flags: --version, -v, -c.
• New examples: examples/fp.p6, examples/shuffle.p6.
• New syntaxes: ?? ::, our(), chained =>, END{}, =begin END, unless.
• New primitives: defined(), unlink(), ref(), join(), require(), int()
• New magicals: $?SUBNAME, %*ENV, &?BLOCK, &?CALLER_CONTINUATION, @*INC.
• New literals: \d1234, 0d1234, qq[], qw[].
• Interpolation in double-quoted strings.
• Hashes and subscripting.
• Bare blocks now assumes outer scope's lexical pad.
• Bare blocks in statement level is now always executed.
• Closures closes properly; errors are propagated upwards.
• A first draft of "Kwid" documentation format.
• A first draft of builtins declarations.

Special thanks goes to amahabal and stevan for tracking down some last-minute bugs and helped getting the unit tests into a releasable shape. Also, much thanks to all of #perl6 and perl6-compiler list for making it possible. As I said to obra today on #perl6:

<obra> is the project actually still accelerating?
<autrijus> I think so, yeah.
<autrijus> I'm not accelerating, but the project seems to be.

Looking forward to 6.0.10, I am aiming to remove all bugs covered in todo() test, which will require putting Rule.hs under Eval monad, as promised before. I'm also glad that Isaac Jones (SyntaxNinja), the equivalent of Andreas Koenig in the Haskell world, is now hacking pugs and has sent me lots of useful cleanups that fixes both the style and subtle uncovered cases in the program logic. And it also makes "ghc -Wall" happier with Pugs, so it's all good.

Also, I'm happy that Pugs is continuing to provoke Perl 6 discussions on p6l, PerlMonks, and no doubt other places as well. See you tomorrow! :)

Day 27: Dialup.

Sadly I did not manage to release Pugs 6.0.9 (or helped getting SVK 0.30 out), chiefly because my backup battery was not with me, and the room I'm currently staying in only has ~14.4k dialup. So, while it's good to get things done, it's very bad at getting things pushed out. So I'll have to wait till tomorrow morning, and hope for wireless access during Debian Miniconf.

So, at the code front today, there's still many things happened:

• I have added a -c flag for dumping ASTs, to better illustrate what AST is in this PerlMonks thread. Volunteers welcome to hack Pretty.hs to actually use Text.PrettyPrint -- it would be a good exercise for Haskell beginners.
• Stevan committed examples/fp.p6 to demonstrate functional programming style using Perl6 subs. Amazingly, it just works. :)
• I've changed Makefile.PL so it now aborts under Cygwin; GHC (and currently Pugs) can only works with MinGW Perl, VC++ Perl (e.g. ActivePerl), but not Cygwin Perl.
• The parsing bug of <> globbing everything is fixed. It now requires a leading dot, otherwise allowing no whitespace. This prevents 2 < 3 > -1 from being parsed as hash subscripting.
• Chained comparison is resurrected after luqui's change of making binary operators takes two invocants (instead of two parameters).
• String interpolation now Really Works.
• Finally, all tests now passes for Pugs because I've either implemented or TODOified the failing things.

Now we have a tree in a releasable shape; all I need now is some bandwidth tomorrow... :-)

Oh, and kudos for viirya and scw for translating PA01 into Chinese. Translators for other languages are also be very welcome!

Day 26: Preflight, Builtins, Kwid

Today I was caught up in $day_job and various preflight errands, which left me little time for Pugs and SVK release engineering. However, smpeters has been quietly doing much of the releng work for me, so I still hope to get it released tomorrow noon, perhaps when I'm waiting for boarding in the CKS airport.

Ingy graced us with his Kwid format, which is intended to be a drop-in replacement for POD, using Wiki syntax. When I was preparing PA01, the lack of labelled hyperlinks prevented me from turning it into POD, so Kwid is a welcome alternative solution.

Building on his work of yesterday, luqui finished double-quoted string interpolation, so "Hello, $world" now works, as does closure interpolation: "2 + 2 = { 2+2 }".

Also, building on Patrick's previous work, luqui has started writing out complete signatures for all primitives implemented in Pugs. We will eventually switch to generate the whole primitive/precedence table from parse that file, so volunteers are welcome to keep that list complete, clean and sane!

Finally, during a discussion the pattern matching and pattern guards features of Haskell, wolverian and me both think it will be cool to have something like that in Perl 6, maybe building on the smart matching ~~ operator. I encouraged wolverian to summarize those ideas into a p6l post; let's hope it will provoke some constructive discussions.

Oh, and we have a channel log page now; much thanks to feb's help!

Day 25: Apocryphon 1!

Today I completed the first installment of the Pugs Apocrypha -- Pugs Apocryphon 1: Overview of the Pugs project. If you are still wondering what Pugs was about, please read that document first. :-) Many thanks to people who have contributed questions; I have postponed the more technical ones to the next Apocryphon.

In this Apocryphon, I have stated my decision of not submitting a TPF grant for Pugs. Although Ingy mentioned some psychological factors, the real reason is that I think it would be more helpful to donate time to the Perl 6 project by hacking Pugs, rather than asking TPF for money to do the same thing.

So with Apocryphon out, I will finally get some time to hack Pugs code during my stay in Beijing for AOSS, between Feb 27th and Mar 6th. During that week, I'll also talk with perlchina folks about hcchien's initiative to translate Perl 6 documnents into various languages, as well as try to get more people to try out Pugs.

In other news, we have two more committers on board. viirya was the one who sent the first patch to pugs; after 3 days of work, he finally tracked down the hideously silly bug that caused undef(0) to be treated as nullary, and we promptly fixed that problem. Much kudo for his persistence.

Another committer is Luke Palmer (luqui), who threatened to implement Rules in Parsec, so he can use it to parse Perl 6 and rewrite Pugs in Perl 6. On his first day into learning Haskell, he made good on his threat by implementing qq[] with matching brackets, string interpolation and hash subscripting in the Pugs parser. Wow.

Stevan, bsmith and duff continues to polish the test suite, bringing the total number of tests up to an impressive 437 in a couple days. nnunley committed a particuarly nifty send+more=money solver into the examples/ directory, showcasing the use of junctions in Perl 6. It doesn't run yet, but will be soon.

Jesse Vincent (obra) plays the junior Perl 6 manager role pretty well: at his suggesion, I have moved tests for &?CALLER_CONTINUATION, a speculative implementation of escape continuations not found in Synopses, into a new t/unspecced/ directory. The idea is everything else in t/ and Perl6/lib/ can then be trivially shared between different Perl 6 implementations.

As I'd like to release 6.0.9 before I fly to Beijing, tomorrow I'll try to get all examples/ running, implement multidimensional (and hopefully sliced) array assignment needed for is_deeply(), and cut a release. Let's hope it go well. :-)

Oh, and I'm thrilled that ziggy is finding a use for Parsec on parsing domain-specific languages. It's nice to see Haskell improving camelfolk's lives. It's not only one direction, though. #perl6 is already talking about improving GHC's tool chain by writing a code coverage tool, based on pjcj's excellent Devel::Cover module, so we can identify untested conders in Pugs. 'Tsall good. :-)

Day 24: An amazing team

Today there has been 32 commits in the repository, but only 5 were mine. bsmith, Stevan, smpeters, along with two new committers nnunley and duff, has implemented many things among themselves:

• Test.pm receives skip(), plan(), fail(), todo_fail() and pass(); those functions were subsequently refactored to avoid duplicate code.
• Lots of unit tests has been converted to use Test.pm.
• James Mastros submitted a helpful patch to show svn patchlevel as part of Pugs version string.
• New and improved tests for various language features, including multidimensional arrays, push() and more.
• New examples: fisher-yates shuffle and send+more=money solver.
• Support for while/until loops.
• The int() primitive.
• Imported synopses as t/Synopsis/* as a beginning to POD-based acceptance tests.

In other news, Pugs was mentioned on The Haskell Sequence today. Indeed, I have noted that a significant part of questions asked in #haskell are from camelfolks. Conversely, we saw a large influx from lambdafolks to #perl6 as well. Lots of knowledge transfer is happening, which makes me really happy.

So, I have finally sorted out most of the non-technical questions from the large question list, and produced a draft version of Pugs Apocryphon 1. It needs more work (especially on the two paragraphs about Apocalypses and Synopses), but it is a start.

I'll merge the mess that is the technical-oriented questions tomorrow. Please Feel free to submit more questions, or comment on the draft above. Enjoy!

Day 23: Test.pm flies!

Today is the most productive day since Day 1... But you already know that. :-) Seriously, we reached a breakthrough today, because the first genuine Perl 6 module, Test.pm, now works correctly, and we are converting existing unit tests to use it. It should make writing unit tests much easier in the future.

In other news, we have three new committers today, bringing the count up to 10. Together we have hacked in these features:

• Closure are now closed under their creation site, not caller's site. This has the effect of making "my $x; sub { eval $_ }" always see the $x in the lexical scope, which differs from Perl5's treatment (which only sees $x if it is explicitly mentioned in the eval string). It may require a clarification, but I like the Pugs approach.
• Errors in assignments are now trapped; in general, the error propagation logic is now straightforward.
• push() and unshift() implemented.
• END block, @*INC, and ported %*ENV on win32.
• Lots and lots of tests for things resolved in the last few days.
• The very exciting "require" primitive.
• Parsing for pointy subroutines no longer conflicts with unary negation. Similarily, binary != is no longer blocked by unary !.
• undef in list context won't flatten into a null list now.

Oh, and finally, I have relicensed Pugs under the Artistic 2.0b5 license, as specified in RFC 346. It is still dual licensed under GPLv2, too.

That's it for today. See you tomorrow!

Day 22: Many hackings

I think I'll cease to say "today is the most productive day since Day 1", because today is, again, the most productive day ever. ;-)

Many thanks for people who've sent me questions via various channels; the Perl Monks thread is especially helpful; in it sleepingsquirrel pointed out a wonderful technique that can generate a compiler for free with quasiquoting, while reusing all current interpreter infrastructures. Now I'd need to read "targetting parrot" in the near future...

Also, in #perl6 we have solved many problems in interpreting Synopses, namely:

• The effect of "return" in parameter defaults has been determined; it returns to the caller and defeats autothreading.
• Literal syntax has been clarified as 0d1234, 0o123, \d1234, \o123, etc.
• Signature of a hypothetical call_cc specified as &code<Code>.
• Bare block as the last statement in a subroutine is always evaluated.
• CALLER:: sees global variables in caller's scope too.

There is nearly 30 checkins in the repository today. Some highlights include:

• eval and reduce are merged into a single pass that always return a Val.
• Parsec imported as Rule.hs; work has begun to integrate inside the Eval monad.
• Steve Peter provided a test case for right-associate => that creates linked lists; it's promptly implemented.
• CALLER:: in toplevel is now caught as an error. Also it sees global variables as specified above.
• &BLOCK and &CALLER_CONTINUATION (escape-only) implemented.
• An optional dot is allowed in function application syntax.
• ingy added subversion revision info to the banner.
• rgs added read-only %ENV support to Posix platforms.
• Various things (notably Code) are numified as NaN now; this is pending discussion.

See you tomorrow!

Day 21: Your questions wanted!

Most of my time today has been spent on writing Apocrypha 1, "Overview of the Pugs project". As I plan to write it as Catechism (i.e. Q&A format), I solicited #perl6 for questions. Here is a list of unsorted questions I compiled:

• What is Pugs?
• Where can I download Pugs?
• Why isn't Pugs called Perl6?
• Is Perl6 fully specified now?
• Is Pugs's goal to be a fully fledged perl6 compiler or interpreter?
• Does Pugs run on every platform perl runs on?
• Who owns Pugs?
• When will Pugs be finished?
• Is Pugs compiled or interpreted?
• Will Autrijus teach haskell for other perl6 developers?
• I want to learn Perl6. What texts should I read?
• I want to learn Haskell. What texts should I read?
• Is Haskell portable?
• Why do you use both darcs and svk?
• Is Pugs a perl foundation project?
• Will Pugs die when the official perl6 is done?
• Will Autrijus unexpectly abandon the project?
• Is there a Cabal?
• How can I help?
• Will you accept my patch?
• Do I have to know Haskell to hack Pugs?
• Will Pugs always be in haskell?
• Where can i learn haskell?
• How does Pugs relate to parrot ?
• Aren't Haskell and Perl incompatible by nature?
• Can Pugs run parrot code?
• Will CPAN work on Pugs?
• Can Pugs run perl5 code?
• What does the Pugs name come from?
• Are there other real programs out there written in haskell ?
• What's featherweight Perl6?
• Why did Autrijus go beyond larry's back to create Pugs?
• What's the goal of Pugs?
• Why not just use Ruby? it's perl6 now.
• Is Pugs a *schism* in the Perl community?
• Since you did it in 3 weeks, now we don't need to pay the perl foundation half a million dollars to create perl6, right?
• Will you implement a DynaLoader equivalent for Pugs?
• does Pugs run under .Net and Java?
• Can Pugs run all the code in Apocalypse, Exgesis and Synopses?
• Does lwall approve of Pugs?
• This all sounds very exciting. what drugs are you on ?
• Will Pugs compile and interpret haskell code to run in parrot vm?
• Perl5 is bloated, untyped, fragile, slow and obfuscated; why should I spend time in hacking Perl6?
• The parrot/perl6 program hasn't made perl6 happen in 5 years. What makes you think you can do it?
• Perl5 is wonderful. Why should I care about perl6?
• If I have a .pl6 file, will Pugs run it?
• Can I write CGI code with Pugs?
• Is there a mod_pugs?
• I replaced /usr/bin/perl with Pugs, but now my perl5 programs don't run. why not?
• Where do i report bugs in Pugs to?
• I filed a bug against Pugs via perlbug, but they said i had to report it somewhere else. where is that?
• I have written some Pugs modules. Where should I upload it?
• Why doesn't Pugs ship with CPAN or CPANPLUS?
• Is there an Inline::Perl6 or Inline::Pugs?
• Is Pugs free software?
• If I use Pugs, do i have to share my source code with everyone?
• Is Pugs licensed under the same terms as perl itself?
• What is your worst/best (haskell) hack in Pugs?
• What is a "golfing system"?
• Who is Autrijus anyway?
• How many people are actually behind the autrijus pseudonym?
• Were animals harmed during the making of Pugs?
• Is there a RSS feed for Pugs news?
• What's with those Tolkien poetry in Pugs source code files?
• Is there a commitlog email list for Pugs?
• What is a monad, and what's in the Eval monad?
• If I enter ICFP 2005 with Perl6 and Pugs, what is my chance of winning?

I need more questions of all sorts, so you can help. Please leave a comment here that contain any number of questions that you'd like to see answered. Thanks! :-)

...now, to the journal itself...

Today #perl6 is busy resolving edge cases and inconsistencies in the typing system. I'm truly happy that my two outstanding question on p6l (IType vs VType and Context paradox in multisub dispatching) has met with suitable solutions.

Of particular importance is that, with help from luqui (Luke Palmer), pmichaud (Patrick Michaud) and lathos (Simon Cozens), we have got the to bottom of the "punning" problem of ambiguous of "Array" as a implementation type, and "Array" as a value type. The solution in Pugs is simply to excise all implementation types, except for "Variable" and "Constant". We will see more of implementation types when we get to the Tieable class.

Also of note is that the Parser will begin to do typechecking and inferencing, and ill-typed multisub dispatches will be caught at compile time, instead of delaying until runtime paradox appears.

On the code front, hcchien and juerd continue adding new tests and removing TODO tags from tests that now works, while ycheng implemented unlink(). I've implemented defined() and undef() (both nullary and unary variants) at Juerd's request, and ref() at Steve_p's.

Oh! And I'm happy that http://dev.perl.org/perl6/ links to http://pugscode.org/ now. Wonderful. :)

Day 20: 6.0.8

Today we've hit 170 unit tests. Only three days ago, Pugs were at revision 62; now it is at revision 140 and counting. Despite me wasting an entire day reinstalling my hard disk, our trusty team at #perl6 still finished many things:

• Many Posix-based IO primitives.
• Hashes, Pairs and their access methods.
• Prefix ++ and --; context for postfix ++ and -- also fixed.
• The CALLER:: namespace.
• Detection for deep recursion.
• The magical Y combinator, &?SUB.

For a demonstration of the last point, see the code below; isn't it just beautiful? :-)

pugs> sub { $_ ?? $_ * &?SUB( $_ - 1 ) :: 1 }.(10).say
3628800

So, with this, Pugs 6.0.8 has been released. The next big goal is to partially rewrite Parsec to use IORef in our ReaderT, put it into the Eval monad, and call it Rule.hs -- that is the easiest way to get BEGIN {} blocks to work.

Also I've started planning on a serializing interface for the ContT continuations in the Eval monad; more details later.

Enjoy!

Day 19: A team!

First, the bad news: A couple hours ago, while I'm busy hacking the infix "Y" (zipping) operator, there was a muffled "boom" inside my laptop's hard disk; then the disk is spinning like mad, and the screen is overflown with bad sector messages. Next thing I know, I'm left with a thoroughly broken disk that is sprinkled with some sort of white powder stuff, and nearly all sectors are unreadable. Sigh. So I'm stuck with this horribly slow iBook G3, at least until I can get the hard disk repaired.

But now, the really, really good news. The Pugs Development Team has 7 committers now -- alinbsp, bestian, hcchien, ingy, juerd, ycheng and yours truly -- and the speed of code churning has more than doubled. Wow.

Pugs also has its own site pugscode.org now, after the fashion of poniecode and parrotcode, hosted in a machine sponsored by xs4all. We also started a new IRC channel, irc.freenode.net #perl6, to coordinate daily (or, more likely, hourly) development tasks. All is welcome to join!

At the code front, lots happened today:

• Hash support has landed. There's a new caster that respect LValue context, and converts between List/Array/Hash/Scalar/Undef in a perlish fashion. Unfortunately it broke the slurpy scalar multisub in quicksort.p6; I'll fix it tomorrow.
• The ever-helpful viirya prompted me to check in the dynamic parser code that puts parseOp into the State monad -- that is, it dynamically rebuilds its unary/list prefix function table, and distinguish between their precedence levels. It also clears the path for user-defined precedence and associativities.
• "make test" on platforms with older perls should now work, thanks to clkao's prodding. The fun thing is that perl runs a pugs -le 'print join qq[ ], @INC' before the actual test, and the solution is to simply implement join, qq and @INC in pugs. ;)
• Also fixed is the often-complained-about ReadLine dependency. It's now lifted unless you have Term::ReadLine::Gnu installed on your system. ingy also checked in fixes to make "make clean" happy.
• tbone suggested that we change the abstract in META.yml from "Perl6 User's Golfing system" to "A Perl6 Interpreter" to reduce confusion, which I promptly did.
• ycheng made the IO operations return false on error; he also added "rename", "chdir", "sleep" and fixed up "close", "open", "mkdir" and "rmdir".
• Some small primitive and constructs like "join", "split" and "unless".
• Juerd begins committing various real-worldish tests, as well as a "Dialect" test that will exercise the macro facility in the future.
• I have begun rework the Parsec Expr parser builder, to support nested unary prefix like "!!! 1", which now works, much thanks to Juerd's testing.

So, there you have it: A team, an IRC channel, and a homepage in one day. Can't ask for more. :-)