Pugs

YAPC-SA-2007 Hackathon

Here is the report on the 4 days of the YAPC-SA-2007 Hackathon, from April 12 to 15, 2007, in Porto Alegre, Brazil (during fisl8.0).

v6/docs
- added the KindaPerl6 diagram
- added a KindaPerl6 memory layout diagram - with help from #perl6
- added the MiniPerl6 talk (pdf)

MiniPerl6
- started the Perl6-in-Parrot target: desugars MiniPerl6 into p6parrot supported features
- grammar fix: semicolon is optional after a block
- perl5 backend: implemented warn()
- new: mp6-to-AST script
- nferraz started porting Test.pm to mp6

KindaPerl6
- memory management: BEGIN-block side effects are logged
- updated plan, discussed a possible release schedule
- fixed the container proxy algorithm (the implementation needs to be fixed; pugs has the same problem)

v6.pm
- fixed the "fish-operator" =<>
- fixed the quoted-word-operator <...>

Pugs::Compiler::Rule
- ratchet emitter fix, array interpolation now works

misc/pX
- nferraz and fglock started an adventure game implementation (with v6.pm), to be presented by nferraz at the Nordic Perl Workshop.

The network proxy at the conference did not allow commits to pass through, and there will be some delay until all the changes get to the repository.

Thanks to cmarcelo, david_fetter, eden_c, lorn, merlyn, and nferraz for 4 great days!

- Flavio S. Glock (fglock)

Off to Portland...

My plane to Portland is taking off in a few hours.  So, short recaps:

• fglock++ is hacking relentlessly on the next generation of Perl6-to-Perl5 emitter, incorporating type constraints and autoboxing into the mix.
• lanny++ fixed Perl6::Doc's Makefile.PL to work with nmake 1.5. This reminds me that we still need to upload it to CPAN separately as a replacement to the horribly outdated Perl6::Bible.
• andara++ continues to tune runpugs to avoid exhausting the dreaded resource limits. Help welcome, especially for some Ajaxy sugar to make it as sexy as tryruby...

Some short term post-release plans:

• Drop support for GHC 6.4.1, as soon as ghc-6.6 (with extralibs) makes it to Debian and FreeBSD.  The build system is haunted by workarounds for old semi-broken Haskell Cabal versions, and I'd be very happy to get rid of them.
• Integrate the MOH (Meta Objects in Haskell) codebase, a port of the MO (Meta Objects in Perl) framework currently being worked on by cmarcelo++and nothingmuch++, into Pugs itself so we can finally provide useful introspection like $obj.^methods.
• Rewrite the multi dispatcher to use new-style Capture/Signature objects. They are already available as literal expressions in 6.2.13 thanks to gaal++, but normal sub foo () {...} declarations need to be switched to use them as well.

However, let jag may take precedence as usual. We'll see how fast I recover this time... :-)

Pugs 6.2.13 preflight!

Greetings again after my nine-week hiatus.  Wow that's a really, really long time...

I've been very busy deploying Pugs/GHC/Jifty for $job, as well as helping other people putting Perl 6 to productive use (cf. TimToady++'s "I upgraded to GHC 6.6 and my program ran 60 times faster" comment), so there's been a general lack of time to write about things.

Partly due to $job requirements, I've delayed the release of 6.28.0 with its full metamodel integration.  Instead the 6.2.13 release, due next Tuesday, offers numerous real-world related improvements:

• Perl 5 embedding is now enabled by default, with support for function imports and two-way  passing of references and memory-leak-free unboxed values.
• Pugs's runtime performance is more than 2x faster, with even more gains when compiled with the newly released GHC 6.6.
• Pugs no longer depends on Parrot for its Perl 6 grammar engine.  Instead it comes with Pugs::Compiler::Rule, a pure-perl5 grammar engine natively supported via a sane embedding interface.
• Significantly simplified code reuse via support for mixin Roles, virtual attributes with initializers, as well as dynamic ( $obj.$meth ), quantified ( $obj.*meth ) and hyperized ( @objs>>.meth ) method calls.
• A wider range of new builtins made available through Perl 5 embedding, such as crypt(), fork() and "\c[UNICODE CHARACTER LITERALS]".
• Support for closure traits, including PRE/POST blocks for design-by-contract programming, and FIRST/NEXT/LAST blocks for fine-grained loop controls.
• Fully reentrant continuations! Thanks to that, the interactive shell environment is now much more robust.
• The multi-dispatcher now handles named, slurpy and optional arguments instead of only positional ones.
• Embedding data in programs through POD variables ($=DATA / @=DATA), as well as indentation-aware here-documents.

There is much, much more -- #perl6 is still working on the ChangeLog as of this writing.  Compared to 6.2.12, Pugs is definitely approaching the Hey, what was the big deal? stage now...

Meanwhile, please test out this preflight build, and see if it also passes 100% on your system with "make smoke".  We recommend getting a binary installer of GHC 6.6, but it should also work with GHC 6.4.1 and 6.4.2.

After the release, I'll take a month off $job to do some speaking abroad (at Intel, OOPSLA/DLS, Amazon, CONSILI/YAPC::SA) and hackathon'ing (on Jifty with obra/clkao/schwern, and v6-on-perl5 with fglock in Brazil).

So, expect more blogs (and talk slides) in the weeks to come, and do let us know if the preflight works on your system.  Have -Ofun! :-)

In Release Preparation Mode

The Pugs team is now in release mode. The focus is reviewing and addressing all the failing tests.

Reports on failing tests can be found by looking for a fresh report posted to smoke.pugscode.org , or by running "make smoke" yourself, and reviewing your own smoke.html.

The test may be failing for a number of different reasons:

• The test is broken for some reason and needs to be repaired. Maybe the spec changed. Who knows.
• New tests have been added for unimplemented features. These should have :todo<feature> added as the last argument. These kinds of of failures are more likely to occur at the end of the test script.
• There is a regression. Something that worked in the last release quit working. This should be fixed before the release, and should be left failing if you are unable to fix it yourself.
  

The other activity that's important during release mode is "changelogging"-- figuring out what's new and adding it to the ChangeLog. Some helpful people have already made good progress on this by reviewing the commit history.

Once this release is out, I expect we'll see some significant OO improvements and other larger scale changes land in the tree. 

Pugs Janitor Tasks.

During the YAPC::NA hackathon, chromatic++ had this great idea of a weekly-refreshed list of Janitor Tasks from Parrot and Pugs -- things that requires no internal knowledge to implement, can be done within a couple days, but nevertheless would have a significant positive impact on the project.  He plans to syndicate this list as part of the perl.com weekly newsletter; here is this week's list for Pugs:

• Go through examples/ and see if any of the examples fails to run; if so, seek advice at #perl6 to get them fixed.  Along the way, write regression tests for those examples -- currently only a handful of examples are tested under t/examples/.
• Harness Pugs's interactive shell with perl5's IPC::Run (or something else) and produce a web-based evalbot, similar to http://tryruby.hobix.com/.  Remember to set the PUGS_SAFEMODE environment variable to true -- use &Pugs::Safe::safe_getc, &Pugs::Safe::safe_readline and &Pugs::Safe::safe_readline for interaction with the user, if needed.
• Put the current http://pugscode.org/ homepage into the repository and merge it with the docs/feather/, and reorganize the homepage a bit to make e.g. downloads easier to locate.
• Look over test files in t/bugs/, especially the ones that has been fixed (i.e. without :todo in them), and move them into other more descriptive directories under t/.

If you are interested to help out, please apply for a commit bit on irc.freenode.net #perl6, or perl6-compilers. More suggestions welcome!

Diagram for Perl 6's CPAN stack.

With #perl6's help (in particular from putter++ and TimToady++), I have converted the layers from yesterday's bullet points into this diagram (click to zoom in -- thanks to Gordon McCreight for the thumbnail).

The Perl 5 Sugar layer is really key: it ensures that CPAN authors can use the relatively stable Perl 6 semantics without worrying about the more experimental Perl 6 syntax, and also ensures that v6.pm generates idiomatic Perl 5 code as output.

Each layer depends only on the layers below it. Everything above the XS layer will be pure-perl and hence very portable. The XS layer currently depends on some underdocumented areas of perl core (PL_check, for example), and we'd like to document them and bring them into the core regression test suite, so that they become fully supported in the future.

The diagram currently only lists existing modules, so there are a lot of "..." to be filled in each layer -- in particular we need sugar for all of semantic-layer modules -- but that's an exercise for tomorrow. :-)

Possible $job talk at OSCON.

So I've submitted a 45-minute talk proposal to OSCON'06 after all:

Programming the Kernel for Web 2.0

For a highly interactive web application to work well, the framework needs to manage user states, logic processes, and server resources intelligently.

This talk uses the Jifty kernel to demonstrate some interesting new concepts in this ever-expanding design space, including API reflection, client-side continuations, context switching, and mobile code.

We will also take approaches of several other web frameworks (Links, WaSH, Seaside, Rails, Catalyst, Springs, Cocoon, etc.), and express them as kernel modules on the Jifty system.

If accepted, that's going to be the most buzzword-compliant talk I've ever given. It will be a lot of fun preparing for the talk, though...

Conference Updates.

The YAPC::NA schedule has been announced. The "Perl 6" track of tuesday reads very nice, with talks from TimToady+dconway on the language, pmichaud on Parrot/Perl6, and gaal on Pugs internals.

Amusingly, without pre-coordination, the talk titles from merlyn+bdfoy, nothingmuch and me reads as a coherent whole: first "Learning Perl 6", then "Compiling Perl 6", then finally "Deploying Perl 6". :-)

Aside from the Perl 6 track, Tuesday also sees many talks on  #perl6-inspired technologies applicable to the CPAN world: Stevan on Moose.pm, the object model implementation for Perl6-on-perl5; Ingy on Module::Compile, the magic behind v6.pm; and gaal on the Pugs testing system -- you too can set up a smoke-server for your Perl 5 projects in no time.

Wednesday is #parrot-focused: we have talks on Parsing, Testing and Targetting Parrot, as well as Allison's talk on Punie that puts all of them into use to implement Perl 1. Woot!

So far 18 people has signed up for the post-conference hackathon, making it the largest Perl 6 hackathon to date. After July 1, several lambdacamels will relocate to Boston to continue hacking; if you'll be around Boston during July 2~6, feel free to leave a note in the Hackathon wiki page.

Prior to YAPC::NA, I'm also going to Nordic Perl Workshop in June 15/16; sadly I cannot stay there for a Hackathon, as on June 20 I'll visit Erik Meijer (known for LINQ, Parsec, HSP, HaskellDB, LtU rants, etc.) at Redmond, and give two talks there on Perl 6 and Pugs.

Erik is known for advocating static where possible and dynamic where needed, as well as for reconcilling multiple paradigms. His new ICFP'06 paper, Confessions of a used Programming Language salesman (getting the masses hooked on Haskell), is an excellent read; it outlines many of the same issues the Perl 6 design team is dealing with, on how to (in TimToady's words) provide -- but hide -- the fancy stuff. It's indeed very challenging, but also lots of fun! :-)

Licensing clarification.

(Two-sentence summary: I intend to relicense the Pugs internals code (i.e. the bin/pugs binary) to the MIT license, and only if fellow contributors are happy with that.  Everything else under the Pugs tree is unaffected, and the the "compilation-right disclaimer" in README does not affect any code committed in Pugs in any way.)

---

I apologize that I've not conveyed the scope of licensing to our contributors properly, causing a lot of confusion.  While I have been saying "only the src/ tree will be affected" from the start, I did not clarify that the src/ here explicitly means the src/ directory in the Pugs repository, containing Haskell/C/Perl6 code that builds the main pugs executable -- i.e. it's Pugs's internals code.

I'm really, really sorry to cause people misparsing it as the "Pugs source tree".  All other subdirectories, such as t/, examples/, perl5/ ext/, doc/ and util/, remains exactly in the same licensing situation (Artistic 2.0b5/GPL) as they were.  Please see the README file for the full details, as well as my public apology to chromatic.

So, to reiterate again:

• For Pugs contributors working on non-src/ directories, such as modules, tests, examples or documentations, nothing has changed in the licensing situation, and there is no plan at all to change them.
• For Pugs contributors who have checked in src/, I would like to survey if you are okay with releasing your previous and future work under the MIT license.  If you are explicitly not okay, I'd like to know why.
• chromatic has requested that his work be pulled out from the src/ directory, because he was unwilling to release it under SQLite's terms.  However, he later amended that he would be comfortable with the MIT license.
• As my intention with src/ was merely allowing maximum reuse, I'm certainly happy to change to the MIT license's terms; I have reflected this change in the Pugs README.
• The README does not change src/'s licensing currently; nothing will take effect until the next Pugs release, which will happen only after sufficient discussion time for src/ contributors to voice their opinions.

Aside from the src/ directory, I am uncomfortable with using Artistic license as whole-distribution compilation copyright license, because according to TPF, that may make it illegal to distribute Darren Duncan's Rosetta.pm module (GPL2+) along with Pugs, and I may have already been guilty for doing that for the past Pugs releases, just like Ruby people may be criminal for that with their regex.c (LGPL2+).

Because of this, I still wish to disclaim my compilation copyright, which basically means there is no global license that covers all of Pugs's subdirectory contents. This does not affect any contributed code in any way, and new commits in non-src/ trees are still under the Artistic2.0b5/GPL license by default, unless explicitly marked otherwise in that subdirectory or file.

Again, sorry for the confusion raised around this matter. I wish my English was more clear and succint...

On adopting the SQLite (non)license.

Aside from $job, I spent an entire day parsing the freshly announced release of Artistic License 2.0 beta 14, which feels much like Perl 6 -- in that it streamlined and formalized the mechanism, while staying true to the original idea.

As part of this, I engaged in an extremely long discussion with Allison, on her warning in #parrotsketch that inclusion of the Judy LGPL source tree in the Pugs repository will cause a licensing conflict with the Artistic license -- Inclusion of Artistic code under a LGPL project's tree is okay, but the converse is forbidden.

# Update: After some long followup discussions, it turns out it's not forbidden by law (Ruby does it with their regex.c), it's just considered extremely bad style by TPF; according to Allison, claiming the Artistic license applying to a compilation work also covers individual files in it, and it's legally dangerous -- though maybe not instantly criminal -- to have more strictly-licensed subparts in the repository tree, the way Ruby does it.

This has motivated me to move away from Pugs's current license, the now-outdated Artistic License 2.0 beta 5/GPL, as I don't want to restrict people doing anything with my compilation copyright.

All I really want to do is to let other people redistribute Pugs, in part or as a whole, without worrying anything about Pugs's license interferring with their redistribution policy. Conversely, I'd like to make use of third-party libraries without worrying about them disputing my copyright claims.

This is in resonance with the SQLite (non)license, which states:

The author disclaims copyright to this source code.  In place of
a legal notice, here is a blessing:

    May you do good and not evil.
    May you find forgiveness for yourself and forgive others.
    May you share freely, never taking more than you give.

Taking a page from this wonderful checkin from the extremely successful SQLite project, I'd like to begin the process on disclaiming my copyright over Pugs, as well as on my CPAN modules, perhaps registering this fact on the Creative Commons repository at the end of the process.

I'd like to take these steps:

• Move all third-party dependencies to a clearly marked contrib/ subtree.
• Disclaim my own copyright over the parts of the source files that makes up the pugs executable, adding to them per-file copyright-free declarations.
• Clearly mark that each aggregated modules under ext/ are under their own copyright owners and licenses.
• Implement the contributor policy, similar to the CC click-through screens, as part of the OSSF signup process on new commits for committers.

However, if people who have contributed to the src/ tree before feel uncomfortable about this change, that will necessitate a rewrite of those parts, so please voice your concerns.

Note that pretty much the same process needs to happen for Pugs relicense to Artistic 2, so this choice is simply that I think the SQLite non-licensing works better toward the goal of -Ofun. :-)