Pugs

How to Implement Perl 6 in '10

(Originally posted as a reply to chromatic++'s How to Implement Perl 6 in 10 Years on PerlMonks.)
(Also thanks to Limbic_Region++ for forwarding the original thread link to me. :-))

---

Hi. This is 唐鳳, a.k.a. Audrey Tang.

For the record, I think what chromatic wrote above contains a fair and accurate assessment to Pugs.hs (Perl6-on-Haskell).

However, please note that it elided over our collective shift of focus to the Perl 5 runtime during 2006 on #perl6, which resulted in the first CPAN releases of Moose, Module::Compile, v6-alpha (now evolved into Perlito), Pugs::Compiler::Rule, etc.

So while Pugs.hs was indeed suddenly unmaintained due to my '07 hepatitis outbreak, already by '06 we have redirected our main efforts into coding Perl6-on-CPAN.

Concretely speaking, that means we took various Pugs.hs structures (Signatures, MetaObject logic, Grammar parser, etc) and coded counterparts for the Perl5 runtime.

I'm very happy with what turned out - indeed MooseX::Declare and Perl 5.12.0 went way far beyond our original vision, in a very good way.

---

As for Pugs.hs, the 6.2.x series has already fulfilled its goals.

In order to code the 6.28.x series (compile-time gradual typing) without unreasonable pain, it required several significant changes in the host language (Haskell).

Some of them were codified into Haskell 2010 (then known as haskell-prime), such as PatternGuards; some were implemented in GHC, such as Type Families and Quasi-Quoting.

In addition to the language changes, a better theoretical understanding of GADTs (which was deep black magic when Pugs.hs 6.2.x first used them), of OO+Functional type inference (Martin Odersky et al), of sound STM semantics and gradual typing (Jeremy Siek et al), was also essential in coding the type system of Perl 6 as originally envisioned.

Also notable was basic groundworks for 6.28.x such as Parsec Transformers, Dynamic-linkable binaries and Data Parallelism (to name a few) has gradually materialized as of early 2010, so folks who'd like to tackle type systems now have a significantly easier compilation-environment support than even a year before.

---

However, speaking for myself, though Haskell became sufficiently attractive to implement compile-time type analysis for Perl 6, the success of Moose and Pluggable Keywords in Perl 5.12.0 has convinced me that we can also fruitfully implement such analysis directly in Perl 6, or in Perl6-flavoured CPAN modules, which is a much more straightforward way to amass a developer ecosystem than coding it in Haskell.

As lambdamoose demonstrated, real programmers can write Perl 6 and/or Haskell in any language, particularly if that language is as polymorphically existentially recursive as Perl 5. :-)

YAPC-SA-2007 Hackathon

Here is the report on the 4 days of the YAPC-SA-2007 Hackathon, from April 12 to 15, 2007, in Porto Alegre, Brazil (during fisl8.0).

v6/docs
- added the KindaPerl6 diagram
- added a KindaPerl6 memory layout diagram - with help from #perl6
- added the MiniPerl6 talk (pdf)

MiniPerl6
- started the Perl6-in-Parrot target: desugars MiniPerl6 into p6parrot supported features
- grammar fix: semicolon is optional after a block
- perl5 backend: implemented warn()
- new: mp6-to-AST script
- nferraz started porting Test.pm to mp6

KindaPerl6
- memory management: BEGIN-block side effects are logged
- updated plan, discussed a possible release schedule
- fixed the container proxy algorithm (the implementation needs to be fixed; pugs has the same problem)

v6.pm
- fixed the "fish-operator" =<>
- fixed the quoted-word-operator <...>

Pugs::Compiler::Rule
- ratchet emitter fix, array interpolation now works

misc/pX
- nferraz and fglock started an adventure game implementation (with v6.pm), to be presented by nferraz at the Nordic Perl Workshop.

The network proxy at the conference did not allow commits to pass through, and there will be some delay until all the changes get to the repository.

Thanks to cmarcelo, david_fetter, eden_c, lorn, merlyn, and nferraz for 4 great days!

- Flavio S. Glock (fglock)

MO bridge landing!

After 3 months of delay, it was quite amusing that the initial MO bridge took only 3 hours to land:

pugs> vv('hello, world').HOW
^Str
pugs> vv('hello, world').HOW.HOW
^Class

(The vv() above is a makeshift call that casts 6.2.0-land values into 6.28.0-land objects; it will go away once all existing built-in types are wrapped into MO.)

To understand what the ^Class notation means, let's take a brief look at Perl 6's object model, as implemented by Pugs 6.28.0's MO (Meta Object) subsystem.

In a traditional class-based object system such as Ruby's, the String class would be an instance of the Class class:

irb(main)> String.instance_of?(Class)
=> true
irb(main)> String.instance_of?(String)
=> false

Because of this, you can't call .new on a String instance, and you can't call .length on the String class:

irb(main)> String.new.new
NoMethodError: undefined method `new' for "":String
irb(main)> String.new.length
=> 0
irb(main)> String.length
NoMethodError: undefined method `length' for String:Class

Perl 5, on the other hand, takes an unique approach: When we say IO::String->new, the IO::String is not a Class object -- rather, it is a prototypical IO::String object that has no attributes!

IO::String->isa('Class');      
#=> false (perl5 doesn't have a built-in Class)
IO::String->isa('IO::String'); 
#=> true
IO::String->new->new;          
#=> IO::String=GLOB(0x18ab160)
IO::String->new->getpos;       
#=> 0
IO::String->getpos;            
Error: Can't use string ("IO::String") as a symbol ref...

The advantage of this arrangement is that several object systems -- prototype-based, closure-based, et cetera -- can exist simultaneously in the same program, without having to inherit from an universal Class class.

However, a bare literal "IO::String" is a terrible way to represent a prototypical object: it makes reflection needlessly difficult, and the error message for accessing the non-existing attribute slot ("Can't use string as a symbol ref...") seems to obey the principle of most surprise.

Perl 6's solution to this is simple: The prototypical string object, spelled ::Str or simply Str (if it's in scope), is a genuine Str instance.  However, any attempt to access its attributes raises a sensible exception.  Just as in Perl 5, so-called class methods such as .new are simply those methods that does not access instance attributes, and you can call them on both ::Str and regular Str instances.

Calling an object's WHAT method returns the prototypical object.  This replaces Perl 5's ugly ref($x) || $x idiom:

pugs> 'hello'.WHAT
::Str
pugs> Str.WHAT
::Str

On the other hand, because Perl 6's builtin objects are backed by a normal class-based dispatch system, you can reliably obtain a list of all Str's supported methods, by querying the Class instance that implements ::Str:

pugs> vv('hello').HOW
^Str
pugs> vv('hello').HOW.methods
["HOW","WHICH","bless","reverse"]

So there we have it: the Perl5ish prototypical object Str.WHAT is also spelled ::Str, and the Rubyish class object Str.HOW is also spelled ^Str.

The next step is to expose all MO's meta-objects (Role/Method/Class/Object) into Perl 6 land, and adapt our Perl 5 bridge to use Moose.pm, such that a class Foo {...} declaration in Pugs can generate both Haskell-side and Perl5-side representations, and work seamlessly with libraries on either side.  Stay tuned!

Various uses of embedded Perl 5.

For a long time now, Pugs embeds a Perl 5 runtime alongside its GHC runtime, providing seamless support for CPAN modules, including XS ones like DBI:

use perl5:DBI;
my $dbh = DBI.connect('dbi:SQLite:test.db');
$dbh.disconnect;

In addition to explicit use, there are several less well-known places where Pugs implicitly delegates to the Perl 5 runtime.

For example, today jisom++ asked on #perl6 about Pugs's counterpart to Perl 5's -T operator, i.e. testing if a file looks like plain text). Syntactically, filetest operators in Perl 6 are done via smartmatches against pair literals:

say "text file" if 'README'~~:T;
given 'README' { when :T { say "text file" } }

Semantically, because the heuristics for :T is rather hard to duplicate, we simply delegate to the Perl 5 runtime:

-- Pugs.Prim.FileTest:53
evalPerl5 ("sub { -" ++ testOp ++ " $_[0] }")


Another example is named Unicode charaters in double-quoted string literals:

say "\c[BLACK SMILING FACE]"; # ☻

Again, we simply delegate to charnames.pm in the Perl 5 runtime:

-- Pugs.Parser.Charnames:17
evalPerl5 ("use utf8; use charnames ':full'; ord(qq[\\N{"++name++"}])")

This is certainly a win -- the alternative way involves shipping the entire UnicodeData.txt with Pugs, which would be no fun at all.

The same principle applies to the crypt built-in function, which has this one-line implementation that simply calls the Perl 5 function with the same name:

-- Pugs.Prim:1083
op2 "crypt" = \x y -> opPerl5 "crypt" [x, y]

As moritz++ remarked on #perl6, such pass-through measures is a fine example of laziness, a virtue recognized by Perl folks and Haskell folks alike. Yay for laziness!

MiniPerl6 and use v5;

Today's hackathon with fglock++ focused on the design and implementation of MiniPerl6, an extended surface syntax language for native PIL nodes.  Oud rallying cry is one single Pugs codebase in Perl 6, as we seek to remove the massive code duplication in the current Haskell and Perl 5 runtimes.

As a strict subset of Perl 6, the design of MiniPerl6 removes some of the most complex parts of Perl 6:

• List context
• Inheritance
• BEGIN{...} blocks
• Runtime-evaluated type constraints
• Global and package-wide variables
  
• Named arguments to positional parameters
• Distinction between assignment and binding

On the other hand, it still retains many handy Perl 6 constructs that massively simplifies syntax tree construction and deconstruction:

• Closures and currying
• Class accessors and delegators
• Array, Hash, and Object unpacking
• Type annotations on variables and parameters
• Deconstructing pattern bindings with smartmatch and Signatures
• Temporary, hypothetical, state, and constant declarators
• Class accessors and delegators

To facilitate the migration, I've amended the relevant part of the Modules spec (S11), so now we can trivially inline Perl 5 code inside a Perl 6 programs, with use v5 at the beginning of a lexical block. Such blocks can nest arbitrarily deeply to switch between Perl versions:

    use v6-alpha;
    # ...some Perl 6 code...
    {
        use v5;
        # ...some Perl 5 code...
        {
            use v6-alpha;
            # ...more Perl 6 code...
        }
    }

And indeed, this now Just Works in Pugs:

    sub add_with_perl5 ($x, $y) {
        use v5;
        $x + $y; # This is  Perl 5 code
    }
    say add_with_perl5(21, 21); # 42

With this in place, we can bootstrap the self-hosted MiniPerl6 trivially, by reusing the existing v6.pm code and gradually take away "use v5" declarations in each block.

Because we carefully designed MiniPerl6 to emit to native Perl 5 that has no runtime overhead, that means whatever part bootstrapped in Perl6 -- e.g. the Grammar engine -- will actually gain performance over e.g. fglock++'s hand-coded PCR codebase, effectively make MiniPerl6 a DSL for Perl 5.

That's it for today.  Tomorrow cmarcelo++ will join us, and maybe MiniPerl6 will turn into a DSL for Haskell as well... :-)

Run Pugs: a Web terminal for Pugs

You can now play with the latest Pugs via a web terminal: http://run.pugscode.org/

The source code behind it lives in the Pugs repository as misc/runpugs/; more detailed information is available on its info page. Please read the TODO file if you'd like to help out.

Kudos  go to Why's Try Ruby, which inspired runpugs in the first place.   Have fun!

Interview: Perl 6 on Perl 5

Tonight I had chance interview Flávio Soibelmann Glock (aka: "fglock") about  the status of the 'v6', the project to get Perl 6 run on Perl 5, and eventually to have Perl 6 run on Perl 6. Read on for the interview.

Moose... it's the new camel.

Aside from a cameo appearance in my RHOX talk, Moose.pm saw a couple more interesting mentions in this year's OSCON.

The first is TimToday++'s State of the Onion talk.  He explained the now-slightly-less-imaginary Perl 6 Timeline, as well as the v6.pm stack, then went on to say that Moose.pm is his favorite piece in the 6-on-5 puzzle.

The next day, Tim Bunce discussed the plan toward DBI 2.0 as a lightning talk.  It's very exciting to hear that the new version of DBI will clearly be based on Moose, in particular using its Role support to offer a better interface.  Yay for the lambdamoose! :-)

v6.pm: 920 tests pass!

Some power of Mystery dragged Chia-Liang Kao, maker for our version control system svk, into hacking v6.pm over the weekend with fglock.  See his use.perl journal post for a quick walkthrough into v6.pm hacking!

XML::Literal 0.01.

As part of today's work on my OSCON slides,  I've put XML::Literal 0.01 on CPAN:

# This is not a source filter: it just augments glob().
use XML::Simple;
use XML::Literal \&XMLin;

# Simple element
my $xml1 = <hr/>;

# With variable interpolation
my $xml2 = <input value='$ARGV[0]' />;

# With an extra pair of angle brackets
my $xml3 = < <a href='/'> Some Text </a> >;

# With escaped angle brackets 
my $xml4 = <a href='/'\> Some Text \</a>;

# Direct call to the xml-building glob'' constructor
my $xml5 = glob'
    <p><em>
        Some Text
    </em></p>
';

# This does not look like XML, so it's still shell glob
my @files = <*.*>;

Of course, having it simply construct hashes via XML::Simple is not  very interesting. More fun is to use it to construct XML patterns and composable, self-validating fragments -- more (ab)uses are left as exercise for the reader. :-)