SMP parallelization comes to Pugs!

The Intel talk went well.  Met several interesting hackers working on Harmony, STM, program transformation, and other cool projects.

During the follow-up discussion, a few similar languages were brought up: Steele et al's Fortress, the recently open-sourced Strongtalk, and of course Scala. Perl 6 compares rather favorably with them in many areas, although there's always more interesting ideas (case classes, monad comprehensions, etc) that we can learn from.  Pugs's dire need of JSR292 for targeting JVM was also discussed in some detail.

In explaining Perl 6's approach to SIMD parallelism in the form of  hyper operators and junctions, one of the Haskell hackers there reminded me of GHC 6.6's shiny new support for SMP parallelism, where you can say "env GHCRTS=-N2 ./pugs ..." and have Pugs allocate two OS threads for those hyper/junction operations, which will (in theory) make them automagically  twice as fast.

So I hacked it into Pugs right away; it's just a ten-line change! As usual, it revealed some hidden corners of the language.  (For example, $x++ should be implicitly atomically { $x++ }.)  The preliminary results are quite impressive on feather, the main Linux machine for the Perl 6 development community:

$ time env GHCRTS=-N1  ./pugs -e 'my @x = 1..50000; @x.>>sqrt'
real    0m16.623s
user    0m12.729s
sys     0m0.212s
$ time env GHCRTS=-N2  ./pugs -e 'my @x = 1..50000; @x.>>sqrt'
real    0m13.605s
user    0m12.045s
sys     0m0.320s
$ time env GHCRTS=-N3  ./pugs -e 'my @x = 1..50000; @x.>>sqrt'
real    0m9.438s
user    0m8.293s
sys     0m0.308s

It's not quite linear scalability yet, and the Channel-based hyperiser is not yet optimized, but it's quite encouraging already.  Once the recent flurry of work on running parallel Haskell on GPUs come to fruition, the numbers may become even more impressive...

Also, thanks to fglock++'s recent work on v6.pm's new emitter, I was able to explain the concept of contexts clearly for the first time:

"In Perl 6, we think of a function as compiled in three different ways, depending on the kind of evaluation strategy you'd like to use: either you use it just for the side effects, or you'd like an lazily evaluated stream, or you'd like an eagerly evaluated value."

While it's also a Perl 5 legacy which had nothing to do with evaluation strategies, it's nice that the concept has evolved into something much more attractive in Perl 6. :-)

Comments

[kasei]

Do you have any pointers for the cited "Haskell on GPUs" work?

[Audrey Tang]

Kasei: http://www.cse.unsw.edu.au/~pls/members.html and http://conal.net/Vertigo/ are two projects mentioned in #ghc.

[Nicholas Clark]

Nice.

Just out of curiousity, which 10 lines? Can you provide a link to a commit mail diff, or a svn viewer? Or the revision number(s) so that one can diff it with one's own svn client.

[Audrey Tang]

Nicholas: I've just posted another entry to answer your question. :-)