The original focus was about interoperability between YAML::Syck, YAML.pm, and Pugs's Syck support.
We thought that we'd need a shared, specification-based test suite for them. So we started porting Test::Base to Perl 6, starting with this Perl 6 rules grammar of the Test::Base specification format. (If you havn't been exposed to Test::Base before, Ingy's YAPC::NA 2005 talk is a good start. (To navigate to the next page, press the Enter key.))
Then, because Ingy is learning Haskell and trying out parser technologies, we looked into how to combine top-down rules parsing with bottom-up tokens parsing, which is (as of this writing) an underspecified part of the Subroutine spec (look for "tokens").
As we are discussing this push/pull style of parsing, we realized there's an ideal use case: the Kwid format, with its no-backtracking, no-syntax-errors-possible, cross-level-end-tags (think HTML's <b><i><u>...</b>) characteristics, should be the first test case.
Plus, there's this whole Perldoc project from Pugs's inception -- Ingy lured the lambdacamels into writing our changelog and much of the module documentations in Kwid, not to mention the Kwid spec, which was also written in Kwid.
So we set out to work. Ingy has picked up Haskell from the excellent Haskell tutorial for C programmers, so we paired up a prototype in Haskell as Text.Parser.Kwid in the Pugs tree pretty quickly. The prototype uncovered many corner cases in the algorithm, so we kept simplifying the rules until they agreed with each other.
However, having it available in Haskell is not going to help Test::Base (and other Perl 5 modules) to switch from POD to Kwid; what we need is a seamless way of turning Kwid into Pod/HTML/Manpage, plus the neccessary Module::Install hooks to make this transformation an automagic part of make dist.
Fortunately, during the Pugs Hackathon in Toronto, Ingy had already designed how this system should work -- see his Perldoc talk in YAPC::NA 2005 for details. Therefore we need to port this Kwid parser to Perl 5, and have it generate Perldoc-compatible events to hook into HTML/Pod/Manpage emitters.
Fortunately, Perl 5 proved to be a fine functional programming language: the port took me two hours, carrying over mutual recursive structures, lazy instantiation, thunks, closures, et cetera. (Readers of Higher Order Perl may find them familiar.) The only thing missing is type safety, but it's not really a concern when it's been typechecked in another language already.
After Ingy woke up, he took my decidedly non-OO code, put an OO shim around it, and released the new Perldoc.pm framework. The bundled perl-doc command line utility can convert Kwid to HTML now -- something I've longed for for many months. (Prior to this I had to use the very fragile, not-released-to-CPAN, unmaintained Spork::Formatter::Kwid and extract the HTML from Spork slides. Ow.)
With Kwid and Perldoc back in an usable state, Ingy is looking into filling in PPI's longstanding gap of a subparser for =directive ... =cut segments. Again, read the presentation for the wonderful things that can follow from this path.
Now... Where were we anyway? :-) It's been a lot of fun, surely, but this yak is much bigger than I anticipated...
Recent Comments