Planet Perl Six

perl6.announce: Parrot 5.4.0 "Austin Parrot" Released! by Bruce Gray

Wed, 22 May 2013 18:51:15 +0000

Jimi Hendrix, deceased, drugs.
Janis Joplin, deceased, alcohol.
Mama Cass, deceased, ham sandwich.
-- Austin Powers (making a list of friends from the Summer of Love)

On behalf of the Parrot team, I'm proud to announce Parrot 5.4.0, also known
as "Austin Parrot". Parrot (http://parrot.org/) is a virtual machine aimed
at running all dynamic languages, and currently focusing on Perl 6.

Parrot 5.4.0 is available on Parrot's FTP site
(ftp://ftp.parrot.org/pub/parrot/releases/devel/5.4.0/), or by following the
download instructions at http://parrot.org/download. For those who would like
to develop on Parrot, or help develop Parrot itself, we recommend using Git to
retrieve the source code to get the latest and best Parrot code.

Parrot 5.4.0 News:
- Core
+ Implemented the coth() and acot() math functions.
+ Fixed chomp to only trim a newline when it ends the string. [GH #958]
+ Added readlink() and Parrot_file_readlink(), with tests. [GH #967]
- Build
+ Parrot now detects the CPU model on Linux systems, as well as
detecting more CPU models on BSD, Cygwin, Solaris, Win32, and Darwin.
ARM v7 is also now recognized. [GH #962]
- Documentation
+ Threads examples now have proper POD sections and useful descriptions
with links to references.
+ Added main description for Task PMC.
+ Added descriptions to trig methods in Float PMC.
- Tests
+ Added improved test coverage targets "cover_new" and "fullcover_new".
+ Improved tests for acot(), coth(), acot() math functions.
+ Added tests for options passed to debugger.
+ Updated native PBC test files for string, number, and integer,
which resolved 11 TODOs in the test suite. [GH #959]
+ Fixed test for the auto/arch config step.
- Release process
+ Added message digests to crow.pir.
+ Added in release.json: "release.type" can be "devel" or "supported".
+ Refactored common code to sub in auto_release.pl.
- Community
+ Parrot is part of the Hackathon at YAPC::NA::2013, in Austin, TX, USA!
http://www.yapcna.org/yn2013/wiki?node=Hackathons
+ Parrot has been accepted to Google Summer of Code 2013!
+ Currently there are two high-quality proposals being worked on:
https://gist.github.com/sa1/5468408 - parrot-libgit2
https://gist.github.com/denisboyun/5472762 - App::Parrot::Create

The SHA256 message digests for the downloadable tarballs are:
4e37686911b446f5e5f2c0aa62138988ba0c411d2c5e2ba231d1a3421a85ad10 parrot-5.4.0.tar.gz
91d0e46fe3ef08e692e80756f26ee0e7311fe58e49d6c31f3f5180d4eb475696 parrot-5.4.0.tar.bz2

Many thanks to all our contributors for making this possible, and our sponsors
for supporting this project. Our next scheduled release is 18 Jun 2013.

Enjoy!

Konrad Borowski: Perl 6 changes - 2013W20

glitchmr@myopera.com (Konrad Borowski) — Sat, 18 May 2013 07:00:00 +0000

This week, the most notable change would be Rakudo sort of working on Java Virtual Machine. It still needs lots of improvements, but it can already run “Hello, world” program, what is important milestone in Rakudo for JVM.

id='incompatible_changes'>Incompatible changes>

Perl 6 specification>

You cannot create a set or bag from list of pairs or hash using a constructor.

The `bag {a => 2}` was ambiguous, because it could mean either `bag` containing `'a'` with value `2`, or `bag` containg `{a => 2}` with value `1`.

You’re expected to use `.Bag` or `.Set` instead, if you mean that.

STD.pm6>

You cannot use `set` or `bag` to mean empty set or bag.

If you really meant that, you are expected to use `set()` or `bag()` instead.

id='new_features'>New features>

Perl 6 specification>

Slice adverbs support false booleans.

You can write code like `%a :!exists` to check whatever the element doesn’t exist.

You can also write code like `%a :!delete` which does nothing, but can be used for conditionally deleting hash keys (for example `%a :delete(%a<c> :!exists)`).

Sets and bags use `===` semantics.

Previously the specification hasn’t said which semantics should be used. As anything other than `===` semantics would be slow, `===` semantics are used.

Defined `INIT`/`END` across modules.

Added `FINAL` and `COMPOSE` phases.

`FINAL` is ran at link time. `COMPOSE` is ran when a role is composed into a class.

STD.pm6>

`set<a b c>` has special error message.

This syntax wasn’t useful for anything, except for perhaps obfuscation. It meant creating empty set, and getting keys `'a'`, `'b'`, and `'c'` from it. As the empty set doesn’t have these keys, it was returning `False` 3 times.

As the behavior doesn’t make any sense, you probably wanted to use `set <a b c>` instead. The old behavior is still accessible using `set()<a b c>`, but I don’t think you would want to use this.

Perl 5 grammar allows version in `require` statement.

In Perl 5, you can write code like `require 5.010` do make runtime version check (as opposed to `use 5.010` which is compile time check). The Perl 5 grammar, included as part of `STD.pm6` now supports this syntax.

`FINAL` and `COMPOSE` phases is now recognized.

Now that these phases are in specification, the grammar was modified to recognize them.

Rakudo Perl>

JVM version of Rakudo supports simplest “Hello, world!” program.

Fixed infinite recursion in `IO::Spec::Unix.rel2abs`.

`rel2abs` made infinite recursion if neither `$path` or `$base` were absolute.

`infix:<~>` is constant folded.

`<prefix:<~>` is constant folded.

String comparison operators are constant folded.

String bitwise operators are constant folded.

`defined` is constant folded.

`Set`, `KeySet`, `Bag`, and `KeyBag` have `default` method.

When reading undefined property of those classes, `default` method contains value that you should receive. In case of sets, it’s `False`. For bags, it’s `0`.

Regexpes support more delimiters.

Previously, Rakudo didn’t accept regexpes like `m^42^` or `m€cash€`. Now, they are allowed, just like in Perl 5.

Niecza>

`Set`, `KeySet`, `Bag`, and `KeyBag` have `default` method.

Just like in Rakudo, except with Niecza.

Jonathan Worthington (6guts): Rakudo on JVM progress update, and some questions answered

Tue, 14 May 2013 23:29:33 +0000

It’s time for another progress update on the ongoing JVM work. Last time I posted here, we’d reached the point of having a self-hosting NQP ready to merge into the master branch of the NQP repository. That has now happened, so the May release of NQP will come with support for running on the JVM (note, this does not mean the May release of Rakudo will come with this level of capability, and a JVM-based Star release with modules is further still!) . In this post, I will discuss some of the things that have happened in the last few weeks and also try to answer some of the questions left behind in the comments last time.

The Rakudo Port

With NQP pretty well ported (there are some loose ends to tie up, but it’s pretty capable), the currently ongoing step is porting Rakudo. At a high level, Rakudo breaks down into:

The core of the compiler itself: the grammar (for parsing), actions (which assign semantics to the things we parsed), world (which takes care of the declarative aspects of programs), optimizer (tries to cheat without getting caught) and a few other small pieces to support all of this. This is written in NQP.
The Perl 6 MOP (meta-object protocol) implementation, which defines what classes, roles, enums, subsets and so forth mean. This is also written in NQP.
The bootstrap, which uses the MOP to piece together various of the core Perl 6 types. It does Just Enough to let us start writing Perl 6 code to define the rest of the built-ins. Also written in NQP.
The core setting, which is where the built-in types, operators and methods live. This is written in Perl 6.
A chunk of per-VM code that does lower-level or performance-sensitive things.

The first 3 of these need…an NQP compiler. And, it turns out that we have one of those hosted on the JVM these days. But could it handle compiling the Perl 6 grammar? It turns out that, after some portability improvements to the various bits of Perl 6 compiler code, the answer was a resounding “yes”. In fact, the grammar was compiled down to JVM bytecode sufficiently accurately that I didn’t encounter a single parse failure on the entire CORE.setting (though there were lots of other kinds of failures that took a lot of work – more on that in a moment). As for the rest of the compiler code, there were bits of lingering Parrot-specific stuff throughout it, but with a little work they were abstracted away. By far the hardest was BOOTSTRAP, which actually runs a huge BEGIN block to do its setup work and then pokes the results into the EXPORT package. This is kinda neat, as it means the setup work is done once when building Rakudo and then serialized. Anyway, onto the next pieces.

Compiling the Perl 6 setting depends on the Perl 6 compiler working. Since the first thing the setting does is use the bootstrap, which in turn uses the MOP, it immediately brings all of the above three pieces together. While we talk about “compiling” the setting, there’s a little more to it than that. Thanks to various BEGIN time constructs – such as traits, constant declarations and, of course, BEGIN blocks – all of which show up in the CORE setting – we actually need to run chunks of the code during the compilation process. That’s right – we run bits of the file we’re in the middle of compiling while we’re compiling it. Of course, this will be nothing new to Perl folks – it’s just most Perl programmers probably don’t worry about how on earth you implement this. :-) Thankfully, it’s a solved problem in the NQP compiler toolchain, and the stuff that makes NQP BEGIN blocks work largely handles the Rakudo ones too.

Anyway, all of this means that even getting the setting to finish the parse/AST phase requires doing enough to run the various traits and so forth. And that in turn brings in the fifth piece: the per-VM runtime support. This includes signature binding, container handling and a few other bits. Thankfully, it no longer involves multiple dispatch, since that is written in NQP these days (apart from some caching infrastructure, which is shared with NQP’s multiple dispatch, and thus was already done). Getting through the parse/AST phase of the setting doesn’t need all of the runtime support to be implemented, but it did require a decent chunk of it. Of course, at the start everything is missing, so getting from line 1 to line 100 was O(week), from 100 to 1000 O(day) and each thousand or so from there O(hour). It’s 13,000 or so lines in all.

The parse/AST step is just the first (though biggest) phase of compiling Perl 6 code, however. Next comes the optimizer, followed by code generation. In theory, the optimizer could have been bypassed. I planned to do that, then discovered it basically worked for the set of optimizations that didn’t need signature binding to participate in the analysis, so I left it in. Code generation is part of the backend, and so is shared with NQP. So it shoulda just worked, right? Well, yes, apart from code generation is the place where nqp:: ops get resolved to lower level stuff. And Perl 6 uses a lot more of them than NQP. Note that they only have to be mapped to somewhere; the JVM is late bound enough that it won’t complain unless you actually hit the code path that tries to use something that is not yet implemented. In reality I did a bit of both: implementing those that would surely be hit soon or that were trivial, and leaving some others for later.

So, some time yesterday, I finally got to the point of having a CORE.setting.class. Yup, a JVM bytecode file containing the compiled output of near enough the entire Perl 6 core setting. So, are we done yet? Oh, no way…

Today’s task was trying to get the CORE setting to load. How hard could that be? Well, it turns out that it does a few tasks at startup, most of which hit upon something that wasn’t in place yet. Typically, it was more of the runtime support, though in some cases it was unimplemented nqp:: ops. Of course, there were a handful of bugs to hunt down in various places to.

Anyway, finally, this evening, I managed to get the CORE setting to load, at which point, at long last, I could say:

perl6 -e "say 'hello world, from the jvm'"
hello world, from the jvm

Don’t get too excited just yet. It turns out that many other simple programs will happily explode, due to hitting something missing in the runtime support. There’s still plenty of work to go yet (to give you an idea, trying to say a number explodes, and a for loop hits something not yet implemented), but this is an important milestone.

Interop

A couple of the comments in response to my last post asked about interop with Java. There’s two directions to consider here: using Java libraries from Perl 6, and using Perl 6 code from Java. Both should be possible, with some marshalling cost, which we’ll no doubt need to spend some time figuring out how to get cheap enough it’s not a problem. It may well be that invokedynamic is a big help here.

The Java stuff from Perl 6 direction can probably be made fairly convenient to use by virtue of the fact that Perl 6 has a nice, extensible MOP. The fact the object you’re making a call on lives in Java land can be just a detail; we can hide it behind the typical method call syntax, and should even be able to populate a 6model method cache with delegation methods that do the argument mapping. I’m sure there will be plenty of interesting options there. I suspect we’ll want to factor it a little like NativeCall – some lower level stuff in the runtime, and some higher level sugar.

Going the other way will be more “fun”. I mean, on the one hand the marshalling is just “in the other direction”, which we’d need to do for values coming back from Java land anyway. But trying to work out how to make it feel nice from Java land could be trickier. I don’t believe the “.” operator is very programmable, which probably leaves us with string lookups or code-generated proxy thingies. Or maybe somebody will come up with a Really Great Solution that I hadn’t thought of.

My JVM related Perl 6 dev focus for now will be getting Rakudo to work decently and start getting Perl 6 modules working on the JVM also, but interop with Java land is certainly on the roadmap of things I think should happen. As with all things, I’m delighted to be beaten to it, but will work on it if it goes undone for too long. :-)

Performance?

The first thing to say on this is that it’s too early to have a really good idea. The final pieces of the gather/take transform (which has global consequences) have yet to land, which will certainly have some negative impact and will need to happen soon. At the same time, I’ve been very much focused on making things work on the JVM at all over making them especially clever or optimal. Numerous things can be done in ways that will not only perform better in a naive sense, but that will also be much easier for the JVM’s JIT to do clever stuff with. There are many, many things we will be able to do in this area.

Since I only have a sort-of-working Perl 6 compiler, I can’t say that much about Perl 6 performance. The only result I have to share is that the CORE setting parse completes in around a third of time that it does on Parrot (noting it’s not only about parsing, but also some code generation and running of stuff). This is not especially great – of course, we need to do better than that – but it’s certainly nice that the starting point before I really dig into the performance work is already a good bit faster.

The other result I have is NQP related. nwc10++ has been doing performance testing with a Levenstein benchmark of each commit to the NQP JVM work, which is really great in so far as it gives me a rough idea if I accidentally regress (or improve ;-)) something performance wise. There, we saw a larger factor performance win, around 15 times faster than the same program running in NQP on Parrot.

The big negative news on performance is startup time. Part of this is just the nature of the JVM, but I know another enormous part of it is inefficiencies that I can do something about. I’ve plenty of ideas – but again, let’s make things work first.

From Here

Things will be a little quieter from me over the next week and a bit, due to a busy teaching schedule. But now we have a fledgling Rakudo on JVM, and from here it’s going to be making it gradually more capable, first passing the sanity tests, then moving on to the spectests and the ecosystem. There are ways to help for the adventurous. Some ideas:

Profile the code generation phase, which is one of the pieces that is slower than expected. Try to figure out why.
Have a look at how multiple dispatch stuff is currently set up, and see if the dispatch logic could possibly be shuffled off behind invokedynamic.
Try something. See how it explodes. See if you can fix it. (Yes, generic I know. :))
Have a look at a “make install” target for Rakudo on JVM.

I’ll be speaking on the JVM work at the Polish Perl Workshop the weekend after next, and hope to have something a bit more interesting than “hello world” to show off by then.

Konrad Borowski: Perl 6 changes - 2013W19

glitchmr@myopera.com (Konrad Borowski) — Sat, 11 May 2013 07:00:00 +0000

So, well, what’s new. I’m going to answer - lots of changes and fixes. Even the Niecza, where lately development had slowed down got lots of fixes.

id='incompatible_changes'>Incompatible changes>

Rakudo Perl>

Array references aren’t treated as arrays in regular expressions.

Following code used to find 1, 2, or 3.
```
my $array = [1, 2, 3];
my $number = prompt 'Give me a number!';
say "It's in array" if $number ~~ / ^ $array $ /;
```
It doesn’t do that anymore, because variables are expected to be stringified inside regexpes. If you want the old behavior, be explicit and write @$array instead of $array.

Please note that currently regular expressions are exception to this rule. This is going to be fixed in the future. Lots of code depends on regexpes to be not stringified, and first the warnings will have to be implemented. You are expected to use <$regexp> instead of $regexp.

Niecza Perl>

Changed `capitalize` to `wordcase`.

In the current specification, that method is named `wordcase`.

Removed `ucfirst`.

`ucfirst` was long deprecated, and it’s finally time to remove it. You are expected to use `tc` instead.

Stringification of `Set`, `KeySet`, `Bag`, and `KeyBag` works properly when they are undefined.

Methods `Str`, `gist` and `perl` expected they deal with defined object. This isn’t the case when dealing with the classes themselves.

Deprecated `$%` and `$@` variable checkers are working just like in STD.

The `$@(1)` was complaining about use of deprecated variable, when the real problem was use of invalid hard reference syntax.

id='new_features'>New features>

Rakudo Perl>

`IO::Spec::Win32` no longer warns when using relative pathes.

The check for initial prefix was using optional alternative (`([a | b | c])?`). Because of that, it was `Nil` if it matched nothing. Since the result of match was stringified just after it was matched, it returned a warning when `?` matches nothing. It was fixed to do `([a | b | c]?)` instead.

Added tests for `IO::Path` and `IO::Spec` to list of tests that should pass.

As `IO::Path` and `IO::Spec` were implemented, their tests are now ran when you request full test from roast test suite.

Interpolation of quotes in regexpes isn’t done twice anymore.

Previously, quotes inside RegExpes were doing interpolation two times.

`match` method is fast again.

https://github.com/rakudo/rakudo/commit/c255f1df the older commits) made it 10 times slower. The performance regression was fixed, and now `match` is as fast as it used to be.

`wordcase` method now accepts `where` which is `Mu`.

Previously, only `Any` values were accepted. While using `Mu` doesn’t make much sense, there is no reason to disallow it, so it’s allowed now.

Configuration instructions don’t say to use –gen-nqp.

It was pointless, since `--gen-parrot` automatically implied `--gen-nqp`. If you want Parrot, it’s safe to assume you want NQP for that Parrot too, right?

`our` hashes and arrays work.

If you had used `our @array`, and `our %hash`, the object wasn’t initialized properly, and it wasn’t working like array or hash. This made `our` work only with `$scalar`s previously.

Previously, trying to use `our` with arrays or hashes caused strange uninitialized variable to appear. It didn’t have `STORE` method, so you couldn’t have put anything in this variable, and had `Any` instead of empty array of hash.

It’s now possible to augment empty list.

`MONKEY_TYPING` couldn’t be used to modify lists. Now it can.

Niecza Perl>

`Pair.perl` can now deal with the cases when `Pair` is the key.

Doing `((a => 'b') => 'c').perl` used to output `"a" => "b" => "c"`, what actually meant `(a => (b => 'c'))`. `.perl` now outputs correct code in that case.

Implemented Hash.ACCEPTS.

Carl Masak: After the #masakism workshop

Sun, 05 May 2013 21:22:45 +0000

On May 1st I learned what happens if you tell a bunch of people on the Internet, not all of whom you've met before, that you're going to teach Perl (5 and 6) for free on an IRC channel for four hours.

It worked well. Actually, it worked well beyond my expectations.

Successes

Prepare. I had written a bunch of material; short-ish texts mixed with exercises. I didn't hear any feedback about the texts, but people threw themselves on the exercises. Which was lucky, because I basically didn't have time to interact much with people. Everyone did their own thing.
Map. The week before the workshop, I was taking a walk, and it occurred to me that the topics for the workshop should be laid out on a subway map. That turned out to work unexpectedly well. People followed the structure, I think. The material was pretty clear that they were free to skip/jump around if they wanted, so some did that.
Github. People suggested fixes and improvements to the material during the workshop. I handled the easy ones, and delegated what could be delegated. If anything about the workshop felt 2013-futuristic, it was the fact that participants were hacking on the workshop in real time, as it was being played out. Github was totally right for this.
- tadzik++ did an exercise and got a solution that I didn't expect. We agreed that a test was missing to force people to do the right thing, so he went ahead and added it.
- LlamaRider++ added hyperlinks between all the files.
- grtodd++ made the subway map clickable.
- choroba++ added hyperlinks to a file that I added later.

Things learned

Newbies. This was never a workshop for total beginners. Still, we got a number of those. I'm not sure how many. If there's a next time, I'll want to add a track for the people who haven't done Perl before.
Exercises. Based on the tasks that people actually solved, I got a lot of feedback on what worked and what didn't. Which exercises served the overall message of the course, and which ones didn't.
Message. After the workshop, I know much better what I wanted the whole thing to be about. I think I can go back and make that even more clear. I'm still surprised at how well it worked already the first time around... but some bits in there can certainly be improved.

Message

So... what was the real message of the workshop? What is "masakism"?

Two things:

Testing gets you far.
- My pain threshold for writing things without tests is half a screen of code. If I have more than that, there's probably something I should be writing tests for.
- That said, there's no need to be a fundamentalist about anything. I'm just saying that tests are good, and still somewhat undervalued sometimes. But it's always a question on spending time testing the stuff that matters. Of course.
- Even with tests, there's no substitute for knowing what you're doing and where you're going. Tests help a lot, but they don't write the program for you.
Keep it small and simple.
- "Inside every big interesting problem is a small interesting problem, struggling to get out."
- The realization that you can write objects that focus entirely on your core problem, is a strong one.
- People tend to bring in persistence too early, "polluting" their core classes with database code. Hindering testability. This is backwards.
- Your program should be like an onion, with the most precious stuff in the middle, and layers around that. Outer layers should point to inner layers, but not vice versa.

Once more?

So, should we do another #masakism workshop?

Yes, maybe we should. People seemed to like this first one. I'm open to finding a datetime for another one.

If you have any suggestions, get in touch.

Konrad Borowski: Perl 6 changes - 2013W18

glitchmr@myopera.com (Konrad Borowski) — Sat, 04 May 2013 07:00:00 +0000

Perl 6 gets more and more stable, less features are added, and the features that are left are worked on. Also, Rakudo Perl is being ported to JVM.

id='new_features'>New features>

Perl 6 specification>

Updated IO:: classes documentation pages.

To many minor changes to describe, just follow the link.

Added `IO::Spec`, `IO::Path.cleanup`, and `IO::Path.resolve`.

Flip flops now use smartmatch semantics instead of boolean semantics.

Now you can write code like `"BEGIN" ff "END"`, instead of using regexes for that.

STD.pm6>

Use of & in infix position is now catched after whitespace.

Previously, code like `$some-object.some-method &sub-routine` appeared to work. However, it didn’t meant what you would expect, it didn’t called the method with subroutine argument, but assumed junction and. In order to avoid surprises, this is now disallowed.

The message about no parens or colon after method name is more specific.

Now code like `$some-object.some-method 42` shows information that colon or parens are required.

Added lots of function declarations.

The STD now recognizes certain functions that were added lately.

`is-prime`

`expmod`

`mkdir`

`rmdir`

`socket`

`listen`

`accept`

`connect`

`bind`

`infix:<leg>`

`infix:<cmp>`

Added `LoL` class and `X` role.

Rakudo Perl>

Implemented IO::Spec.

This class is intended to do OS-specific manipulation of pathes. It can get the localizations of special folders (for example home directory, root directory, temponary directory), and can manipulate pathes (by joining them, spliting them, converting them to absolute or relative, and so on).
Implemented IO::Path::<os subclasses>.

Thanks to this change, the pathes objects are OS specific. You can have path marked to be Windows path, so you can parse it in OS specific way, without having to use that OS.
Repeated shift is faster.

On large arrays, shift is now noticeably faster. For example, if array @a has 50_000 elements, running following snippet of code is processed within 2.5 seconds (and not 9.0 seconds).
```
@a.shift while @a;
```
Failed regex matches now return Nil.

Previously, failed regex matches were returning the Match object that stringified to #<failed match> and was converted to False.
&diag function in Test.pm accepts Mu.

This, I think is self explanatory. &diag is a debugging function, outputing its argument. Forbidding Mu didn’t made sense, so it’s now allowed.
Exporting of subroutines now works properly.

use Module qw(sub1 sub2) should now work.
Fixed dir() to work on Windows.

This makes panda work on Windows.

Gabor Szabo: To merge or not to merge?

Sun, 28 Apr 2013 18:04:29 +0000

I have a problem: Should I keep the 3 Perl Maven sites ( Perl 5 Maven, Perl 6 Maven, and Perl Maven) separate, or should I merge them together? Or maybe I should merge P5M into PM and leave P6M separate.

For the full article visit To merge or not to merge?

Konrad Borowski: Perl 6 changes - 2013W17

glitchmr@myopera.com (Konrad Borowski) — Sat, 27 Apr 2013 07:00:00 +0000

Continuing, still with commit IDs.

id='incompatible_changes'>Incompatible changes>

Perl 6 specification>

`DateTime` doesn’t accept `Callable` timezones anymore. 5692ced333

Rakudo Perl>

Renamed `IO` to `IO::Handle`. `IO` itself is a role. 2dca6ed950 37c9956502

`DateTime` doesn’t accept `Callable` timezones anymore. 71ea142406

id='new_features'>New features>

Rakudo Perl>

Implemented `Cool.lines`. 0d1d1194a4

Carl Masak: t2: Rectangle haikus

Thu, 25 Apr 2013 20:50:50 +0000

sheep hide at bad rains

bitterly modest slumber

its star to their rains

     -- generated by one of the t2 solutions

How time flies. It's two months since I reviewed the first batch of p6cc solutions. I solemnly swear the next blog post will appear sooner than that.

## Generate rectangle haikus

Write a program that generates haikus. A haiku has three lines, where the first
and last lines have five syllables each, and the middle line has seven. For the
purpose of this exercise, each line consists only of letters and spaces.

The haikus generated have the additional requirement that each line be of equal
length. The length requirement should not be gamed in any way, for example by
padding lines with spaces.

    since this one does it
    rectangle plus a haiku
    it serves as fine text

The program should attempt to generate a new haiku each time. The haiku should
consist of English words. If the words make sense in some kind of sentence
structure, that's considered a big bonus. Humor and/or deeper meanings are even
bonuser.

If you attempt to cheat at this task, you will be defeated by people who don't.

You're free to supply your own wordlist. How you count syllables is up to you,
and part of the task. You won't be challenged on trivial differences in
syllable counts.

It's OK for the program to run for a while, but it should preferably terminate
within a reasonable span of time.

I warmly recommend people to check out the solutions to this one. People are all over the place, coming up with ways to generate a rectangular haiku.

Some patterns recur, though. Let's break things down into subtasks and discuss them separately.

Word lists. Some contestants had their wordlists inlined in the code. Others had it in a separate file. Some stored part-of-speech information together with the word, or syllable count.

Sentence structure. Solutions are all across the spectrum, all the way from "just spew words", via "generate something that could perhaps pass for a sentence", all the way to "try really hard to be grammatical". In my view, being more high-end on this bit pays off big in quality. But it also costs in code complexity and program speed.

Search. This is a biggie, because it bleads through into the whole design of the program. Some of the solutions simply brute-force it. Others generate stuff randomly but store things in a hash table, guaranteeing that eventually some triplet of lines will form a haiku of some length. Some algorithms do clever things here, like turning problems around, going from "how many syllables does this line have?" to "ok, give me a word with this many syllables". One solution partitions the integers 5 and 7 in all possible ways, and hard-codes the resulting table.

But the really interesting bit in all solutions is the syllable counting. This is where the participants really differ in approach. One went with porting the CPAN module for counting syllables. Someone else did the same, but put the module on modules.perl6.org for everyone to use. Two contestants seem to have come up with their own (flawed) syllable-counter. A few people evade the syllable-counting by just storing the values along with the word lists. (Which is fine.)

All in all, an interesting set of solutions to a fun problem. Somehow, after reviewing all of these, I have the distinct feeling that a "best" solution could be put together by combining the best bits of everyone's solutions. I might just do that myself, if no-one beats me to it.

Next up: arranging wire crossings to rearrange wires!

Carl Masak: The #masakism workshop

Wed, 24 Apr 2013 22:37:50 +0000

So, as part of my $dayjob, I was teaching a Perl course. Then this happened:

* masak is teaching Perl 5 today! :)
<Heather> masak teach me
<masak> Heather: later; these people are paying me. :P
[...]
<masak> seriously though; if there were enough interest here on IRC, I
        would totally do a 1-day Perl 5 IRC workshop, pro bono.
<masak> ditto a one-day Perl 6 IRC workshop.
<vaelxon> (Perl 6 IRC workshop)++
<masak> it'd be very awesome, I promise. I'd need some time to prepare.
        but then we can grab a channel and basically party/learn for a
        day together.

And that's now what's happening. It'll be a loose-knit group of us, joining up in the #masakism IRC channel for four hours on May 1.

The course will be two-pronged; Perl 5 or Perl 6, whatever people prefer.

There will be prepared study material and exercises, but basically, people who participate can go off on a tangent, just spend the four hours solving one of the exercises really well, or solving a different exercise that they make up themselves. People can focus on doing the exercises, or on discussing various ways to solve them, or on discussing general principles of programming. Think of the workshop as a hybrid of people writing/showing nice code, and people discussing programming best practices.

Why am I doing this? Because as part of teaching Perl courses for $dayjob, I realized how much I love teaching Perl. I literally spend those two course days teaching Perl in a mild state of euphoria. I teach a lot of other courses, and some of them are great to teach, but none of them are like Perl. I think it's because I feel at home in the language, my brain has somehow "shaped itself" around Perl, and the questions I get can often be answered with a happy "let's find out!" where (a) we do some quick experiment in a terminal window, and (b) I tend to know what we'll get before, so it's easy to explain.

The whipuptitude of Perl really works to one's advantage there. That, combined with knowing my way around most of the perldoc pages and most of the special variables. Oh, and I probably have a slight advantage being fairly deeply involved in Perl 6 stuff, too.

Anyway, I absolutely love teaching Perl. Five and six. So much so that I'd do it for free. So this is me doing it for free.

As I write this, there are 13 participants signed up for the course. This is already twice as big as the typical Perl course I teach in a corporate setting. It's also a wide range of people; from relative newbies to people who could probably give the workshop better than me. I expect people further up on the learning curve to step in and assist with people further down. But I hardly need to say that; this is Perl, and we're a helpful bunch.

Here's more info about the workshop, including when, where, what, and how to sign up.

Konrad Borowski: Perl 6 changes - 2013W16

glitchmr@myopera.com (Konrad Borowski) — Sat, 20 Apr 2013 07:00:00 +0000

It’s sixteen article in the series. The game goes so fast, and I still haven’t done pagination on my website (lazy as always). The scrollbar is still shorter (1 pixel on my PC) than on Planet Six, but most likely because of longer lines.

NQP was bootstrapped on JVM. That means it can compile itself. It still doesn’t support Perl 6 perfectly, because some code still uses pir::, but it progressed really far. Also, the new version of Rakudo Perl, 2013.04 “Albany” was released. Every improvement mentioned here, is in.

I’ve decided to experiment with commit links this time. This way, you can click (or press, or whatever, depends on your device, to get code of this change).

id='potentially_incompatible_changes'>Potentially incompatible changes>

Perl 6 specification>

Removed `IO::Dir` and `IO::File` classes. fb25fa475854

id='new_features'>New features>

Rakudo Perl>

`IO::Path` is closer to actual spec. 24532aca13

Added `Bool` method to `Capture`. 9f873d89e3

`List.ACCEPTS` now has special `Whatever` handling. f17d339185

`do...for` doesn’t propose `repeat...for` now. 41a92e808e

Documented build memory requirements (1GB of memory). 4109be8905

`require PATH|$variable|Module:file(PATH)` syntaxes are now accepted. c946d0f7f7

Removed reporting variable name for uninitialized variable warnings because it was producing too many false positives. c39b887388

Replaced more `pir::` to `nqp::`. In particular, getting env variables is now done using NQP, not Parrot directly. e9e1b5d962

Jonathan Worthington (6guts): NQP on JVM bootstrapped, soon will land in NQP master

Thu, 18 Apr 2013 16:37:13 +0000

The work to get NQP running and bootstrapped on the JVM has reached an interesting milestone, and I thought I’d take a few moments from working on it to talk about what has taken place since my last post here, as well as taking a look at what will be coming next.

Getting NQP on JVM Bootstrapped

When I last posted here, I had reached the point of having an NQP cross-compiler to the JVM that covered quite a lot of NQP’s language features. Cross-compilation meant using NQP on Parrot to parse NQP source, build an AST, and then turn the AST into a kind of JVM assembly language. This in turn was then transformed into a JVM bytecode file by a small tool written in Java, which could then be executed on the JVM.

Since the last post, the cross-compiler became capable of cross-compiling NQP itself. This meant taking the NQP source files and using the cross-compiler to produce an NQP that would run on the JVM – without depending on Parrot. This also enabled support for eval-like scenarios. I reached this stage last month; the work involved tracking down a range of bugs, implementing some missing features, and doing a little more work to improve NQP’s portability. So, we had an NQP that ran on the JVM. Port done? Well, not quite.

A little vacation later, I dug into the next stage: making NQP on the JVM able to compile (and thus reproduce) itself. While I’d already implemented the deserialization used to persist meta-objects (representing classes, grammars and so forth), for this next step I had to implement the serialization side of things. Thankfully, there are a bunch of tests for this, so this was largely “just” a matter of working through making them pass. Finally, it was time to work through the last few problems, and get NQP on JVM able to build the NQP sources – and therefore able to compile new versions of itself. I decided to do this work as part of moving the JVM support into the main NQP repository.

Since we’ve only had NQP running on one backend (Parrot) up until now, certain aspects of the repository structure were not ideal. Before starting to bring in the JVM support, I first did a little bit of reorganization to segregate the VM specific components from the VM independent ones. Happily, much of NQP’s implementation falls into the latter category. Next came gradually building up the bootstrapping build process, working a file at a time, tracking down any issues that came up. This was a little tedious, especially given a couple of the problems were separate compilation leakages (where things from the running compiler would get confused with the version of the compiler that it was compiling). It was pretty clear that this was the problem from the errors I was seeing, but such problems show up long after things actually go wrong, requiring some careful analysis to hunt down. With those leaks plugged, and a few other relatively small bugs fixed, I had a working NQP on JVM…compiled by NQP on JVM.

The work from there has been to fill out the rest of the build process, adding in the second bootstrap stage and the test targets. The good news: the NQP produced by NQP on JVM passes all the tests that the original cross-compiled version did, so we’ve got no regressions there as a result of the bootstrap.

This work is currently sat in the jvm-support branch of the NQP repository. After the upcoming NQP release, it will be merged.

Supporting invokedynamic

Amongst all of this progress, we’ve also gained infrastructure to support using the invokedynamic instruction. This is a mechanism that enables those implementing non-Java languages on the JVM to teach its JIT about how their dispatch works. Most of the hard work here was done by donaldh++. I’d initially built things using BCEL in order to do code generation. While it served well up to a point, it turns out that ASM has much better support for invokedynamic, as well as being a little faster. So, donaldh got things switched over, and I soon was able to emit invokedynamic.

So far, we are not using it a great deal (just for making QAST::WVal compile to something a bit – or potentially a lot – cheaper), but in the future it will be used for things you’d typically think of as invocations (sub and method calls). I’ll write in more detail about it as things evolve.

What next?

With NQP now ported, the focus will fall on Rakudo itself. Quite a lot of preparations have already been made; for example, many pir:: ops have been replaced with nqp:: ones, multiple dispatch has been ported to NQP from C (fixing some bugs along the way), the way HLL boundaries work has been updated to cope with a fully-6model world (this also let me fix a long-standing introspection bug).

The path through getting Rakudo ported will largely follow the build order. This means starting with the module loader, then the compiler, followed by the MOP and its bootstrapping. After that comes the setting – the place where the built-ins live. There’s around 13,650 lines worth of that, so of course I expect to take it a little at a time. :-) I’ll try to remember to get a progress update here in a couple of weeks time.

Moritz Lenz (Perl 6): The REPL trick

Thu, 18 Apr 2013 04:54:42 +0000

A recent discussion on IRC prompted me to share a small but neat trick with you.

If there are things you want to do quite often in the Rakudo REPL (the interactive "Read-Evaluate-Print Loop"), it makes sense to create a shortcut for them. And creating shortcuts for often-used stuff is what programming languages excel at, so you do it right in Perl module:

use v6;
module REPLHelper;

sub p(Mu \x) is export {
    x.^mro.map: *.^name;
}

I have placed mine in $HOME/.perl6/repl.

And then you make sure it's loaded automatically:

$ alias p6repl="perl6 -I$HOME/.perl6/repl/ -MREPLHelper"
$ p6repl
> p Int
Int Cool Any Mu
>

Now you have a neat one-letter function which tells you the parents of an object or a type, in method resolution order. And a way to add more shortcuts when you need them.

perl6.announce: Parrot 5.3.0 "W00tstock Parrot" Released! by Bruce Gray

Wed, 17 Apr 2013 19:53:19 +0000

We are stardust.
Billion year old carbon.
We are golden.
Caught in the devil's bargain
And we've got to get ourselves back to the garden.
(To some semblance of a garden.)
-- "Woodstock", by Joni Mitchell

On behalf of the Parrot team, I'm proud to announce Parrot 5.3.0, also known
as "W00tstock Parrot". Parrot (http://parrot.org/) is a virtual machine aimed
at running all dynamic languages, and currently focusing on Perl 6.

Parrot 5.3.0 is available on Parrot's FTP site
(ftp://ftp.parrot.org/pub/parrot/releases/devel/5.3.0/), or by following the
download instructions at http://parrot.org/download. For those who would like
to develop on Parrot, or help develop Parrot itself, we recommend using Git to
retrieve the source code to get the latest and best Parrot code.

Parrot 5.3.0 News:
- Build
+ Files generated by `make cover` are now correctly cleaned by `make`
and ignored by `git`.
- Tests
+ Internal testing of the Configure probe for Fink now works correctly
with the --verbose flag.
+ Tests added for .sort method of ResizableFloatArray and
ResizableIntegerArray. [GH #926], [GH #927]
+ Benchmarks added for .sort methods of various Array objects.
[GH #175]
+ Coverage analysis added for pbc_disassemble.

The SHA256 message digests for the downloadable tarballs are:
79d6f1fe20645b0afbc496cd0d7850a78b8940230e7637c5356d780f5aa1750b parrot-5.3.0.tar.gz
4cff32521c79d8a783ad57d9a13e205ea3c1b1585085e0da80138b58b77d0ed5 parrot-5.3.0.tar.bz2

Many thanks to all our contributors for making this possible, and our sponsors
for supporting this project. Our next scheduled release is 21 May 2013.

Enjoy!

Konrad Borowski: Perl 6 changes - 2013W15

glitchmr@myopera.com (Konrad Borowski) — Sun, 14 Apr 2013 07:00:00 +0000

I apologize I forgot about Perl 6 changes article yesterday. This ought not to happen, but it did. As for reason why it happened, well, I guess I will tell you anyway - it’s that game.

id='new_features'>New features>

Perl 6 specification>

`Parcel`s are allowed to have 0 elements.

Rakudo Perl>

`.DUMP` method can detect recursion for hashes.

Added `ForeignCode` for non-Perl 6 code objects (like C code).

Removed lots of `pir::` usage for JVM port of Rakudo.

Semicolon after keyword that takes a block, like `PRE ;` now has specific error message.

Added `symlink` and `link`.

`require` can take paths.

Niecza Perl>

Added special cases for `big_pow` when either of values is `-1`, `0`, or `1`.

Konrad Borowski: Perl 6 changes - 2013W14

glitchmr@myopera.com (Konrad Borowski) — Sat, 06 Apr 2013 07:00:00 +0000

You may have noticed may page went down for some time. This appears to be connected to newest domain change. In order to be more… modern, I’ve moved the repository to glitchmr.github.io, but that made entire page to go down. I apologize for inconvience, but it didn’t affected Planet Six, so I think it’s fine.

As always, I’m going to describe changes in Perl 6.

id='new_features'>New features>

Perl 6 specification>

Renamed `weekday-of-month` to `day-of-week-in-month`.

Rakudo Perl>

Fixed signature of `Pod::Block.gist`, so it works properly with `multi`s.

`for` loops are eager now.

Added `Buf.perl`.

Fixed `Buf.gist`, so it works properly with `multi`s.

Moritz Lenz (Perl 6): Rakudo's Abstract Syntax Tree

Mon, 01 Apr 2013 00:22:59 +0000

After or while a compiler parses a program, the compiler usually translates the source code into a tree format called Abstract Syntax Tree, or AST for short.

The optimizer works on this program representation, and then the code generation stage turns it into a format that the platform underneath it can understand. Actually I wanted to write about the optimizer, but noticed that understanding the AST is crucial to understanding the optimizer, so let's talk about the AST first.

The Rakudo Perl 6 Compiler uses an AST format called QAST. QAST nodes derive from the common superclass QAST::Node, which sets up the basic structure of all QAST classes. Each QAST node has a list of child nodes, possibly a hash map for unstructured annotations, an attribute (confusingly) named node for storing the lower-level parse tree (which is used to extract line numbers and context), and a bit of extra infrastructure.

The most important node classes are the following:

QAST::Stmts: A list of statements. Each child of the node is considered a separate statement.
QAST::Op: A single operation that usually maps to a primitive operation of the underlying platform, like adding two integers, or calling a routine.
QAST::IVal, QAST::NVal, QAST::SVal: Those hold integer, float ("numeric") and string constants respectively.
QAST::WVal: Holds a reference to a more complex object (for example a class) which is serialized separately.
QAST::Block: A list of statements that introduces a separate lexical scope.
QAST::Var: A variable
QAST::Want: A node that can evaluate to different child nodes, depending on the context it is compiled it.

To give you a bit of a feel of how those node types interact, I want to give a few examples of Perl 6 examples, and what AST they could produce. (It turns out that Perl 6 is quite a complex language under the hood, and usually produces a more complicated AST than the obvious one; I'll ignore that for now, in order to introduce you to the basics.)

Ops and Constants

The expression 23 + 42 could, in the simplest case, produce this AST:

QAST::Op.new(
    :op('add'),
    QAST::IVal.new(:value(23)),
    QAST::IVal.new(:value(42)),
);

Here an QAST::Op encodes a primitive operation, an addition of two numbers. The :op argument specifies which operation to use. The child nodes are two constants, both of type QAST::IVal, which hold the operands of the low-level operation add.

Now the low-level add operation is not polymorphic, it always adds two floating-point values, and the result is a floating-point value again. Since the arguments are integers and not floating point values, they are automatically converted to float first. That's not the desired semantics for Perl 6; actually the operator + is implemented as a subroutine of name &infix:<+>, so the real generated code is closer to

QAST::Op.new(
    :op('call'),
    :name('&infix:<+>'),    # name of the subroutine to call
    QAST::IVal.new(:value(23)),
    QAST::IVal.new(:value(42)),
);

Variables and Blocks

Using a variable is as simple as writing QAST::Var.new(:name('name-of-the-variable')), but it must be declared first. This is done with QAST::Var.new(:name('name-of-the-variable'), :decl('var'), :scope('lexical')).

But there is a slight caveat: in Perl 6 a variable is always scoped to a block. So while you can't ordinarily mention a variable prior to its declaration, there are indirect ways to achieve that (lookup by name, and eval(), to name just two).

So in Rakudo there is a convention to create QAST::Block nodes with two QAST::Stmts children. The first holds all the declarations, and the second all the actual code. That way all the declaration always come before the rest of the code.

So my $x = 42; say $x compiles to roughly this:

QAST::Block.new(
    QAST::Stmts.new(
        QAST::Var.new(:name('$x'), :decl('var'), :scope('lexical')),
    ),
    QAST::Stmts.new(
        QAST::Op.new(
            :op('p6store'),
            QAST::Var.new(:name('$x')),
            QAST::IVal.new(:value(42)),
        ),
        QAST::Op.new(
            :op('call'),
            :name('&say'),
            QAST::Var.new(:name('$x')),
        ),
    ),
);

Polymorphism and QAST::Want

Perl 6 distinguishes between native types and reference types. Native types are closer to the machine, and their type name is always lower case in Perl 6.

Integer literals are polymorphic in that they can be either a native int or a "boxed" reference type Int.

To model this in the AST, QAST::Want nodes can contain multiple child nodes. The compile-time context decides which of those is acutally used.

So the integer literal 42 actually produces not just a simple QAST::IVal node but rather this:

QAST::Want.new(
    QAST::WVal(Int.new(42)),
    'Ii',
    QAST::Ival(42),
)

(Note that Int.new(42) is just a nice notation to indicate a boxed integer object; it doesn't quite work like this in the code that translate Perl 6 source code into ASTs).

The first child of a QAST::Want node is the one used by default, if no other alternative matches. The comes a list where the elements with odd indexes are format specifications (here Ii for integers) and the elements at even-side indexes are the AST to use in that case.

An interesting format specification is 'v' for void context, which is always chosen when the return value from the current expression isn't used at all. In Perl 6 this is used to eagerly evaluate lazy lists that are used in void context, and for several optimizations.

Konrad Borowski: Perl 6 changes - 2013W13

glitchmr@myopera.com (Konrad Borowski) — Sat, 30 Mar 2013 07:00:00 +0000

The month is coming to end, and with that, I’m going to make yet another list of changes.

id='new_features_like_anything_is_other_than_that'>New features (like anything is other than that)>

Perl 6 specification>

Added many `IO::Path` methods: `.contents`, `.is-absolute`, `.is-relative`, `.absolute`, and `.relative`.

`.path` is now NOP that returns itself. For its previous behavior, use `.Str`.

Changed `Numeric.conjugate` to `Numeric.conj`.

Rakudo Perl>

`+(23..2300000000)` is optimized.

`--help` reports `-I` and `-M` options.

Tadeusz Sośnierz (tadzik): Polish Perl Workshop status update

Wed, 27 Mar 2013 11:56:41 +0000

Exactly two months remain until Polish Perl Workshop this year. In case you didn’t know, I’m shamelessly reposting the status update from our ACT website.

We’ve got the first sponsor offers, and about 15 talk submissions so far. Awesome! However, as we are well aware that most of you will wait to submit talks until there’s less than 2 days until the deadline (or later :)) I’m hereby announcing: The talk submission deadline for Polish Perl Workshop 2013 is 14 of April. That’s about two weeks to finally make up your mind and submit something. Please do! There’ll be only one chance to talk at the First Polish Perl Workshop: there’ll be no other :)

Of course, if you’d prefer to organize more of a hands-on event, we have the entire sunday dedicated to hackathon, BoFs, classes etc. If you want to submit something more lenghty, that’s the right way to do it.

In other interesting news, we started looking at conference t-shirts (yes, they’re free for all attendees). We have something on our mind that never happened before on a Perl conference (or at least I’ve never seen or heard of something like this). We’ll keep you informed.

perl6.announce: Parrot 5.2.0 "Stuffed Parrot" Released! by Bruce Gray

Sun, 24 Mar 2013 13:58:55 +0000

I am not dead yet I can dance and I can sing
I am not dead yet I can do the Highland Fling
I am not dead yet No need to go to bed
No need to call the doctor Cause I'm not yet dead
-- Spamalot

On behalf of the Parrot team, I'm proud to announce Parrot 5.2.0, also known
as "Stuffed Parrot". Parrot (http://parrot.org/) is a virtual machine aimed
at running all dynamic languages.

Parrot 5.2.0 is available on Parrot's FTP site
(ftp://ftp.parrot.org/pub/parrot/releases/devel/5.2.0/), or by following the
download instructions at http://parrot.org/download. For those who would like
to develop on Parrot, or help develop Parrot itself, we recommend using Git to
retrieve the source code to get the latest and best Parrot code.

Parrot 5.2.0 News:
- Core
+ IO now only syncs buffers for the IO types where syncing makes sense.
= PIO_VF_SYNC_IO flag added
- Build
+ installable_pdump now has the correct rpath (blib corrected to lib).
- Libraries
+ Tcl/Glob.pir has been removed. (PGE/Glob.pir remains intact)
- Ecosystem
+ All Parrot tarballs are now symlinked to the 'all' directory,
regardless of their true homes ('devel' or 'stable'), to better
allow for automated downloads.
ftp://ftp.parrot.org/pub/parrot/releases/all/

The SHA256 message digests for the downloadable tarballs are:
1245d11f2b2ea44e6465aff6da5a533324d69b6eb3ddf7d84e81385ea95150ad parrot-5.2.0.tar.gz
0c538d780f9c70c510e142a8a663c30474125c9fcf9fe25d2129e68fc7baec8d parrot-5.2.0.tar.bz2

Many thanks to all our contributors for making this possible, and our sponsors
for supporting this project. Our next scheduled release is 16 Apr 2013.

Enjoy!

--
Bruce Gray (Util of PerlMonks)

Konrad Borowski: Perl 6 changes - 2013W12

glitchmr@myopera.com (Konrad Borowski) — Sat, 23 Mar 2013 07:00:00 +0000

I really should optimize my blog layout, so empty intro wouldn’t be ugly. But I guess I’m too lazy to do it right now, so I have to explain stuff. Every week, I list the list of changes in Perl 6 specifications, and its implementations. That’s just everything, boring, I know. Still, it doesn’t look that bad on Planet Six, so I guess I will continue to make these articles :-).

This week, the Rakudo 2013.03 was released. Every change mentioned here was implemented in it.

id='new_features'>New features>

Perl 6 specification>

Added `X::Range::InvalidArg`.

Rakudo Perl>

Removed lots of `pir::` usage.

Multi dispatch is now implemented in NQP, not C.

Released Rakudo 2013.03.

Perl 6 Maven: Reading from a file in Perl 6

Tue, 19 Mar 2013 20:52:02 +0000

It can be interesting to do operations on existing data structures in Perl 6, but without Input and Output, that would have limited usability in the real world.

Therefore reading from files is a critical operation.

Konrad Borowski: Perl 6 changes - 2013W11

glitchmr@myopera.com (Konrad Borowski) — Sat, 16 Mar 2013 07:00:00 +0000

I would put nothing here, but the layout of blog would make this look ugly. So, well, like usually I put something here, even if it is meaningless. Just look below to see list of changes.

id='new_features'>New features>

Rakudo Perl>

Closing brace, followed by newline and infix operator was parsed incorrectly. For example, following code.
```
my %a;
{ }
%a = 1;
```
Was parsed like:
```
my %a;
{ } % a() = 1;
```
Instead of correct.
```
my %a;
{ };
%a = 1;
```
Done more pir:: to nqp:: replacements in order to help JVM port of Rakudo Perl.

Case insensitive matches with strings now work.

my $nonsense = 'asdf';
say "OK" if "ADSF" ~~ m:i/$nonsense/;

Typing else if and elif after if block shows “Please use ‘elsif’” error message.
&?ROUTINE variable is available inside regex foo { ... }.
The named params are optimized in compile time when possible.
X::Method::NotFound is raised even when dealing with "" (empty) method names.
Fixed proto of infix:<ne> in order to require Mu objects.
Stash.at_key method now works.
Postcircumfix calls can be cached now.
LEAVE is now fired on next/last in MapIter.
Exception is thrown when Range endpoint is used to create a Range.

Niecza Perl>

`Empty.gist` returns nothing.

Perl 6 Maven: Getting started with Rakudo * (2013.01) on Windows

Fri, 15 Mar 2013 20:52:02 +0000

After a break of several months, I rebuild Rakudo on the server, a few days ago, and regenerated the Perl 6 Maven pages. This promptly broke the site. The fix was easy, and as expected the bug was mine.

Today I decided to try Rakudo on my Windows machine as well, but I went for the easy route and installed the pre-built msi file.

Konrad Borowski: Perl 6 changes - 2013W10

glitchmr@myopera.com (Konrad Borowski) — Sat, 09 Mar 2013 08:00:00 +0000

During making this update, I had lots of problems with Internet. I have managed to load commits history for every repository I check, but I haven’t checked commits. Will update this after Internet access will be more stable.

id='new_features'>New features>

Perl 6 specification>

`Rat.perl` returns decimal number when denominator is normalizable to a power of 10.

Added FALLBACK methods.

Included Failure in smartmatch passthroughs.

Loosened relationship between `Nil` and `()`.

Added `List.permutations`.

`rat64` type has unsigned denominator.

STD.pm6>

`Nil` isn’t suggested for empty list in error messages anymore.

Rakudo Perl>

Fixed expanding of variables that don’t exist when converting string to RegExp. For example, `"a" ~~ / <$blah> /` is fixed.

`Rat.perl` returns decimal number when denominator is normalizable to a power of 10.

Removed lots of `pir::` usage in order to help porting Rakudo to JVM.

`Rat.perl` provides angle brackets around fractional forms.

Added missing shellwords postcircumfixes (`<< >>` and `« »`).

Generalized “augment” check, and made error message more generic.

User defined circumfixes are parsed just like STD.pm6 does.

Added colorcircumfixes.

Rakudo now warns about integers starting with `0`.

Mistakes like `if(1 < 2) { ... }` (where mistake is that `if` is called like a function) give good errors.

`Nil` isn’t suggested for empty list in error messages anymore.

Niecza Perl>

`Rat.perl` returns decimal number when denominator is normalizable to a power of 10.

`CommonEnum.gist` returns `(Type)` instead of `Type()` for undefined.

Restored `infix:<cmp>` sorting for different types.

Failed matches return `Nil`.

Solomon Foster: Philosophical Issues with Rakudo’s .parse

Wed, 06 Mar 2013 12:59:01 +0000

For a little bit now I’ve had the ABC module working on both Niecza and Rakudo. I got a new Linux box yesterday; this morning I was playing around with my ABC tools there, trying to see if I could get them to work. I installed them with panda, so I was using ABC with Niecza by default on my MBP and with Rakudo by default on my new machine. I tried it with a file of my own compositions. Apparently I hadn’t tried this in a while, because it had several issues. And the two systems handled them completely differently.

On Niecza, I got this message:

Unhandled exception: Did not match ABC grammar: last tune understood:
 A Hat for the Whales
d|:e~A3 BGEF|G2BG DG (3ABd|eBdB ~A3G|Bdef gfed|
e~A3 BGEF|G2BG DG (3ABd|eBdB AAGA|[1Beed e3d:|

On Rakudo, there were no error messages, but “A Hat for the Whales” was the last tune processed, even though there were a dozen more tunes in the ABC file after it.

Without thinking about it too much I corrected this batch of errors. Now on Niecza the entire file parsed, but I got the error
Unhandled exception: Illegal key signature B minor
Under Rakudo, the same ABC file just appeared to work.

What’s going on here? Well, key signature processing looks something like this (simplified):

    token key-def { <basenote> <chord_accidental>? <mode>? }
    token mode { <minor> | <major> }
    token minor { "m" ["in" ["o" ["r"]?]?]? }
    token major { "maj" ["o" ["r"]?]? }

No space allowed between basenote and mode. So when you invoke parse with that rule on “B minor”, in Niecza it fails, because the entire string was not successfully parsed. In Rakudo it succeeds, because “B” is a valid key signature, and it doesn’t care that the entire string was not parsed. In practice, this means that three of the tunes in the collection got the wrong key signature: B major instead of the correct B minor.

To my mind, Niecza’s behavior is more intuitive. If you’re trying to parse a file, parsing half of it and then silently ignoring the rest is wildly unhelpful behavior. In every use I’ve had for parse so far, I’ve wanted it to parse everything in the string I sent to the function.

But hey, I understand that some people might have different needs. Can we at least add a flag that will give you a parsing failure if the entire string is not parsed? :all, perhaps?

Konrad Borowski: Perl 6 changes - 2013W09

glitchmr@myopera.com (Konrad Borowski) — Sat, 02 Mar 2013 08:00:00 +0000

I would put a description, but who seriously cares?

id='new_features'>New features>

Perl 6 specification>

Inside of `try` is considered eager context, unless `try` is called in sink context, when it’s in sink context.

Rakudo Perl>

Type placeholders (like `Int`, which is null, except Int typed) now have their `.gist` method show their name within parens to suggest being undefined. For example, `(Int)`, `(Mu)`.

Changed “Cannot assign to a non-container” error message to “Cannot modify an immutable value” that hopefully is more clear.

Release announcements now use Markdown.

`Cool.substr` doesn’t coerce its first argument to `Int` anymore.

Konrad Borowski: I made a pointless webpage

glitchmr@myopera.com (Konrad Borowski) — Fri, 01 Mar 2013 08:00:00 +0000

So, well, I have a VPS, and decided to put a webpage on it for lulz. You can see it at mango.uk.to, and the source (sauce) is available on my GitHub profile.

Just mentioning, not that this site is important. I just wanted to run a web server just to run a web server.

By the way, this blog is also open sauce, you can see its source code on GitHub. It’s full of hacks, but that’s Jekyll for you.

rakudo.org: Rakudo Star 2013.02 released

Sun, 24 Feb 2013 19:21:21 +0000

On behalf of the Rakudo and Perl 6 development teams, I’m happy to announce the February 2013 release of “Rakudo Star”, a useful and usable distribution of Perl 6. The tarball for the February 2013 release is available from http://rakudo.org/downloads/star/. A Windows .MSI version of Rakudo star will usually appear in the downloads area shortly after the tarball release.

In the Perl 6 world, we make a distinction between the language (“Perl 6″) and specific implementations of the language such as “Rakudo Perl”. This Star release includes release 2013.02.1 of the Rakudo Perl 6 compiler, version 4.10.0 of the Parrot Virtual Machine, and various modules, documentation, and other resources collected from the Perl 6 community.

Some of the new features added to this release include:

“Did you mean …” suggestions for symbol-not-found errors
Compile-time optimization of some cases of junctions in boolean context
IO::Socket.get now works again with non-ASCII characters
constant folding for routines marked as ‘is pure’
natively typed variables and better error reporting in the REPL
speed up eqv-comparison of Bufs
warnings for useless use of (some) literals, variables and constant expressions in sink context

This release also contains a range of bug fixes, improvements to error reporting
and better failure modes.

The following features have been deprecated or modified from previous
releases due to changes in the Perl 6 specification, and are being removed
or changed as follows:

.gist on a type object will return ‘(Typename)’ instead of ‘Typename()’. If you want to get the class name alone, continue to use $obj.^name
postcircumfix:<[ ]> and postcircumfix:<{ }> will become multi-subs rather than multi-methods. Both at_pos and at_key will remain methods.
Unary hyper ops currently descend into nested arrays and hashes. This will change to make them equivalent to a one-level map.
The Str.ucfirst builtin is deprecated; it will be replaced by Str.tc.
Leading whitespace in rules and under :sigspace will no longer be converted to <.ws> . For existing regexes that expect this conversion, add a <?> in front of leading whitespace to make it meta again.
The ?-quantifier on captures in regexes currently binds the capture slot to a List containing either zero or one Match objects; i.e., it is equivalent to “** 0..1″. In the future, the ?-quantifier will bind the slot directly to a captured Match or to Nil. Existing code can manage the transition by changing existing ?-quantifiers to use “** 0..1″, which will continue to return a List of matches.

There are some key features of Perl 6 that Rakudo Star does not
yet handle appropriately, although they will appear in upcoming
releases. Some of the not-quite-there features include:

advanced macros
threads and concurrency
Unicode strings at levels other than codepoints
interactive readline that understands Unicode
non-blocking I/O
much of Synopsis 9

There is an online resource at http://perl6.org/compilers/features
that lists the known implemented and missing features of Rakudo
and other Perl 6 implementations.

In many places we’ve tried to make Rakudo smart enough to inform the
programmer that a given feature isn’t implemented, but there are
many that we’ve missed. Bug reports about missing and broken
features are welcomed at
href="mailto:rakudobug@perl.org">rakudobug@perl.org.

See perl6.org for links to much more information about Perl 6, including documentation, example code, tutorials, reference materials, specification documents, and other supporting resources. A draft of a Perl 6 book is available as UsingPerl6-draft.pdf> in the release tarball.

The development team thanks all of the contributors and sponsors for making Rakudo Star possible. If you would like to contribute, see http://rakudo.org/how-to-help, ask on the perl6-compiler@perl.org mailing list, or join us on IRC #perl6 on freenode.

Konrad Borowski: Perl 6 changes - 2013W08

glitchmr@myopera.com (Konrad Borowski) — Sat, 23 Feb 2013 08:00:00 +0000

Now, I’m back to the your regularly scheduled Perl 6 changes article that isn’t. Well, it is, but it’s not. Or something. Rakudo 2013.02.1 was released (well, 2013.02 was released too). The version bump was caused by the fact that NQP 2013.02 had broken native call support.

id='incompatible_changes'>Incompatible changes>

Perl 6 specification>

Failed matches result `Nil`, not `(Match)`.

id='new_features'>New features>

Perl 6 specification>

Removed `Set.powerset`. It was removed after 10 hours of existing. Consider it a printing error in my previous article, or something. Not really an incompatible change, as it practically never existed.

Type placeholders (like `Int`, which is null, except Int typed) now have their `.gist` method show their name within parens to suggest being undefined. For example, `(Int)`, `(Mu)`.

STD.pm6>

Added a warning for smartmatch of `(Match)` or `(Nil)`.

Rakudo Perl>

The sink context now can be detected. Side-effect-free operations, constants and variables warn about it.

Compiler warns about apparent subscripts in regexes being treated as regexes (for example, `@a[0]`, which is treated as any of `@a` elements followed by `0`).

It’s possible to interpolate arrays in regular expressions.

Added `int64` native type.

`/./` and `rx/./` now matches against `$_` in sink context.

`$feature is not yet implemented. Sorry.` error message doesn’t have `is` anymore.

Coercive type declarations make not yet implemented error.

Niecza Perl>

Type placeholders (like `Int`, which is null, except Int typed) now have their `.gist` method show their name within parens to suggest being undefined. For example, `(Int)`, `(Mu)`.

Carl Masak: t1: Tell knights from knaves

Fri, 22 Feb 2013 22:39:50 +0000

<grondilu> masak: are you sure problem #1 is computationnaly doable?  It's not NP = P or something, is it?

* grondilu thinks of it and realizes the number of possibilities is not so large

<grondilu> .oO( 2**number_of_islanders  possibilities anyway )

And so, the time-honored tradition of reviewing p6cc solutions begins. Again.

For our first task, we have the very human conundrum of figuring out who is telling the truth and who is lying. Fortunately, on the island of Smul, that's a much more tractable problem than in the real world.

The problem description follows.

## Tell knights from knaves based on what they say

On the mythical island of Smul, people suffer from a rare genetic disorder that
make them either tell the truth all the time, or lie all the time. These are
the only two types of people on the island, known as knights and knaves,
respectively.

Write a program that takes as input a number of utterances by islanders, and
outputs for each person whether that person is a knight or a knave. If there is
no possible assignment that works, the program should report that no solution
exists. In the case of multiple solutions, the program should report every
possible solution.

The islanders can make four different classes of utterances:

    X is a knight.
    X is a knave.
    X and Y are of the same type.
    X and Y are of different types.

(Here, X and Y are used as metavariables, of course, and can in fact be any
name of an islander.)

Islanders can refer to each other. The same islander can make several
utterances. If an islander mentions another islander that doesn't say anything,
your program should consider the entire input to be erroneous.

Here are a few examples:

    A: A is a knight.

Both a knight and a knave would assert the same thing. So this input has two
solutions.

    B: B is a knave.

Neither a knight or a knave would ever say this about themselves. So this input
allows no solution.

    C: C and D are of the same type.
    D: D and C are of different types.

Here, the two islanders are contradicting each other, so one of them must be a
knight and the other a knave. But this is exactly what D is saying, so D is the
knight. One solution.

Let's get one obvious thing out of the way: we can solve this with a brute-force method, supposing for each islander in a situation first that he is a knight and then that he is a knave. Each islander thus bifurcates the universe in two possible universes. Two islanders will yield four possible universes to investigate. Three islanders will yield eight universes. Ten islanders will yield 1024 universes. The universes grow exponentially with the islanders. The brute-force solution will always work, but at an exponential slowdown.

Many people choose the brute-force solution. I don't blame them; it's there for the picking. Some people get fancy with logical propositions or "affiliations" between islanders, but they all fall down the exponential pit at one point or another. The brute-forcers are all over the place in terms of style and brevity, and it's quite fun to watch.

Then there's this one contestant that gets the nice solution. It always thrills me when that happens.

Here, let me lay it out for you. Let 0 mean False/knave and 1 mean True/knight. Then let's translate the four possible utterances to equations:

"A: X is a knight." → A = X
"A: X is a knave." → A = 1 xor X
"A: X and Y are of the same type." → A = 1 xor X xor Y
"A: X and Y are of different types." → A = X xor Y

By some amazing coincidence, all the utterances can be put into more or less the same mold, and the only operator used is xor. It's actually fun to translate these formulas back, and get alternative formulations of things, sometimes giving a different perspective on things:

A = X → "A is as truthful as X"
A = 1 xor X → "A or X is truthful, but not both"

From that first formulation we see why, when A is asserting that A is a knight, that utterance devolves into a tautology A = A. It's as if an islander cannot assert his own knighthood solely on his own authority.

From the second formulation we see why, when A is asserting that A is a knave, such an utterance devolves into a contradiction A = 1 xor A. There just isn't any such number A. (We'd have to look in the murky domain of fuzzy logic, but that's outside the realm of the island of Smul.)

Anyway, translating all the utterances to this form is a big win. Now we can make a linear equation system of all the utterances, and solve the linear equation system. Do we have fast algorithms for that? Yes, we do! Gaussian elimination has an arithmetic complexity of O(n³). That's quite an improvement on exponential brute force!

We must take care to do all the additions and subtractions as xors, though. This is because our underlying algebra is truth values, outside of which we may not stray. So we're really doing Gaussian elimination on the quotient ring Z/2Z.

In the literature, we find this problem as XOR-satisfiability. Wikipedia dignifies it with two sentences.

One contestant did it this way. Yay him. You should check out his solution.

Here are the solutions, and my reviews of same. Enjoy.

Next up: rectangle haikus, possibly the most fun I've ever had with a p6cc task.

Konrad Borowski: friendly interactive shell

glitchmr@myopera.com (Konrad Borowski) — Fri, 22 Feb 2013 08:00:00 +0000

I’m not sure if anybody noticed, but lately, nothing was happening on my blog. So, I’m going to write an article about friendly interactive shell (fish), the best shell I’ve used ever (it’s my personal opinion, in case you haven’t noticed, see my footer blog). Actually, I think I’ve moved to it half of the year ago or something.

These days, the two popular shells exist - bash and Z shell. I’m aware of that more shells exist - just nobody uses them these days By the way, if you try to find logical fallacy, I agree, cmd.exe and Windows PowerShell are often used, but those aren’t UNIX shells. Oh, and programming language REPLs are often used too (like irb), but they aren’t UNIX shells too.

id='bash'>Bash>

The bash shell is standard shell, included in many Linux distributions. It will make you bash your head against the brick wall (pun intended).

One of examples of bash code I could have found is this. And trust me, it’s not the worst thing (thanks, Wikibooks).

# print the powers of two, from 1 to 512:
for ((i = 1; i < 1000; i *= 2)); do
  echo $i
done

Do you see double parens here? This is really strange math syntax that only supports integers. Why double parenthesis? Well, the answer is compatibility with Bourne shell (really, really old shell, usually called sh (but sometimes sh is actually bash, to make you bash)).

Also, you could have noticed keywords do and done. Bash could easily not require do keyword before for loop body. Yet, it’s required. The other strange quirks is that you aren’t allowed to put semicolon after do keyword. Why? I’ve no idea, really. The done word, while I think it’s needed, differs between blocks. For example, if blocks end with fi.

Bourne again shell has lots of strange syntax you really wouldn’t ever use. For example, bash has $'' syntax. It unescapes escape sequences inside. The strange part is '' in bash doesn’t do any processing at all, yet $'' does. Bash also has $"" syntax, but it doesn’t do escape processing - instead it uses gettext to translate string inside.

# Prints 'Hello world.', with new line in middle.
echo Hello$'\n'world.
# Prints 'Hello\nworld.', literally.
echo Hello$"\n"world.

Consistency in bash is even worse than in languages such as PHP. There are many ways to do something, but none of them are obvious. And I mean that, none. Expanding bash is adding new syntax that totally doesn’t make sense, but was a syntax error in previous versions. $ sign is used for many features that have nothing with variables.

Also, bash integrates many programs in it that really shouldn’t be inside. For example, echo.

[glitchmr@pineapple ~]$ echo --help
--help
[glitchmr@pineapple ~]$ which echo
/bin/echo
[glitchmr@pineapple ~]$ /bin/echo --help
Usage: /bin/echo [SHORT-OPTION]... [STRING]...
  or:  /bin/echo LONG-OPTION
Echo the STRING(s) to standard output.

...

NOTE: your shell may have its own version of echo, which usually supersedes
the version described here.  Please refer to your shell's documentation
for details about the options it supports.

Report echo bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report echo translation bugs to <http://translationproject.org/team/>
For complete documentation, run: info coreutils 'echo invocation'

The /bin/echo on my system is GNU echo. The bash on my system is GNU bash. They were both made by GNU. Yet, they act differently. Is there a better recipe for failure than program that actually isn’t called because your shell is “clever”.

It’s not just echo. Many commands are implemented in shell, when they should be implemented outside shell. For example, pwd. Why exactly? To confuse you when dealing with recursive structures.

[glitchmr@pineapple ~]$ mkdir recursion
[glitchmr@pineapple ~]$ cd recursion/
[glitchmr@pineapple recursion]$ ln -s . recursion
[glitchmr@pineapple recursion]$ cd recursion
[glitchmr@pineapple recursion]$ pwd
/home/glitchmr/recursion/recursion
[glitchmr@pineapple recursion]$ which pwd
/bin/pwd
[glitchmr@pineapple recursion]$ /bin/pwd
/home/glitchmr/recursion
[glitchmr@pineapple recursion]$

The shell outright lies about your location. You’re in /home/glitchmr/recursion. Why shell does that? Well, it does that to not confuse you when typing cd ... And indeed, if you type cd recursion/recursion, and cd .. after that, you will end in recursion directory, not the directory where you created the first recursion directory.

Symbolic links in UNIX are simply redirects. If you made a symbolic link to /usr/share/hell, you shouldn’t expect that cd .. made after following link follow return you anywhere that isn’t /usr/share. Yet, implementation of symbolic links in bash lies.

The lie is obvious after using anything that isn’t bash builtin. For example, ls .. will always show contents of /usr/share, even if you made symbolic link in /tmp/heaven. Yet, cd .. will return to /tmp/heaven.

By the way, in case you ask, cd is also a builtin. But that’s acceptable, because external command cannot change environment of caller (such at PATH). So it has to be a builtin. Yet, somehow, some people are still confused.

id='z_shell'>Z shell>

Z shell is a shell that is way better than bash. Well, if you configure it properly. It’s a shell that appears to have some compatibility with Bash scripters (for example $'' syntax works, but $"" doesn’t (or rather, it returns literal ’$’ and contents of "")).

Z shell has lots of features. In fact, the author is “not aware of a major interactive feature in any other freely-available shell which zsh does not also have (except smallness)”. Something that would be sane default actually isn’t default. In fact, with default settings it mostly works like bash.

For example, completion. By default, Z shell only completes file names and command names. If you will type sudo apt-get install, press space and then press Tab, it will give you file names. Useful? I don’t think so.

Of course, the option to enable better Tab completion is possible to turn on. It’s just not default. Why the shell would have default that don’t make sense. Simiarly, by default Tab completion cannot expand globs. Yet, it’s possible to enable.

When you start Z shell for first time, you see the configuration wizard (called zsh-newuser-install). The problem is, it’s the most non-user-friendly thing I’ve seen. It asks about minor details you don’t care about, without proposing any useful default. For example, the maximum number of errors you can make for the command to be still proposed in error message. I just have pressed Enter, and the response is “Please enter a number”. Do you think I care about this?

Actually, as far I know, nobody actually uses this wizard. In most cases, zsh is configured using so called oh-my-zsh. What it is actually? Well, it’s a config script with sane things configured by default. Yet, it has got 8,648 stars on GitHub and 3,326 forks as I’m writing it.

Yes, it’s just ZSH config file that has things that should be default, but they aren’t. Oh, and plugins array that contains list of plugins that mostly make aliases and completion adders that aren’t in main zsh (like CoffeeScript). For example git plugin adds aliases like gc for git commit -v.

I don’t know about you, but I don’t like programs which take 4 hours to configure. Z shell is one of these. oh-my-zsh makes the job easier, but I still don’t think it’s worth wasting the time. Perhaps if fish wouldn’t exist, it would be worth it.

id='friendly_interactive_shell'>friendly interactive shell>

Z shell, aside of totally insane configuration isn’t a bad shell (if I would want to talk about bad shells, it would be csh or cmd.exe). friendly interactive shell is as good as Z shell (with good Z shell configuration), except without having to configure it.

In fact, most of things cannot be configured. You cannot configure nonsense like maximal history length (by design). You cannot configure interpretation of numbers starting with 0 as octal (you can in Z shell - I have no idea why Z shell even allows you to type “echo 010” to get “8” when configured to do so - but it does). You cannot disable file globs.

Remember .bashrc/.zshrc configuration file? You won’t need it. The colors and themes can be easily configured using a web service (but if you prefer, you can use set command directly).

For example, if you want to configure your EDITOR to be vim, you can run the following command.

set -Ux EDITOR vim

The -U modifier means that this value is stored between session. The -x means that this variable is exported to programs ran by shell. After you have done that, your variable magically appears in every fish session.

Adding new functions is also easy. It’s enough to type funced function-name. After

Fish also offers features I haven’t seen in other shells, like syntax highlighting. Ommited quote? Syntax highlighting will help you.

In fish, everything is tab-completeable. Even if the program doesn’t have specific tab completion definitions, it’s still tab-completeable, thanks to generating tab completions from man pages. For example, gedit doesn’t have specific tab completions, yet fish managed to create them.

glitchmr@pineapple ~> gedit -
--background                                         (Run gedit in the background.)
--encoding        (Set the character encoding to be used for openi… [See Man Page])
--geometry                     (Set the X geometry window size (WIDTHxHEIGHT+X+Y).)
--help                                           (Prints the command line options.)
--list-encodings  (Display list of possible values for the encodin… [See Man Page])
--new-document            (Create a new document in an existing instance of gedit.)
--new-window       (Create a new toplevel window in an existing instance of gedit.)
--standalone                                        (Run gedit in standalone mode.)
--version                                    (Output version information and exit.)
--wait                                    (Open files and block the gedit process.)
-b                                                   (Run gedit in the background.)
-g                             (Set the X geometry window size (WIDTHxHEIGHT+X+Y).)
-s                                                  (Run gedit in standalone mode.)
-w                                        (Open files and block the gedit process.)

Of course, just like zsh (with correct configuration), it can expand more stuff. For example, kill command expands process IDs.

glitchmr@pineapple ~> kill 
1            (systemd)  127           (systemd-udevd)  479                (gconfd-2)
2           (kthreadd)  132         (systemd-journal)  508                   (gvfsd)
3        (ksoftirqd/0)  229               (hd-audio0)  512              (gvfsd-fuse)
5       (kworker/0:0H)  230               (kpsmoused)  527         (cinnamon [tty1])
7       (kworker/u:0H)  299              (irq/46-mei)  528                  (colord)
8        (migration/0)  307                (cfg80211)  534  (polkit-gnome-au [tty1])
9        (rcu_preempt)  308                    (hci0)  536          (dropbox [tty1])
10            (rcu_bh)  309            (kworker/u:1H)  539        (nm-applet [tty1])
11         (rcu_sched)  310               (scsi_eh_6)  540  (gnome-screensav [tty1])
12        (watchdog/0)  311         (rts5139-control)  558      (gsd-printer [tty1])
13        (watchdog/1)  312         (rts5139-polling)  566         (gvfs-udisks2-vo)
14       (ksoftirqd/1)  314                (ttm_swap)  568                 (udisksd)
15       (migration/1)  325          (NetworkManager)  598             (gvfsd-trash)
17      (kworker/1:0H)  329             (dbus-daemon)  728          (firefox [tty1])
18            (cpuset)  331          (systemd-logind)  744          (hexchat [tty1])
19           (khelper)  332                   (login)  746      (pxgsettings [tty1])
20         (kdevtmpfs)  335                 (polkitd)  797           (dconf-service)
21             (netns)  341                  (mysqld)  798   (gnome-terminal [tty1])
22       (bdi-default)  344          (wpa_supplicant)  805  (gnome-pty-helpe [tty1])
23           (kblockd)  348                   (nginx)  806            (fish [pts/0])
26        (khungtaskd)  349                   (nginx)  1374            (kworker/1:2)
27           (kswapd0)  351                 (php-fpm)  5790           (fish [pts/1])
28              (ksmd)  352                 (php-fpm)  10502        (gvfsd-metadata)
29        (khugepaged)  353                 (php-fpm)  13551          (fish [pts/2])
30     (fsnotify_mark)  373                (dhclient)  13635           (ssh [pts/2])
31            (crypto)  385             (fish [tty1])  13866           (kworker/1:0)
35          (kthrotld)  387                   (fishd)  18892            (gvfsd-http)
38           (deferwq)  412           (startx [tty1])  20044          (fish [pts/1])
62             (khubd)  429            (xinit [tty1])  20046        (python [pts/1])
63           (ata_sff)  430                       (X)  20405           (vim [pts/0])
64         (scsi_eh_0)  434    (gnome-session [tty1])  21370           (kworker/0:1)
65         (scsi_eh_1)  437      (dbus-launch [tty1])  21373           (kworker/u:2)
66         (scsi_eh_2)  438             (dbus-daemon)  22399          (fish [pts/3])
67         (scsi_eh_3)  440         (at-spi-bus-laun)  22472           (vim [pts/3])
68         (scsi_eh_4)  444             (dbus-daemon)  22521           (kworker/u:0)
69         (scsi_eh_5)  447         (at-spi2-registr)  22863           (kworker/0:0)
93      (kworker/0:1H)  455  (gnome-settings- [tty1])  22945           (kworker/u:1)
95      (kworker/1:1H)  464              (pulseaudio)  23238            (ps [pts/4])
103      (jbd2/sda6-8)  465            (rtkit-daemon)  23239          (grep [pts/4])
104  (ext4-dio-unwrit)  468                 (upowerd)  23240          (tail [pts/4])
110        (flush-8:0)  476            (gconf-helper)

Of course, fish has some disadvantages. One of disadvantages is the syntax. It’s not POSIX compatible by design. For example, the history substition doesn’t exist. Instead, you’re supposed (if you wanted, let’s say, sudo) to press Up, Home, and type ‘sudo’. Command substitution uses (), not `` or $(). Still, if you will accidentally use $() bashism, the shell will inform you with an error.

Also, the newest stable is really old and slow. Instead I would use version from git, which is usually more stable than stable version. In Arch Linux, it’s fish-shell-git from AUR, on other operating systems (like CentOS I use on my VPS), you have to compile it yourself. It isn’t so difficult. The GitHub repository is fish-shell/fish-shell.

As for why I’m mentioning fish shell, well, I have lately contributed a small patch to hint -C suboptions in perl - like -CSDL. Nothing special, but hey, why not make a blog post, considering I write them rarely.

perl6.announce: Parrot 5.1.0 "Zombie Parrot" Released! by Bruce Gray

Wed, 20 Feb 2013 18:36:17 +0000

Flat on the bunk again, he ran for his life. The Parrot stalked him
through the grey hours of morning, smoothing its fractal feathers,
shuffling itself slowly into clarity as though at the end of a
flashy film-dissolve, until at last his mind's eye had to acknowledge
a shape,
a shape,
a wink
-- From BLIT, a short story by David Langford
http://www.infinityplus.co.uk/stories/blit.htm

On behalf of the Parrot team, I'm proud to announce Parrot 5.1.0, also known
as "Zombie Parrot". Parrot (http://parrot.org/) is a virtual machine aimed
at running all dynamic languages.

Parrot 5.1.0 is available on Parrot's FTP site
(ftp://ftp.parrot.org/pub/parrot/releases/supported/5.1.0/), or by following the
download instructions at http://parrot.org/download. For those who would like
to develop on Parrot, or help develop Parrot itself, we recommend using Git to
retrieve the source code to get the latest and best Parrot code.

Parrot 5.1.0 News:
- Core
+ The .sort() method was added to the FixedFloatArray PMC
+ Improved detection of system memory for machines with >2GB
+ Improved pbc_to_exe support for spacey paths
+ Fixed Parrot_io_readall_s allocating too much string space
- Build
+ Fixed generated MANIFEST files to omit $destdir
- Documentation
- Tests
+ .readall now checks that prior reads are respected.
- Community
+ Weekly IRC meetings have resumed. #parrotsketch Tuesdays at 1930 UTC

The SHA256 message digests for the downloadable tarballs are:
af26c2fcc806505ec516ebb013bdd37b218633f5fe63faaa6b843ffe55e0135e parrot-5.1.0.tar.bz2
2483963c1bec665be772cb40a71fd3d9d2621feca547932475017c81a2f7e49b parrot-5.1.0.tar.gz

Many thanks to all our contributors for making this possible, and our sponsors
for supporting this project. Our next scheduled release is 19 Mar 2013.

Enjoy!

--
Bruce Gray (Util of PerlMonks)

Jonathan Worthington (6guts): NQP on JVM gets Grammars, Multiple Dispatch

Sat, 16 Feb 2013 23:16:22 +0000

Having just reached an interesting milestone, I thought I’d blog a quick progress update on my work to port NQP to the JVM, in preparation for also doing a port of Rakudo Perl 6.

The big news is that the grammar and regex engine is pretty much ported. This was a sizable and interesting task, and while a few loose ends need to be wrapped up I think it’s fair to say that the hard bits are done. The port includes support for the basics (literals, quantifiers, character classes, positional and named captures, calls to other rules, and so forth) as well as the more advanced features (transitive Longest Token Matching for both alternations and protoregexes, and embedded code blocks and assertions). Missing are anchors, conjunctions and a small number of built-in rules; none of these are particularly demanding or require primitives that don’t already exist, however. It’s also worth pointing out that the NQP code to calculate the NFAs used in Longest Token Matching also runs just fine atop of the JVM.

Another interesting feature that I ported a little while ago is multiple dispatch. This was some effort to port, since the original implementation had been done in C. While it’s sensible to have a close-to-the-VM dispatch cache, there’s little reason for the one-off candidate sorting work (not a hot path) to be done in C, so I ported the code for this to NQP. This meant that on the JVM side, I just needed to implement a few extra primitives, and could then run the exact same candidate sorting code.

I think it’s worth noting again that I’m really doing two things in parallel here: hunting down places where NQP couples too tightly to Parrot and loosening the coupling, and also doing the porting to the JVM. The first half of this work is relevant to all future ports. In many cases, I’m also finding that the changes give us architectural improvements or just cleaner, more maintainable code. I wanted to point this out especially because I’m seeing various comments popping up suggesting that Rakudo (or even Perl 6) is on a one-way road to the JVM, forsaking all other platforms. That’s not the case. The JVM has both strengths (mature, a solid threading story, widely deployed, the only allowed deployment platform in some development shops, increasing attention to supporting non-Java languages through things like invokedynamic) as well as weaknesses (slow startup time, lack of native co-routine support, and the fact that it was originally aimed at static languages). Rakudo most certainly should run on JVM – and it most certainly should run on other platforms too. And, as I wrote in my previous post, we’ve designed things so that we are able to do so. Perl has always been a language where There’s More Than One Way To Do It. Perl also has a history of running on a very wide range of platforms. Perl 6 should continue down this track – but the new reality is that a bunch of the interesting platforms are virtual, not hardware/OS ones.

By now, the JVM porting work is fast approaching a turning point. Up until now, it’s been about getting a cross-compiler and runtime support in place and working our way through the NQP test suite. This phase is largely over. The next phase is about getting NQP itself cross-compiled – that is, cross-compiling the compiler, so that we have an NQP compiler that runs on the JVM, supporting eval and able to run independently.

Carl Masak: Perl 6 is now half as old as Perl

Sat, 16 Feb 2013 18:00:54 +0000

Today Perl 6 is as old as Perl was when Perl 6 was announced.

$ perl6 -e 'say Date.new(2000, 7, 18) + (Date.new(2000, 7, 18) - Date.new(1987, 12, 18))'
2013-02-16

The 1987 date is the release of Perl 1. The 2000 date is the throwing of the mugs, which I consider to be Perl 6's birthday.

It's a bit interesting to compare Perl 6 at this point with Perl back then. They are two fairly different projects, even though the people are overlapping to a great extent.

Bottom-up vs top-down

In any programming project, you can start from the small pieces and build upwards to the overreaching goals and ideas, or you can start from the ideas and build downwards to the nitty-gritty stuff.

Perl is a bottom-up project: the Perl 1 release looks puny today. It didn't do much. But it did run. It did solve people's problems. And there's an unbroken chain of commits leading from Perl 1 to today's Perl 5.

Perl 6 is definitely a top-down project. Larry mulled over those RFC's and wrote the Apocalypses. These eventually resulted in the Synopses, which guide implementation work. Implementations reach up towards the spec, and are in some sense always built bottom-up... but on the most zoomed-out scale, Perl 6 is built from the top down.

This difference was very much deliberate. For the Perl 6 project, it was felt that a specification (and a corresponding test suite) was needed. The Pugs project is currently very dormant and not actively developed, but it did result in both the Synopses and the spectest suite. Both are essential artifacts for any Perl 6 implementer.

Second system syndrome

"It's important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time." — Joel Spolsky, Things You Should Never Do, Part I

Look, here's the thing: everyone knows that The Big Rewrite isn't a good idea. That's like, Software Project Management 101. Even the people going into this project knew that. The jokes about second systems have been with us from the start, as a part of our cultural heritage.

There have been efforts to mitigate the risks and to calm people on the way. The Ponie project was an effort to put Perl 5 on Parrot, for example, to provide for a migration path or simply a communication bridge from Perl 5 to Perl 6.

Early on, it was also felt that Perl 6 wouldn't be so different from Perl 5, syntactically. Sure, the sigils would come out being invariant, and a comma would have to be added here and there for consistency. But that was about it. Perl 5 programmers would still feel right at home.

Well, guess what? The Ponie project, even with brilliant developers behind it, slowed and finally halted at the height of the Pugs era. Turns out the reasons people wanted to further themselves from the existing Perl 5 internals — that they are a big hairy mess of intertwined C macros — also made porting to Parrot exceedingly difficult. Ponie brought some permanent improvements to the Perl 5 core, but in the end it didn't reach its goal of Perl 5 targeting Parrot.

In the meantime, Perl 6 kept improving and evolving, syntactically as well as semantically. The differences piled up. Perl 6, unfettered by backward-compatibility, could take sometimes vast leaps and reach places in the state-space Perl 5 could only dream of. Also, things kept shaking around and stabilizing, features growing together into a unified whole instead of this. All good and well, but it meant that Perl 6 drifted further away from Perl 5.

So here we are, a decade later, with two distinct languages, and no way for them to interoperate. Having them actually talk to each other is still very much on the agenda. It's just that it's a big undertaking. A couple of recent projects are attempting to put Perl 5 on a platform where it can talk to Perl 6.

I think the biggest thing to realize in all this is that Perl 6, even from the very start, could never have followed the trajectory Perl did. There was one thing that existed in 2000 that didn't exist in 1987, when Perl was announced: Perl. That's why Perl 6 is a top-down project. That's why it's a second system. And that's why the obvious measure-stick of Perl 6 is Perl, with its 13-year head start.

Rubber, meet road

I'm not complaining about this, mind. We should compare Perl 6 to Perl. And to all other scripting languages that have popped up in the same niche. We should steal ideas and adapt to modern practices, like we always do, in both Perl and Perl 6.

But for a top-down project, the biggest challenge is always to reach all the way down to the ground: to actually start being useful for someone. That's perhaps our big challenge. It was back in 2008 when I became heavily involved in the Perl 6 effort, and it still is today: be useful. Be usable. Put food on someone's family.

Perl 6 is useful to me today. Not to the extent that I can write Perl 6 code all day, but to the extent that it makes me more effective and more productive now and then. Bringing that kind of usefulness to others requires lots of work writing modules, documentation, books, and tutorials, as has been discussed elsewhere. We're making some headway with this. I'm more optimistic than I was two years ago.

We also need to work on things such as performance. Perl 6 is fast enough for some things, but overall implementations are still fairly slow. Sometimes ridiculously slow. Good work is being done in this area, too.

But here's what makes me the most optimistic about the Perl 6 effort: after a few years of watching things evolve, I've noticed that while Perl 6 is being developed top-down on the outermost scale, it's actually a series of bottom-up projects that drive Perl 6 forwards.

jnthn likes to tell about how he promised to implement junctions in Rakudo back in 2008, and then realized that junctions were actually tied to method dispatch and the object system, so he had to implement those, too. Later, while pmichaud was rebuilding the parser according to our current understanding of it, jnthn was building Rakudo's meta-object protocol, a project called 6model. Each step on the way replacing the layer below based on what we had found that the layer above required. The 6model work is part of what now enables us to port nqp and Rakudo to the JVM.

That's what makes me optimistic. While Perl 6 is undeniably, unchangeably a top-down project, highly competent people are factoring that top-down knowledge into the design of components in a bottom-up way. What's been happening in these 13 years is that we've become increasingly better and more efficient at building Perl 6.

So, how're we doing?

"Chuck Norris has actually been using Perl 6 since 1987, and has been waiting for Larry to play catch-up. :)" — dukeleto on #perl6

Let's stop and compare the state of Perl 6 today with the state of Perl back in 2000.

Mugs. Back then, Jon Orwant threw coffee mugs against a wall because he felt there was nothing to energize the community and people would walk off to other things.

Are sixers doing any mug-throwing nowadays? Yup, we did, at the Perl Reunification Summit. Liz felt, and still feels, that if we don't actively bring Perl 5 and Perl 6 together, then both projects are eventually going to fizzle out.
Crisis. As part of that, the feeling Jon Orwant was expressing was that Perl was stagnating or going nowhere.

Is Perl 6 stagnating? I wouldn't say so. We're still extremely small compared to Perl 5, but my general sense is that we're still gaining speed, becoming stronger every day. Still very much pre-peak.
Usefulness. In 2000, Perl was on its thirteenth year of solving everyday problems for people.

What about Perl 6 today? Well, it's very different. Part of this is due to the top-down nature of the project. People are waiting for it to be useful to them. Part of it is perception: people are waiting, instead of jumping in and using the language in cases where it could already be useful to them. We see this in the verb tenses people are using when they ask stuff: "Will Perl 6..." Perl 6 is about the future, but if we don't do something about that perception, we'll never get there. The problem with "tomorrow" is that it never actually arrives.
Community. In 2000, Perl drew thousands of people to conferences each year.

Is Perl 6 drawing thousands? No. The Perl people are kind enough to let us piggyback on Perl conferences and workshops, and through them we have an audience of thousands. I keep feeling that there's a big interest in Perl 6, both from within the Perl community and from other communities. But there's also a sense of waiting, of not-yet-production-ready. (And I agree.)

What remains?

Which brings us to the last, perhaps most important question. What's left for Perl 6 to become a viable, useful solution to most people out there?

For years I wouldn't go near that question. Just working along, head looking down at the current sub-projects, making Perl 6 more useful to myself and hoping that was enough. But my recent visit to FOSDEM made me realize that some kind of production release is actually within reach — say, a few years away — and a focused effort to reach a releasable state by some criteria would be highly useful for us and for others.

So here are my for criteria. This is what Perl 6 needs to be ready for the world.

Features. Slightly simplified, this is what we've been working on since the Pugs days. We are doing well on features. There's not much I feel is missing nowadays from the core language. It's a pleasant language to use and to express thoughts in. There are bugs, sure, but with each month that passes, those bugs are becoming less frequent and less serious.
Speed. Working on it. I won't make any promises about the JVM port, but I have a strong feeling it will make a big difference. Parallel to this, there are various already-implemented or to-be-implemented efforts to make the compiler smarter about generating efficient code.
Concurrency. There are some good ideas in the specification about concurrency. Some of them will work fine, some probably won't pan out. What's needed now is a solid implementation of concurrency in either Rakudo or Niecza, a test-bed for the ideas in the spec, so that users can get a feel for what's there and what's missing. I'd really like for Perl 6 to have a decent concurrency story, because it's one of the things that were promised early on for Perl 6, and one of the things that Perl never got right. Also, concurrency in all its various forms seems to be growing increasingly important in the programming world of today.
CPAN. Sure, we have an ecosystem for Perl 6. But it isn't CPAN. It's three orders of magnitude as small as CPAN. And it's not realistic to port all of CPAN either — that'll simply never happen. What we really need is a high-bandwidth connection between Perl 5 and Perl 6. We need for them to run in the same runloop. Once they do that, Perl 6 programmers can have their CPAN modules, and Perl 5 programmers can work Perl 6 code into their projects. This is also something that was promised early on. It's not an easy task.

Conclusion

Perl 6 is a multi-year project. Though today's date is a little bit arbitrary, it's worthwhile to look both backwards and forwards, to see where we currently are. And though we're structurally different from the Perl project, it's still interesting to make comparisons.

We believe we're building something really nice with Perl 6. 2013 may not be the year when we're finally production-ready, but it sure feels like a year where a lot of significant things will happen (and are already happening). And, unlike 2012, I finally feel ready to speculate about the light at the end of the tunnel.

Konrad Borowski: Perl 6 changes - 2013W07

glitchmr@myopera.com (Konrad Borowski) — Sat, 16 Feb 2013 08:00:00 +0000

> my $perl = Date.new: '1987-12-18' # Perl 1 release^[1]
1987-12-18
> my $perl6 = Date.new: '2000-07-18' # Perl 6 announcement^[2]
2000-07-18
> Date.today - $perl6
4596
> $perl6 - $perl
4596

Normally, I wouldn’t put irrelevant snippets of code^{[citation
needed]}. But this day is different - we have Perl 6 equidieversary (thanks, moritz). It’s 4596 days since Perl 6 was announced (Wikipedia says it was annouced 2000-07-19, but actually it was earlier). In 2000-07-19, Perl was 4596 days old. But I won’t discuss about this more, as masak already made perfectly fine article about that.

Let’s return to your regularly scheduled program.

id='new_features'>New features>

Perl 6 specification>

Sets and Bags do not flatten in list context.
Metaoperators are treated as normal operators, and aren’t making list of closures. So, for example @array X* * is -> $arg { @array X* $arg }.
Inner classes and roles must be lexically scoped.
Added List.combinations and Set.combinations.

Added Set.powerset. It’s defined like that.

multi method powerset { set ∅, self.combinations }

Rakudo Perl>

`$_` is writable in `-n`/`-p` code (one-liner).

`eqv` is faster for `Buf` objects.

Configure script fails on unknown command line options.

Moritz Lenz (Perl 6): Pattern Matching and Unpacking

Wed, 13 Feb 2013 05:49:13 +0000

When talking about pattern matching in the context of Perl 6, people usually think about regex or grammars. Those are indeed very powerful tools for pattern matching, but not the only one.

Another powerful tool for pattern matching and for unpacking data structures uses signatures.

Signatures are "just" argument lists:

sub repeat(Str $s, Int $count) {
    #     ^^^^^^^^^^^^^^^^^^^^  the signature
    # $s and $count are the parameters
    return $s x $count
}

Nearly all modern programming languages have signatures, so you might say: nothing special, move along. But there are two features that make them more useful than signatures in other languages.

The first is multi dispatch, which allows you to write several routines with the name, but with different signatures. While extremely powerful and helpful, I don't want to dwell on them. Look at Chapter 6 of the "Using Perl 6" book for more details.

The second feature is sub-signatures. It allows you to write a signature for a sigle parameter.

Which sounds pretty boring at first, but for example it allows you to do declarative validation of data structures. Perl 6 has no built-in type for an array where each slot must be of a specific but different type. But you can still check for that in a sub-signature

sub f(@array [Int, Str]) {
    say @array.join: ', ';
}
f [42, 'str'];      # 42, str
f [42, 23];         # Nominal type check failed for parameter '';
                    # expected Str but got Int instead in sub-signature
                    # of parameter @array

Here we have a parameter called @array, and it is followed by a square brackets, which introduce a sub-signature for an array. When calling the function, the array is checked against the signature (Int, Str), and so if the array doesn't contain of exactly one Int and one Str in this order, a type error is thrown.

The same mechanism can be used not only for validation, but also for unpacking, which means extracting some parts of the data structure. This simply works by using variables in the inner signature:

sub head(*@ [$head, *@]) {
    $head;
}
sub tail(*@ [$, *@tail]) {
    @tail;
}
say head <a b c >;      # a
say tail <a b c >;      # b c

Here the outer parameter is anonymous (the @), though it's entirely possible to use variables for both the inner and the outer parameter.

The anonymous parameter can even be omitted, and you can write sub tail( [$, *@tail] ) directly.

Sub-signatures are not limited to arrays. For working on arbitrary objects, you surround them with parenthesis instead of brackets, and use named parameters inside:

multi key-type ($ (Numeric :$key, *%)) { "Number" }
multi key-type ($ (Str     :$key, *%)) { "String" }
for (42 => 'a', 'b' => 42) -> $pair {
    say key-type $pair;
}
# Output:
# Number
# String

This works because the => constructs a Pair, which has a key and a value attribute. The named parameter :$key in the sub-signature extracts the attribute key.

You can build quite impressive things with this feature, for example red-black tree balancing based on multi dispatch and signature unpacking. (More verbose explanation of the code.) Most use cases aren't this impressive, but still it is very useful to have occasionally. Like for this small evaluator.

Konrad Borowski: Perl 6 changes - 2013W06

glitchmr@myopera.com (Konrad Borowski) — Sat, 09 Feb 2013 08:00:00 +0000

This time without any gimmicks (like five word sentences). This will be simple description of changes I do regularly.

id='new_features_this_time_without_building_stuff_part'>New features (this time without building stuff part)>

Rakudo Perl>

Constant folded `^..^` (infix) and `^` (prefix) operators.

Niecza Perl>

Added (borrowed from Rakudo Perl) the `Match.gist` method.

Replaced “Unhandle conext character” error with “Unhandle context character” error.

Added Cool.set method. It can be used to convert argument to set.

Added Cool.bag. Just like method above, it can be used to convert argument to bag.

Konrad Borowski: I have Twitter account

glitchmr@myopera.com (Konrad Borowski) — Wed, 06 Feb 2013 08:00:00 +0000

Yet another ignorable self promotion. http://twitter.com/GlitchMr

(perhaps I should have different news feed for Planet Six…)

Konrad Borowski: Perl 6 changes article - 2013W05

glitchmr@myopera.com (Konrad Borowski) — Sat, 02 Feb 2013 08:00:00 +0000

The changes are so fun. jnthn is working on porting. nqp will be on JVM. And then, Rakudo will be. Java is very, very fast. There are some interesting problems. But nothing impossible to solve. At least, I hope so. I would continue, but well. This is not the topic. This is article about changes. Changes in Perl 6, obviously. If you want, enjoy reading.

Rakudo Star 2013.01 was released. It’s based on Rakudo 2013.01. Those changes aren’t in.

On sidenote, a small note. Five word sentences are interesting. I am not stealing ideas. Well, actually, I am now. I should stop talking now.

id='new_features_to_build_stuff'>New features to build stuff>

Rakudo, a Perl 6 distribution>

`any` and `all` are optimized. To be exact, in statements. The `if` and `while` statements. And opposites - `unless` and `until`.

Many operators are pure now. That means they are sideeffect-free.

Site libraries are loaded first.

Pure subroutines are constant folded.

REPL supports natively typed variables.

REPL has better error reporting.

Jonathan Worthington (6guts): A look at the preparations behind the JVM port, and a progress update

Fri, 01 Feb 2013 23:49:17 +0000

After my last post giving a status update on the JVM porting of NQP and the compiler toolchain Rakudo builds upon, hercynium++ left a comment suggesting that I also blog about the design work behind this effort. I liked the idea, and in this post I’ll attempt to describe it a bit. I can’t possibly capture all of the interesting things in a single post, so if this doesn’t cover aspects that are particularly interesting to anybody reading, let me know and I’ll try and find time to write something on them. :-)

It started long ago…

The first commit to the repository where I’m doing the initial porting work to the JVM may have been back in November, but that isn’t where the journey really started. We’ve known for multiple years now that we would want Rakudo and NQP to target backends besides Parrot. In that time, we’ve had to build a lot of technology in order to be able to build Rakudo at all. Some things we’ve had to build more than once because the first time didn’t produce something satisfactory (where satisfactory means “actually meets our needs”, not “is the most awesome thing ever possible”). Software is, fairly often, as much about learning as it is about building. The more complex the domain you’re working in, there more this applies, and the more likely it is that you’ll have to build one to throw away. By now we’ve thrown away a parser engine, an AST, and about 3 implementations of roles. :-)

Of course, there’s the build/buy thing, where buy in open source really means “buy into”, as in use an existing library. We’ve done a bunch of that too, such as libtommath for our big integer support and dyncall for NativeCall support. But the closer something is to the “core domain” – the thing that makes your product distinctive and special – the less able you are to use something off the shelf. Parsing Perl 6 really needs to be done with a Perl 6 grammar, using Longest Token Matching. Its object system really needs something that supports meta-programming, representation polymorphism and gradual typing. Getting BEGIN/eval right and supporting compilation and having the possibility for lexical and anonymous types and packages, which can be dynamically constructed and exported, also left us with something to build (this is the work that led to bounded serialization).

Eventual portability has been a design factor in what we’ve built for quite a while. While the only 6model implementation to have become complete enough to support all of Rakudo’s object needs so far is the one running on Parrot, the initial prototypes of 6model were done on the .Net CLR. This was in no small part to make sure that there was a feasible way to implement it on such a VM. Granted, what I actually discovered was a less than awesome way to build it on the CLR (and what I’m doing on the JVM this time around fits in far better with the JVM’s world view). But it was a design consideration from the start.

When we updated PAST, the previous AST representation, to QAST (Q is just P++ :-)) then once again portability was a concern; the VM specific bits were all placed under a QAST::VM node type. This makes it easy to escape to the underlying VM where needed or where it is most expedient, but it’s both explicit and done in a way that allows specification of what to do on other backends. As part of this work we also build support for the nqp::op abstraction directly into the AST format. The nqp::ops form an opcode set independent of any particular VM. These get mapped as part of turning a QAST tree into code for the target backend (thus meaning there’s no overhead for them in the generated code). They may map directly to the VM’s opcodes, a function or method call in the VM, or do some more complex code generation.

The other important piece of the groundwork for portability is that we implemented Rakudo in a mixture of Perl 6 and NQP, and over time have got NQP to the point where it is also written in NQP (and thus can compile itself). This has been a gradual thing; the earliest NQP editions were written directly in PIR, and with time we’ve migrated those bits to NQP – usually at the same point we were doing other improvements already. For example, pmichaud++ wrote the latest edition of the regex engine, with LTM support, in NQP. PAST, written in PIR, was replaced by QAST, written in NQP. And 6model’s meta-objects were, from the start, expressed in NQP too. It’s pretty neat that NQP’s definition of things so fundamental as classes is actually written in NQP. It means that we don’t have to port classes and roles, just the primitives they are made out of.

So digging into the JVM port itself…

With all of the above mentioned things in place, it was possible to form a fairly concrete roadmap for porting NQP, then Rakudo, over to the JVM. Being comfortable that the result would enable us to get a fully functional Rakudo on the JVM and an idea of how to get there was important. It’s easy to implement a subset, but if it isn’t factored in a way that lets you do the rest afterwards then you’re in bother and it’ll be time for re-work. My hope was that, after some years of learning about things that don’t work and replacing them with things that do, this time much of the re-work could be avoided. A starting point for this was taking a good look at the JVM’s instruction set, as well as considering what JVMs are typically good at doing.

The JVM is a stack machine. This is in contrast to Parrot, which is a register machine. Thankfully, this is mostly a code generation detail rather than being especially deep. As well as the stack, a given method can have local variables (note that everything that contains code on the JVM is called a method, even subroutines, but they call them static methods because it sounds so much more OO :-)). These can hold longer-lived things, so in a sense could be used a bit like Parrot registers. In general, the code generation from QAST uses the stack where possible and falls back to locals where needed. This is because stack usage fits well with what a JVM expects to be doing, and also what its bytecode format expresses most compactly.

Locals have an important restriction: they can only be accessed directly in the scope where they are declared. There is no notion of nested methods at the JVM level. This means that locals are not suitable for implementing lexical variables. Thankfully, there is a well established solution: promote such things onto the heap, keeping them in some kind of invocation record. This is what happens with closures in C# on the CLR, for example. There are a bunch of ways to do this transform, with various trade-offs. I’ve done one that was fairly fast to implement, but also enables lookup by array indexing rather than needing a named (hash) lookup in the vast majority of cases. As well as an array index being algorithmically cheaper than a hash lookup, the JVM supports array indexing natively in its opcode set, but not hash lookups.

Talking of allocating things on the heap brings us nicely to think about objects. JVMs are very good at fast allocation and collection of objects, because they have to be; there is no stack allocation in Java of anything non-trivial. Of course, that doesn’t mean the VM can’t do escape analysis and stack allocate under the hood. That the VM is really good at object allocation and GC means we don’t need to worry too much about lexicals leading to invocation records on the heap; there’s plenty of performant uses of this approach in the wild. Furthermore, most records will be very short lived, nicely meeting the generational hypothesis (which is that most objects are either short lived or long lived, and so we can optimize separately for each through doing generational garbage collection).

While invocation records are relatively internal, of course NQP and Perl 6 involve lots of user-visible objects. From the things you think about as objects (and call “new” on) to things like scalar containers, strings, boxed integers and so forth, both NQP and Perl 6 lead to plenty of allocations. While some things are quite predictably shaped, most come from user class definitions. Ideally, we’d like it if a Perl 6 class definition like:

class Point {
    has $!surface;
    has num $!x;
    has num $!y;
}

Was to use memory similarly to if you wrote something in Java like:

class Point {
    private Object surface;
    private double x;
     private double y;
}

At the same time, we know that the JVM’s idea of type is some way off the Perl 6 notion of type, so we can’t simply turn Perl 6 classes into JVM classes. Thankfully, 6model has from the start been designed around the idea of representation polymorphism. Really, this is just a separation of concerns: we decouple the details of memory representation and access from the notion of being a type and dispatch. The former is handled by a representation, and the latter two by a meta-object. One early but important observation I made when designing 6model is that the representation will always couple closely to the underlying runtime (and thus would need to be implemented for each runtime we wanted to run on), whereas the other bits can be expressed in a higher level way, with the common cases made efficient by caching. Thus there’s no reason to re-implement classes and roles per VM, but there is a need to provide a different, VM-specific way to do P6opaque (the default representation for NQP and Perl 6 objects).

The C implementation of P6opaque on Parrot works by calculating a memory layout – essentially, figuring out a struct “on the fly”. What’s the JVM equivalent of that? Well, that’s just creating a JVM class on the fly and loading it. Is the JVM capable of doing that? Sure, it’s easily dynamic enough. Furthermore, once we’ve done that little bit of bytecode generation, it’s a perfectly ordinary JVM class. This means that the JIT compiler knows what to do with it. Does doing any of this require changes to the meta-objects for classes in NQP and Rakudo? No, because these details are all encapsulated in the representation. Things like these are good signs for a design; it tends to show that responsibilities are correctly identified and segregated.

So, how’s the cross-compiler going?

Things are going nicely. Having got much of the way there with the NQP MOP, I turned to ModuleLoader and started to get together a basic setting (the setting being the place where built-ins are defined). With those in place, work has moved on to trying to pass the NQP test suite.

The build process cross-compiles the MOP, module loader and setting. To run the test suite, each test is taken and cross-compiled against those, then the result of compiling it is run on the JVM. The fact we invoke NQP, then invoke the JVM twice in order to run each test gives quite a bit of fixed overhead per test; once we have NQP itself (that is, the compiler) cross-compiled and self-hosting on the JVM it’ll be down to a single invocation.

The NQP test suite for the NQP language itself consists of 65 test files. 3 of them are specific to Parrot, so there’s 62 that are interesting to make run. As of today, we pass 46 of those test files in full. While some of those passing tests exercise relatively simple things (literals, operators, variables, conditionals, loops, closures), others exercise more advanced features (classes, roles, basic MOP functionality, runtime mixins and so forth). Of the 16 test files that remain, 9 of them depend on regexes or grammars. Getting those to run will be the focus of the next major chunk of work: porting the regex compiler and getting the NFA, Cursor and Match classes to cross-compile (which will involve some portability improvements). The other 7 relate to non-trivial, but smaller-than-grammars features (for example, 2 are about multiple dispatch, which I’m working on porting at the moment).

It was only three weeks ago when I wrote that the JVM port did not even manage “hello world” yet, and that I had little more to show than something that could turn a range of QAST trees into JVM bytecode. Three weeks later and we’re running around 75% of the NQP test files, and timotimo++ was even able to feed an almost unmodified Levenstein distance implementation written in NQP to the cross-compiler and have it run on the JVM.

So, mad amounts of coding have taken place? Well, only sorta…I’ve taught two three-day classes for $dayjob in the last couple of weeks also. :-) Mostly, progress has been fast now because the foundations it is building upon have proved fairly solid. For the backend, this is in no small part down to having grown a test suite for the QAST to JVM phase of the work as it progressed. The fact we could happily couple this new backend to the existing NQP parser is thanks to the compiler being structured as a pipeline of stages, each one strongly isolated from the others, just passing a data structure between them. In my teaching work, I often encourage automated testing and talk a lot about the importance of enforcing carefully chosen, well-defined, information-centric boundaries between components. It’s pleasing to see these things paying off well in my Perl 6 work also. :-)

Carl Masak: I am going to FOSDEM

Thu, 31 Jan 2013 23:22:38 +0000

(Using exclusively five word sentences.)

Perl Mongers needed speakers quickly. "Very Late Call for Papers". "Why so late", you ask. Perl dev room was denied. Another community got the room. Perl only got a booth. The other community backed out. Perl then got the room. Therefore, talks were requested urgently. Only about one week notice. The announcement is recorded here.

I missed that blog post. But I got an email. Wendy wrote to some people. I was one of them. Talk about very short notice. Eight days before the talk! That must be a record. Nothing to be done, though. The invitation was nicely worded. I considered whether to go. Finally I decided I would.

My talk concerns Perl 6. I have given it before. It was in Bristol, England. You were likely not there. That time, jnthn helped me. Now I will talk alone. I must give it quickly. I only have 20 minutes. That is not a lot. I rather like challenges, though. Looking forward to it all.

Will you come to FOSDEM? I certainly hope you will. If you do, stop by. I will give my talk. "Where is my flying car?" A reference to the future. In the future, cars fly. Also, Perl 6 is everywhere. Especially in the flying cars. It will be totally awesome. My talk is about that. Or sorta kinda about that. It is about Perl 6. Why is it not released? What makes me keep hoping? What has been implemented already? That is what it covers.

Looking forward to the weekend.

rakudo.org: Rakudo Star 2013.01 released

Wed, 30 Jan 2013 18:51:44 +0000

On behalf of the Rakudo and Perl 6 development teams, I’m happy to announce the January 2013 release of “Rakudo Star”, a useful and usable distribution of Perl 6. The tarball for the January 2013 release is available from the download page. A Windows .MSI version of Rakudo star will usually appear in the downloads area shortly after the tarball release.

In the Perl 6 world, we make a distinction between the language (“Perl 6″) and specific implementations of the language such as “Rakudo Perl”. This Star release includes release 2013.01 [0] of the Rakudo Perl 6 compiler [1], version 4.10.0 of the Parrot Virtual Machine [2], and various modules, documentation, and other resources collected from the Perl 6 community.

Some of the new features added to this release include:

Sink context (what some other languages call void context) is now enforced correctly. This means that for-loops are now lazy by default. It fixes the bug where a map in sink context would not execute, and also means that a Failure returned to sink context will be properly thrown.
‘require’ now works with indirect module names
Restored socket read semantics to returning the requested number of bytes
$obj.Some::Role::meth() now passes the correct $obj
try/CATCH now returns Nil when the CATCH is triggered, rather than the exception; this brings it in line with try without a CATCH
whatever-star cases of splice now implemented
can now import multis with the same name from different modules, provided all dispatchers are onlystar

This release also contains a range of bug fixes, improvements to error reporting
and better failure modes.

The following features have been deprecated or modified from previous
releases due to changes in the Perl 6 specification, and are being removed
or changed as follows:

postcircumfix:<[ ]> and postcircumfix:<{ }> will become multi-subs rather than multi-methods. Both at_pos and at_key will remain methods.
Unary hyper ops currently descend into nested arrays and hashes. This will change to make them equivalent to a one-level map.
The Str.ucfirst builtin is deprecated; it will be replaced by Str.tc.
Leading whitespace in rules and under :sigspace will no longer be
converted to <.ws> . For existing regexes that expect this conversion, add a <?> in front of leading whitespace to make it meta again.
The ?-quantifier on captures in regexes currently binds the capture slot to a List containing either zero or one Match objects; i.e., it is equivalent to “** 0..1″. In the future, the ?-quantifier will bind the slot directly to a captured Match or to Nil. Existing code can manage the transition by changing existing ?-quantifiers to use “** 0..1″, which will continue to return a List of matches.

There are some key features of Perl 6 that Rakudo Star does not
yet handle appropriately, although they will appear in upcoming
releases. Some of the not-quite-there features include:

advanced macros
threads and concurrency
Unicode strings at levels other than codepoints
interactive readline that understands Unicode
non-blocking I/O
much of Synopsis 9

There is an online resource at http://perl6.org/compilers/features
that lists the known implemented and missing features of Rakudo
and other Perl 6 implementations.

See http://perl6.org/ for links to much more information about
Perl 6, including documentation, example code, tutorials, reference
materials, specification documents, and other supporting resources.
A draft of a Perl 6 book is available as docs/UsingPerl6-draft.pdf
in the release tarball.

The development team thanks all of the contributors and sponsors
for making Rakudo Star possible. If you would like to contribute,
see http://rakudo.org/how-to-help, ask on the perl6-compiler@perl.org
mailing list, or join us on IRC #perl6 on freenode.

[0] https://github.com/rakudo/rakudo/blob/nom/docs/announce/2013.01
[1] http://github.com/rakudo/rakudo
[2] http://parrot.org/

Konrad Borowski: Perl 6 changes - 2013W04

glitchmr@myopera.com (Konrad Borowski) — Sat, 26 Jan 2013 08:00:00 +0000

Today, the main features is .delta method in DateTime and Date. And as usually, bug fixes that make language do what you mean.

id='new_features'>New features>

Perl 6 specification>

Added `DateTime.delta` and `Date.delta` to the specification.

Rakudo Perl>

`s///` resets `$/` on failure.

`s///` returns `False` on failure.

Regular expressions allow `$()` syntax for arbitrary expressions.

Socket functions work better with non-ASCII characters.

Errors about function only taking positional arguments now specify name of function that was called.

`DateTime.truncated-to` now takes `TimeUnit` enum as argument.

Implemented `DateTime.delta` and `Date.delta`.

Names are suggested for `--> Type` (subroutine return type).

`/< hello >/` now works correctly (matching `hello`), just like `/< hello world >/` already did.

Konrad Borowski: Helpful error messages

glitchmr@myopera.com (Konrad Borowski) — Sun, 20 Jan 2013 08:00:00 +0000

Rakudo lately got an interesting change. Consider following buggy code (for array binary search), written in functional style (I’m using recursion). The bug is that it doesn’t work because of two different identifiers binary-search and binary_search.

Trying to run it shows compile time error message. The compile part is important. If you would remove binary-search call at end, it still would report an error, unlike Python.

===SORRY!===
Undeclared routine:
    binary_search used at lines 2, 8, 9. Did you mean '&binary-search'?

Now you know what’s wrong and you can easily fix it. Unlike let’s say, Jekyll that I use for my blog. Not only it doesn’t work on my PC for some reason, but also refused to highlight my code - it gave some sort of XML error (it simply said “REXML could not parse this XML/HTML”). I gave up and put it on Gist. And the first attempt was failed, because embed code is <script> and Jekyll self-closed the tag (XML-style, but this is HTML). This is annoying.

Carl Masak: The Perl 6 Coding Contest (2012 edition) is now closed

Sat, 19 Jan 2013 17:24:06 +0000

"I've found this year the tasks were harder, but I've also spend more time thinking about the problems and less worrying about bugs in the interpreter... which is definitely good."
        — a contestant

"This year's problems were a lot more approachable to me and motivated me to learn more perl6."
        — another contestant

...aaaand we're done. The five weeks are up.

I'm now all caught up processing submissions sent to me. So let's summarize, as usual:

32 people signed up as contestants this year.
8 contestants submitted solutions.
24 submissions passed a base-test run and were accepted.

These figures during p6cc2010 and p6cc2011 were (18, 5, 26) and (35, 6, 27), respectively. So all in all, it seems that we've entered a steady state both in terms of contestants and submissions. I'm grateful to get so much interesting Perl 6 code to read through and review.

Many people told me this year that they had noble plans to send in lots of solutions, but in the end didn't find the time for it. I'm assuming many contestants I haven't heard from have similar stories. That's fine; the contest is designed to encourage you to do the tasks, but not to force you to. If the contest made you take a look at the tasks, or at Perl 6, then in some sense that's a win, too.

My plan is, just like previous years, to go through the five tasks in order, publish all the reviews and an appropriately thoughtful blog post that summarizes the tasks and its solutions. With luck, we'll all learn something together.

It takes time to do these reviews. Sometimes a lot of time. That said, I hope to get through these submissions quickly. Expect something like a post each week. That's the goal.

After which I'll sum up and select a winner, also in a blog post. Exciting!

So, expect the next blog post to be about the first task: "Tell knights from knaves based on what they say." Onwards!

Konrad Borowski: Perl 6 changes - 2013W03

glitchmr@myopera.com (Konrad Borowski) — Sat, 19 Jan 2013 08:00:00 +0000

It’s another Perl 6 changes article (at this point I wonder why I still do that). But as long the changes are impressive (and the cake is the lie), I guess mentioning changes is worth it.

Rakudo 2013.01 was tagged in the Git repository. It wasn’t yet released, but it’s really close to the releease (and frozen).

id='new_features'>New features>

Rakudo Perl (in Rakudo 2013.01)>

`** 0` in regexpes works correctly matching nothing.

Rakudo Perl (post Rakudo 2013.01)>

Compiler hints invalid variable names. For example, typing `$variabel` when you already have `$variable` declared hints that variable.

Niecza Perl>

More meaningful error message is thrown when `printf` tries to consume more arguments than available.

Jonathan Worthington (6guts): A quick JVM backend update

Fri, 18 Jan 2013 00:44:00 +0000

Things have been moving along quite rapidly on the JVM backend since my last post. Sadly, I’m too sick to hack on anything much this evening (hopefully, this turns out to be a very temporary affliction…) but I can at least just about write English, so I figured I’d provide a little update. :-)

Last time I blogged here, I was able to compile various QAST trees down to JVM bytecode and had a growing test suite for this. My hope was that, by some inductive handwaving, being able to compile a bunch of QAST nodes and operations correctly would mean that programs made up of a whole range of them would also compile correctly. In the last week or so, that has come to pass.

Having reached the point of having coverage of quite a lot of QAST, I decided to look into getting an NQP frontend plugged into my QAST to JVM backend. In the process, I found that NQP lacked the odd VM abstraction here and there in the common prelude that it includes with every QAST tree it produces. Thankfully, this was easily rectified. Even better, I got rid of a couple of old hacks that were no longer required. With those things out of the way, I found that this common prelude depended on a couple of operations that I’d not got around to implementing in the JVM backend. These were also simple to add. And…here endeth the integration story. Yup, that was it: I now had a fledgling NQP cross-compiler. An NQP compiler running on Parrot, but producing output for the JVM.

This result is rather exciting, because…

It’s using exactly the same parse and action stages as when we’re targeting Parrot. No hacks, no fork. The QAST tree we get from the NQP source code that goes in is exactly the one we get when targeting Parrot. Everything that happens differently happens is beyond that stage, in the backend. This is an extremely positive sign, architecturally.
With a couple of small additions to handle the prelude, I was immediately able to cross-compile simple NQP programs and run them on the JVM. There’s no setting or MOP yet, but the basics (variables, loops, subroutines with parameters, even closures) Just Worked.
The program I wrote to glue the JVM backend work and the existing NQP frontend together was about 30 lines of NQP code.
This whole integration process was about an afternoon’s worth of work.

Since I got that working, my focus has been on getting nqp-mo (the NQP meta-objects) to cross-compile. This is where classes and roles are implemented, and thus is a prerequisite for cross-compiling the NQP setting, which is in turn a prerequisite for being able to start cross-compiling and running the NQP test suite. The NQP MOP is about 1500 lines of NQP code, and at this point I’ve got about 1400 of them to cross-compile. So I’m almost there with it? Well, not quite. Turns out that the next thing I need to port is the bounded serialization stuff. That’s a rather hairy chunk of work.

Anyway, things are moving along nicely. The immediate roadmap is to get the bounded serialization to the point where it’s enough for the NQP MOP, then move on to getting a minimal setting cross compiling. Beyond that, it’ll be working through the test suite, porting the regex compilation and seeing what else is needed to cross-compile the rest of NQP.

Konrad Borowski: Perl 6 changes - 2013W02

glitchmr@myopera.com (Konrad Borowski) — Sat, 12 Jan 2013 08:00:00 +0000

I’m going to show another Perl 6 changes article. I’m so lazy, that I really don’t know what to put here, so I guess I’ll now show the list of new features.

id='new_features'>New features>

Perl 6 specification>

`-->` (return type) acts more like a normal subroutine parameter.

Added `squish` method, working like `uniq` utility in UNIX - it only removes elements if it is duplicate of previous element.

Adjusted precedence of set operators.

STD.pm6>

Adjusted precedence of set operators.

Rakudo Perl>

Implemented `require ::($module) 'something'` syntax.

`$socket.read($bytes)` now returns `$bytes` bytes, if possible. Previously it only read only 256 bytes.

`$object.My::Role::method()` works correctly and uses correct `self`.

`try` with `CATCH` returns `Nil` on exception, like it did without explicit `CATCH`.

Failures in sink context won’t throw if they were handled.

`splice` supports whatever star (`*`).

Adjusted precedence of set operators.

`Junction.Str` now gives string, instead of junction with every element converted to the string.

Niecza Perl>

Adjusted precedence of set operators.

Jonathan Worthington (6guts): A Bunch of Rakudo News

Thu, 10 Jan 2013 01:59:04 +0000

Seems it’s high time for some news here. It’s not that I didn’t do any blogging about Perl 6 in December; it’s just that all of those posts were over on the Perl 6 advent calendar. Anyway, now it’s a new year, and I’m digging back into things after an enjoyable Christmas and New Year’s break in the UK with family and friends. Here’s a bunch of things that already happened but I didn’t get around to talking about here yet, and some things that will be coming up.

Better Parse Errors In 2012.12

Ever had Rakudo tell you there’s a problem on line 1, when it’s really on line 50? Or wished that even in the common case where it gets the line right, it would tell you exactly where on the line things went wrong? Or how about the time it told you “Confused” because you got a closing paren too many?

Many of my contributions to the 2012.12 Rakudo release centered around improving its reporting of parse errors. STD, the standard Perl 6 grammar, has had much better error reporting than Rakudo for a while. Therefore, I spent a bunch of time aligning our error reporting more closely with what STD does. Some of this is cosmetic: you get the colored output and the indication of the parse location. But while these cosmetic changes will be the most immediately visible thing, the changes go far deeper. Of note, a high water mark is kept so we can be a lot more accurate in reporting where things came unstuck, and we track what was expected so we can produce better errors. Just doing the cosmetic stuff without being able to give it a better location to report wouldn’t have helped so much. :-)

One other change is that we don’t bail out on the first thing that’s wrong when it’s possible to survive and continue parsing. When this is possible, up to 10 errors will be reported (since that’s typically a screen worth). Of course, some things just hose the parse and we can’t continue in any sensible way.

Hopefully, these improvements will make using Rakudo feel a lot nicer. Already on channel, I can feel the feedback we’re giving about parse errors when people use the evalbot is often a lot more pleasant and informative. Of course, there’ll be more improvements in the future too, but this is a big step forward.

Faster Auto-Threading

The junction auto-threader could sometimes be insanely slow. As in, ridiculously so. After hearing a bunch of reports about this, I decided to dig in and work out why. A rewrite later, the little benchmark I was using with it ran almost 30 times faster. Not so bad… :-) This change also made it into the 2012.12 release.

JVM Backend Preparations Underway

I’ve talked plenty about plans for NQP and Rakudo to run on things besides Parrot for a while now. Over the last year or two, we’ve laid a lot of the groundwork for this. What’s been especially pleasing is that it’s also made Rakudo a better quality Perl 6 implementation on Parrot, thanks to the many architectural improvements. Of note, in many places we’ve closed semantic gaps between what Perl 6 wants and the primitives we were building it out of; the new QAST (Q Abstract Syntax Tree) is a great example.

Anyway, with NQP now being written pretty much entirely in NQP, and many of the right abstractions in place, it felt like time to start slowly picking away at getting 6model ported to the JVM and work on turning QAST trees into Java bytecode. I quietly started on this in November, and mentioned the existence of the work on #perl6 in December. I was delighted to see Coke++ jump in and start working through the Low Hanging Fruit – a file where I’m putting tasks that should be relatively easy to pick off. I actually had to re-fill it, after the last round were depleted. ;-) By now, quite a few bits of QAST are supported and the 6model on JVM implementation is coming along nicely. Yes, this means it’s already capable of doing basic meta-programming stuff.

Note that this work isn’t at the stage where it’s of use for anything yet. You can’t even write a say(“hello world”) in NQP and have it run on the JVM yet, since all the work so far is just about turning QAST trees into JVM bytecode and building the runtime support it needs. That may seem like a curious way to work, but once you do enough compiler stuff you find yourself thinking quite naturally in trees. It meant I didn’t have to worry about creating some stripped-down NQP that could emit super-simple trees to be able to test really simple things. After all, the goal is to run NQP itself on the JVM, and then Rakudo, and only then will things be interesting to the everyday user.

To address a couple of immediate concerns that some may have…

No, this is not a case of “stop running on Parrot, start running on JVM”. It’s adding an additional backend, much like pmurias++ has been working on adding a JavaScript backend for NQP. Of course, I expect resource allocation in the future to be driven by which backends users desire most. For some, the JVM is “that evil Oracle thing” and they don’t want to touch it. For others, the JVM is “the only thing our company will deploy on”. Thus I expect this work to matter more to some people than others. That’s fine.
No, targeting multiple backends doesn’t mean performance-sucking abstractions everywhere. It’s a pretty typical thing for a compiler to do. As usual, it’s about picking the right abstractions. The debugger was implemented as an additional Rakudo frontend without a single addition to Rakudo or NQP or anything anywhere in the stack. That was possible because things were designed well. I’m sure the process of getting things running on the JVM will flag up a few places where things aren’t designed as well as they need to be, but already I’m seeing a lot of places where things are mapping over very nicely indeed.
No, this doesn’t mean that all other Rakudo development will grind to a halt. I’ve been working on the JVM backend stuff in November and December; both months saw a huge amount of Rakudo progress too. Things will go on that way.

Type System Improvements

Rakudo does a lot of things well when it comes to supporting the various kinds of types Perl 6 offers, but there are some weak areas. Here are some of the things I plan to focus on:

Getting the definedness constraint stuff working better (the :D and :U annotations). At the moment, they’re supported as a special case in multiple dispatch and the signature binder. In reality, they’re meant to be first class and work everywhere. You may not think you care about these much. Actually, you probably at least indirectly do, because once the optimizer is able to analyze them, it’ll be able to do a bunch more inlining of things than it can today. :-)
Getting coercion types in place. Again, this is turning the special-cased “as” syntax into the much more general coercion type syntax (for example, Int(Str) is the type that accepts a Str and coerces it into an Int).
Getting native types much better supported. At the moment, you can use them but…there are pitfalls. Having them available has been a huge win for us in the CORE setting, where we’ve used them in the built-ins. But they’re still a bit “handle with great care”. I want to change that.
Implementing compact arrays.
Improving our parametric type and type variable support. Many things work, but there are some fragile spots and some bugs that want some attention.

I intend to have some of these things in the January release, and a bunch more in the February one. We’ll see how I get along. :-)

Konrad Borowski: Perl 6 changes - 2013W01

glitchmr@myopera.com (Konrad Borowski) — Sat, 05 Jan 2013 08:00:00 +0000

I have finally updated the number before ‘W’. Well, we have the new year - 2013. And because of that I would like to say ‘Happy new year!’.

This is yet another article in “Perl 6 changes”. I lately don’t put articles other than those, but I don’t have many ideas for those, really.

By the way, I find it interesting that no new posts on Planet Six appeared between my previous Perl 6 changes article and current Perl 6 changes article.

id='incompatible_changes'>Incompatible changes>

Rakudo Perl>

“SAFE” setting is now removed. You’re supposed to use “RESTRICTED” setting now.

id='new_features'>New features>

Rakudo Perl>

Using non-associative operators when association is needed throws typed error message.

Compile time optimizations for `(|)` signature (accepting anything), It’s worth handling because it’s commonly used for `proto`s.

Fixed typos in `X::Import::Redeclaration` error message.

Removed old `perl6doc` documentation.

`DateTime.Date` now works correctly.

`.pick` now randomizes better on large ranges.

Niecza Perl>

`infix:<->(Instant, Instant)` now supported.

Bitwise operations support big integers.

Konrad Borowski: Perl 6 changes - 2012W52

glitchmr@myopera.com (Konrad Borowski) — Sat, 29 Dec 2012 08:00:00 +0000

This is last “Perl 6 changes” update in this year. Enjoy.

id='new_features'>New features>

Perl 6 specification>

`uniq` method/subroutine, existing for long time in implementations is now mentioned in specification (with `===` semantics).

Rakudo Perl>

Use of suffix regex modifiers (P5-ism) in `s///`, like `s////g` is now detected.

rakudo.org: Rakudo Star 2012.12 released

Thu, 27 Dec 2012 19:53:53 +0000

On behalf of the Rakudo and Perl 6 development teams, I’m happy to announce the December 2012 release of “Rakudo Star”, a useful and usable distribution of Perl 6. The tarball for the December 2012 release is available from the download section. A Windows .MSI version of Rakudo star will usually appear in the downloads area shortly after the tarball release.

In the Perl 6 world, we make a distinction between the language (“Perl 6″) and specific implementations of the language such as “Rakudo Perl”. This Star release includes release 2012.12 of the Rakudo Perl 6 compiler, version 4.10.0 of the Parrot Virtual Machine, and various modules, documentation, and other resources collected from the Perl 6 community.

Some of the new features added to this release include:

Parse errors are much improved, and follow STD, the standard parser,
much more closely; they are more accurate and more information is given
Rakudo now keeps parsing after some less serious errors
Better errors for various parse failures
The junction autothreader is now an order of magnitude faster
Texas (ASCII) versions of the Set and Bag operators implemented
Nested Pairs now give correct .perl output
{ a => $_ } now correctly considered a block, not a hash as
before

This release also contains a range of performance improvements, bug fixes, improvements to error reporting and better failure modes.

The following features have been deprecated or modified from previous releases due to changes in the Perl 6 specification, and are being removed or changed as follows:

‘for’-loops will become lazy, and are only evaluated eagerly in eager or sink (void) context. This means that if a for-loop is the last statement in a routine, it will usually run after the routine has returned, so it cannot call return() anymore.
Unary hyper ops currently descend into nested arrays and hashes. This will change to make them equivalent to a one-level map.
The Str.ucfirst builtin is deprecated; it will be replaced by
Str.tc.
Leading whitespace in rules and under :sigspace will no longer be converted to <.ws> . For existing regexes that expect this conversion, add a <?> in front of leading whitespace to make it meta again.
The ?-quantifier on captures in regexes currently binds the capture slot to a List containing either zero or one Match objects; i.e., it is equivalent to “** 0..1″. In the future, the ?-quantifier will bind the slot directly to a captured Match or to Nil. Existing code can manage the transition by changing existing ?-quantifiers to use “** 0..1″, which will continue to return a List of matches.

There are some key features of Perl 6 that Rakudo Star does not
yet handle appropriately, although they will appear in upcoming
releases. Some of the not-quite-there features include:

advanced macros
threads and concurrency
Unicode strings at levels other than codepoints
interactive readline that understands Unicode
non-blocking I/O
much of Synopsis 9

There is an online resource that lists the known implemented and missing features of Rakudo and other Perl 6 implementations.

In many places we’ve tried to make Rakudo smart enough to inform the programmer that a given feature isn’t implemented, but there are many that we’ve missed. Bug reports about missing and broken features are welcomed at rakudobug@perl.org.

See http://perl6.org/ for links to much more information about Perl 6, including documentation, example code, tutorials, reference materials, specification documents, and other supporting resources. A draft of a Perl 6 book is available as docs/UsingPerl6-draft.pdf in the release tarball.

Perl 6 Advent Calendar: Day 24 – An Advent Calendar

Mon, 24 Dec 2012 00:00:47 +0000

Recently I was unpacking some boxes of books and came across a book entitled "BASIC Computer Programs for the Home" by Charles D. Sternberg. Apparently my father had purchased this book in the early 1980s and had given it to me. In any case, my name was scrawled in the front cover in the manner an adolescent me would have done.

Mostly this book is filled with simple BASIC programs that manage simple databases of various things: recipes, household budget, address book, music collections, book collections, school grades, etc. But the program that caught my eye and made me think of the Perl 6 Advent Calendar was one for printing a calendar starting at a particular month.

Now, the version in this book is a little simple in that it asks for the starting month, year, the day of the week that the first month starts on, and how many months to print. I wanted something a little more like the Unix utility cal(1) program. Luckily, Perl 6 has date handling classes as part of the specification and both major implemenations, Rakudo and Niecza, have actual implementations of these which should make creating the calendar quite easy.

For reference, the output of the Unix cal(1) utility looks like this:

       December 2012
    Su Mo Tu We Th Fr Sa
                       1
     2  3  4  5  6  7  8
     9 10 11 12 13 14 15
    16 17 18 19 20 21 22
    23 24 25 26 27 28 29
    30 31

It also has some options to change the output in various ways, but I just want to focus on reproducing the above basic output.

I'll need a list of month names and weekday abbreviations:

    constant @months = <January February March April May June July
                        August September October November December>;
    constant @days = <Su Mo Tu We Th Fr Sa>;

And it looks like the month and year are centered above the days of the week. Generating a calendar for May shows this to be the case, so I'll need a routine that centers text:

    sub center(Str $text, Int $width) {
        my $prefix = ' ' x ($width - $text.chars) div 2;
        my $suffix = ' ' x $width - $text.chars - $prefix.chars;
        return $prefix ~ $text ~ $suffix;
    }

Now, the mainline code needs two things: a month and a year. From this it should be able to generate an appropriate calendar. But, we should have a reasonable default for these values I think. Today's month and year seem reasonable to me:

    sub MAIN(:$year = Date.today.year, :$month = Date.today.month) {

but if it's not today's month and year, then it's some arbitrary month and year we need info about. To do this we construct a new Date object from the month and year given.

        my $dt = Date.new(:year($year), :month($month), :day(1) );

Looking at the calendar generated for December, it seems like we may actually output up to 6 rows of numbers since the month can start and end on a partial week. In order to implement this, I think I'll need some "slots" for each day. Each slot will either be empty or will contain the day of the month. The number of empty slots at the beginning of the month correspond to the day of the week that the first of the month occurs on. If the first is on Sunday, there will be 0 empty slots, if the first is on a Monday there will be 1 empty slot, if the first is on a Tuesday, there will be 2 empty slots, etc. This is remarkably similar to the number we get when we interrogate a Date object for the day of the week. The only wrinkle is that it returns 7 for Sunday when we actually need a 0. That's easily remedied with a modulus operator however:

        my $dt = Date.new(:year($year), :month($month), :day(1) );
        my $ss = $dt.day-of-week % 7;
        my @slots = ''.fmt("%2s") xx $ss;

That gives us the empty slots at the beginning, but what about the ones that actually contain the days of the month? Easy enough, we'll just generate a number for each day of the month using the Date object we created earlier.

        my $days-in-month = $dt.days-in-month;
        for $ss ..^ $ss + $days-in-month {
            @slots[$_] = $dt.day.fmt("%2d");
            $dt++
        }

Now we've got an array with appropriate values in the appropriate positions, all that's left is to actually output the calendar. Using the header line for our weekdays as a metric for the width of the calendar, and the routine we created for centering text, we can output the header portion of the calendar:

        my $weekdays = @days.fmt("%2s").join: " ";
        say center(@months[$month-1] ~ " " ~ $year, $weekdays.chars);
        say $weekdays;

Then we iterate over each slot and output the appropriate values. If we've reached the end of the week or the end of the month, we output a newline:

        for @slots.kv -> $k, $v {
            print "$v ";
            print "\n" if ($k+1) %% 7 or $v == $days-in-month;
        }

Putting it all together, here is the final program:

    #!/usr/bin/env perl6

    constant @months = <January February March April May June July
                        August September October November December>;
    constant @days = <Su Mo Tu We Th Fr Sa>;


    sub center(Str $text, Int $width) {
        my $prefix = ' ' x ($width - $text.chars) div 2;
        my $suffix = ' ' x $width - $text.chars - $prefix.chars;
        return $prefix ~ $text ~ $suffix;
    }

    sub MAIN(:$year = Date.today.year, :$month = Date.today.month) {
        my $dt = Date.new(:year($year), :month($month), :day(1) );
        my $ss = $dt.day-of-week % 7;
        my @slots = ''.fmt("%2s") xx $ss;

        my $days-in-month = $dt.days-in-month;
        for $ss ..^ $ss + $days-in-month {
            @slots[$_] = $dt.day.fmt("%2d");
            $dt++
        }

        my $weekdays = @days.fmt("%2s").join: " ";
        say center(@months[$month-1] ~ " " ~ $year, $weekdays.chars);
        say $weekdays;
        for @slots.kv -> $k, $v {
            print "$v ";
            print "\n" if ($k+1) %% 7 or $v == $days-in-month;
        }
    }

Normally, cal(1) will highlight today's date on the calendar. That's a feature I left out of my calendar implementation but it could easily be added with Term::ANSIColor. Also, there's a little bit of coupling between the data being generated in the slots and the output processing (the slots are all formatted to be 2 characters wide in anticipation of the output). There are some other improvements that could be done, but for a first cut at a calendar in Perl 6, I'm happy. :-)

Perl 6 Advent Calendar: Day 23 – Macros

Sun, 23 Dec 2012 17:07:05 +0000

Syntactic macros. The Lisp gods of yore provided humanity with this invention, essentially making Lisp a programmable programming language. Lisp adherents often look at the rest of the programming world with pity, seeing them fighting to invent wheels that were wrought and polished back in the sixties when giants walked the Earth and people wrote code in all-caps.

And the Lisp adherents see that the rest of us haven’t even gotten to the best part yet, the part with syntactic macros. We’re starting to get the hang of automatic memory management, continuations, and useful first-class functions. But macros are still absent from this picture.

In part, this is because in order to have proper syntactic macros, you basically have to look like Lisp. You know, with the parentheses and all. Lisp ends up having almost no syntax at all, making every program a very close representation of a syntax tree. Which really helps when you have macros starting to manipulate those same trees. Other languages, not really wanting to look like Lisp, find it difficult-to-impossible to pull off the same trick.

The Perl languages love the difficult-to-impossible. Perl programmers publish half a dozen difficult-to-impossible solutions to CPAN before breakfast. And, because Perl 6 is awesome and syntactic macros are awesome, Perl 6 has syntactic macros.

It is known, Khaleesi.

What are macros?

For reasons even I don’t fully understand, I’ve put myself in charge of implementing syntactic macros in Rakudo. Implementing macros means understanding them. Understanding them means my brain melts regularly. Unless it fries. It’s about 50-50.

I have this habit where I come into the #perl6 channel, and exclaiming “macros are just X!” for various values of X. Here are some samples:

Macros are just syntax tree manipulators.
Macros are just “little compilers”.
Macros are just a kind of templates.
Macros are just routines that do code substitution.
Macros allow you to safely hand values back and forth between the compile-time world and the runtime world.

But the definition that I finally found that I like best of all comes from scalamacros.org:

Macros are functions that are called by the compiler during compilation. Within these functions the programmer has access to compiler APIs. For example, it is possible to generate, analyze and typecheck code.

While we only cover the “generate” part of it yet in Perl 6, there’s every expectation we’ll be getting to the “analyze and typecheck” parts as well.

Some examples, please?

Coming right up.

macro checkpoint {
  my $i = ++(state $n);
  quasi { say "CHECKPOINT $i"; }
}

checkpoint;
for ^5 { checkpoint; }
checkpoint;

The quasi block is Perl 6′s way of saying “a piece of code, coming right up!”. You just put your code in the quasi block, and return it from the macro routine.

This code inserts “checkpoints” in our code, like little debugging messages. There’s only three checkpoints in the code, so the output we’ll get looks like this:

CHECKPOINT 1
CHECKPOINT 2
CHECKPOINT 2
CHECKPOINT 2
CHECKPOINT 2
CHECKPOINT 2
CHECKPOINT 3

Note that the “code insertion” happens at compile time. That’s why we get five copies of the CHECKPOINT 2 line, because it’s the same checkpoint running five times. If we had had a subroutine instead:

sub checkpoint {
  my $i = ++(state $n);
  say "CHECKPOINT $i";
}

Then the program would print 7 distinct checkpoints.

CHECKPOINT 1
CHECKPOINT 2
CHECKPOINT 3
CHECKPOINT 4
CHECKPOINT 5
CHECKPOINT 6
CHECKPOINT 7

As a more practical example, let’s say you have logging output in your program, but you want to be able to switch it off completely. The problem with an ordinary logging subroutine is that with something like:

LOG "The answer is { time-consuming-computation() }";

The time-consuming-computation() will run and take a lot of time even if LOG subsequently finds that logging was turned off. (That’s just how argument evaluation works in a non-lazy language.)

A macro fixes this:

constant LOGGING = True;

macro LOG($message) {
  if LOGGING {
    quasi { say {{{$message}}} };
  }
}

Here we see a new feature: the {{{ }}} triple-block. (Syntax is likely to change in the near future, see below.) It’s our way to mix template code in the quasi block with code coming in from other places. Doing say $message; would have been wrong, because $message is a syntax tree of the message to be logged. We need to inject that syntax tree right into the quasi, and we do that with a triple-block.

The macro conditionally generates logging code in your program. If the constant LOGGING is switched on, the appropriate logging code will replace each LOG macro invocation. If LOGGING is off, each macro invocation will be replaced by literally nothing.

Experience shows that running no code at all is very efficient.

What are syntactic macros?

A lot of things are called “macros” in this world. In programming languages, there are two big categories:

Textual macros. They substitute code on the level of the source code text. C’s macros, or Perl 5′s source filters, are examples.
Syntactic macros. They substitute code on the level of the source code syntax tree. Lisp macros are an example.

Textual macros are very powerful, but they represent the kind of power that is just as likely to shoot half your leg off as it is to get you to your destination. Using them requires great care, of the same kind needed for a janitor gig at Jurassic Park.

The problem is that textual macros don’t compose all that well. Bring in more than one of them to work on the same bit of source code, and… all bets are off. This puts severe limits on modularity. Textual macros, being what they are, leak internal details all over the place. This is the big lesson from Perl 5′s source filters, as far as I understand.

Syntactic macros compose wonderfully. The compiler is already a pipeline handing off syntax trees between various processing steps, and syntactic macros are simply more such steps. It’s as if you and the compiler were two children, with the compiler going “Hey, you want to play in my sandbox? Jump right in. Here’s a shovel. We’ve got work to do.” A macro is a shovel.

And syntactic macros allow us to be hygienic, meaning that code in the macro and code outside of the macro don’t step on each other’s toes. In practice, this is done by carefully keeping track of the macros context and the mainline’s context, and making sure wires don’t cross. This is necessary for safe and large-scale composition. Textual macros don’t give us this option at all.

Future

Both of the examples in this post work already in Rakudo. But it might also be useful to know where we’re heading with macros in the next year or so. The list is in the approximate order I expect to tackle things.

Un-hygiene. While hygienic macros are the sane and preferable default, sometimes you want to step on the toes of the mainline code. There should be an opt-out, and escape hatch. This is next up.
Introspection. In order to analyze and typecheck code, not just generate it, we need to be able to take syntax trees coming in as macro arguments, and look inside of them. There are no tools for that yet, and there’s no spec to guide us here. But I’m fairly sure people will want this. The trick is to come up with something that doesn’t tie us down to one compiler’s internal syntax-tree format. Both for the sake of compiler interoperability and future compatibility.
Deferred declarations. The sandbox analogy isn’t so frivolous, really. If you declare a class inside a quasi block, that declaration is limited (“sandboxed”) to within that quasi block. Then, when the code is injected somewhere in the mainline because of a macro invocation, it should actually run. Fortunately, as it happens, the Rakudo internals are factored in such a way that this will be fairly straightforward to implement.
Better syntax. The triple-block syntax is probably going away in favor of something better. The problem isn’t the syntax so much as the fact that it currently only works for terms. We want it to work for basically all syntactic categories. A solid proposal for this is yet to materialize, though.

With each of these steps, I expect us to find new and fruitful uses of macros. Knowing my fellow Perl 6 developers, we’ll probably find some uses that will shock and disgust us all, too.

In conclusion

Perl 6 is awesome because it puts you, the programmer, in the driver seat. Macros are simply more of that.

Implementing macros makes your brain melt. However, using them is relatively straightforward.

Tadeusz Sośnierz (tadzik): Threads for Rakudo Perl 6

Sat, 22 Dec 2012 10:36:54 +0000

So it has come to this. Threads.pm is up and running, bringing the ever-wanted threaded execution to the most popular Perl 6 implementation.

You’re looking for TL;DR, aren’t you? Here’s what it’s capable of:

use Threads;
use Semaphore;

my @elements;
my $free-slots = Semaphore.new(value => 10);
my $full-slots = Semaphore.new(value => 0);

sub produce($id) {
    my $i = 0;
    loop {
        $free-slots.wait;
        @elements.push: $i;
        $i++;
        $full-slots.post;
    }
}

sub consume($id) {
    loop {
        $full-slots.wait;
        my $a = @elements.shift;
        $free-slots.post;
    }
}

for 1..5 -> $i {
    async sub { produce($i)  }
}

for 5..10 -> $i {
    async sub { consume($i) }
}

Doesn’t look that awesome. I mean, it’s just a producer-consumer problem, what’s the big deal? Let me repeat:

OMG, RAKUDO HAS WORKING THREADS.

So, once we’re done celebrating and dancing macarena all around, there’ll always be someone to ask “hold on, there’s gotta be a caveat. Something surely is missing, explain yourself!”

I’ll be delighted to say “nope, everything’s there!”, but that’d make me a liar. Yeah, there are missing pieces. First, those aren’t really native threads – just green threads. Native OS threads are already implemented in Parrot VM, but NQP (the language that Rakudo is based on) still doesn’t support them, so before some volunteer comes along to fix them, you’ll still have to build parrot --without-threads (which means: use green threads, not OS threads) for Threads.pm to work. But fear not! The API is exactly the same, so once native threads are there, both Threads.pm and the code you write with it should work without any changes.

But green threads are fine too! Except for one minor detail: whenever any of them blocks on IO, the entire Parrot comes to a halt. The plan is for Parrot threads scheduler to handle it nicely, but it’s not there yet, so if you expected nice and easy async IO, sorry, but you’re stuck on MuEvent :)

Yep, we’re not really there yet. But I claim it’s closer than ever. We have working threads implementation. You can write code with that, and it’s not a PITA. Go for it! There’s a lot of room to improve it. I didn’t try really hard for Threads.pm to follow the concurrency synopsis (in my defense, I think niecza doesn’t follow it either :)), and I think that once we unleash a wolfpack of developers which can work towards something that we’ll all love to use.

Konrad Borowski: Perl 6 changes - 2012W51

glitchmr@myopera.com (Konrad Borowski) — Sat, 22 Dec 2012 08:00:00 +0000

TODO: Insert the description of changes. Also, mention that Rakudo 2012.12 (Warszawa) was released and that Rakudo isn’t Rakudo Star.

id='incompatible_changes'>Incompatible changes>

Rakudo Perl (in Rakudo 2012.12)>

When disambiguating hash/block, and `$_` variable is used, Rakudo assumes block. This could break your code if you were expecting a hash.

Rakudo Perl (post Rakudo 2012.12)>

`my $a; { $a; my $a }` is an error.

`for` runs in sink (void) or list context, just like `map` (actually, `for` does exactly what `map` does). If you were depending on it, you can use `eager do` before `for` to force eager execution.

id='new_features'>New features>

Rakudo Perl (in Rakudo 2012.12)>

Added `X::Comp::Group` exception. It’s used when compilation can produce multiple issues.

Added support for warnings. Code like `my ($a, $a);` only causes warnings, not fatal errors.

Added `X::Undeclared::Symbols` exception for variables that weren’t declared, but used.

You can use `.message` on `Exception`s.

Fixed `.perl` output for a `Pair` that has key that is a `Pair`.

`X::Item` is thrown for `@a[NaN]` or `@a[Inf]`. It isn’t incompatible change because it wasn’t working before anyway.

Perl 6 Advent Calendar: Day 22 – Parsing an IPv4 address

Sat, 22 Dec 2012 00:01:22 +0000

Guest post by Herbert Breunung (lichtkind).

Perl 5 brought regexes to mainstream programming and set a standard, one that is felt as relevant even in Redmond. Perl 6, of course, steps up the game by adding many new features to the regex camp, including easy-to-build grammars for your own complex parsers. But without getting too complex, you can get a lot of joy out of Perl 6′s rx (that’s how Perl 6 spells Perl 5′s qr operator, that enables you to save a Regex in a variable).

Because the Perl 6 regex syntax is less littered with exceptional cases, Larry Wall also likes to joke that he put the “regular” back into “regular expression”.

Some of the changes are:

most special variables are gone,
non-capturing groups and other grouping syntax is easier to type,
no more single/multi line modes,
x mode became default, making whitespace non-significant by default.

In summary, regexes are more regular than in Perl 5, confirming Larry’s joke. They try a bit harder to make your life easier when you need to match text. Under the hood, regexes have blossomed out into a complete sub-language within the bigger Perl 6 language. A language with its own parsing rules.

But don’t fret; not everything has changed. Some things remain the same:

/\d+/

This regex still matches one or more consecutive digits.

Similarly, if you want to capture the digits, you can do this, just like you’re used to:

/(\d+)/

You’ll find the matched digits in $0, not $1 as in Perl 5. All the special variables $0, $1, $2 are really syntactic sugar for indexing the match variable ($/[0], $/[1], $/[2]). Because indices start at 0, it makes sense for the first matched group to be $0. In Perl 5, $0 contains the name of the script or program, but this has been renamed into $*EXECUTABLE_NAME in Perl 6.

Should you be interested in getting all of the captured groups of a regex match, you can use @(), which is syntactic sugar for @($/).

The object in the $/ variable holds lots of useful information about the last match. For example, $/.from will give you the starting string position of the match.

But $0 will get us far enough for this post. We use it to extract individual features from a string.

Sometimes we want to extract a whole bunch of similar things at once. Then we can use the :g (or :global) modifier on the regex:

$_ = '1 23 456 78.9';
say .Str for m:g/(\d+)/; # 1 23 456 78 9

Note that the :g — as opposed to prior regex implementations — sits up front, right at the start of the regex. Not at the end. That way, when you read the regex from left to right, you will know from the start how the regex is doing its matching. No more end-heavy regex expressions.

Matching “all things that look like this” is so useful, that there’s even a dedicated method for that, .comb:

$str.comb(/\d+/);

If you’re familiar with .split, you can think of .comb as its cheerful cousin, matching all the things that .split discards.

Let’s tackle the matching of an IPv4 address. Coming from a Perl 5 angle, we expect to have to do something like this:

/(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/

This won’t do in Perl 6, though. First of all, the {} blocks are real blocks in a Perl 6 regex; they contain Perl 6 code. Second, because Perl 6 has lots of error handling to catch p5isms, like this, you’ll get an error saying “Unsupported use of {N,M} as general quantifier; in Perl 6 please use ** N..M (or ** N..*)”.

So let’s do that. To match between one and three digits in a Perl 6 regex, we should type:

/\d ** 1..3/

Note how the regex sublanguage re-uses parts from the main Perl 6 language. ** can be seen as a kind of exponentiation (if we squint), in that we’re taking \d “to the between-first-and-third power”. And the range notation 1..3 exists both outside and within regexes.

Using our new knowledge about the repetition quantifier, we end up with something like this:

/(\d**1..3) \. (\d**1..3) \. (\d**1..3) \. (\d**1..3)/

That’s still kinda clunky. We might end up wishing that we could use the repetition operator again, but those literal dots in between prevent us from doing that. If only we could specify repetition a given number of times and a divider.

In Perl 6 regexes, you can.

/ (\d ** 1..3) ** 4 % '.' /

The % operator here is a quantifier modifier, so it can only follow on a quantifier like * or + or **. The choice of % for this function is relatively new in Perl 6, and you may prefer to read it as “modulo”, just like in the main language. That is, “match four groups of digits, modulo literal dots in between”. Or you could think of the dots in between as the “remainder”, the separators that are left after you’ve parsed the actual elements.

Oh, and you might’ve noticed that \. changed to '.' on the way. We can use either; they mean exactly the same. In Perl 5, there isn’t a simple rule saying which symbols have a magic meaning and which ones simply signify themselves. In Perl 6, it’s easy: word characters (alphanumerics and the underscore) always signify themselves. Everything else has to be escaped or quoted to get its literal meaning.

Putting it all together, here’s how we would extract IPv4 addresses out of a string:

$_ = "Go 127.0.0.1, I said! He went to 173.194.32.32.";

say .Str for m:g/ (\d ** 1..3) ** 4 % '.' /;
# output: 127.0.0.1 173.194.32.32

Or, we could use .comb:

$_ = "Go 127.0.0.1, I said! He went to 173.194.32.32.";
my @ip4addrs = .comb(/ (\d ** 1..3) ** 4 % '.' /);

If we’re interested in individual integers, we can get those too:

$_ = "Go 127.0.0.1, I said! He went to 173.194.32.32.";
say .list>>.Str.perl for m:g/ (\d ** 1..3) ** 4 % '.' /;
# output: ("127", "0", "0", "1") ("173", "194", "32", "32")

If you want to know more, read the S05, or watch me battling with my slide deck and the English language in this presentation about regexes.

Perl 6 Advent Calendar: Day 21 – Collatz Variations

Fri, 21 Dec 2012 12:25:40 +0000

The Collatz sequence is one of those interesting “simple” math problems that I’ve run into a number of times. Most recently a blog post on programming it in Racket showed up on Hacker News. As happens so often, I instantly wanted to implement it in Perl 6.

sub collatz-sequence(Int $start) { 
    $start, { when * %% 2 { $_ / 2 }; when * !%% 2 { 3 * $_ + 1 }; } ... 1;
}

sub MAIN(Int $min, Int $max) {
    say [max] ($min..$max).map({ +collatz-sequence($_) });        
}

This is a very straightforward implementation of the Racket post’s max-cycle-length-range as a stand-alone p6 script. collatz-sequence generates the sequence using the p6 sequence operator. Start with the given number. If it is divisible by two, do so: when * %% 2 { $_ / 2 }. If it is not, multiply by three and add 1: when * !%% 2 { 3 * $_ + 1 }. Repeat this until the sequence reaches 1.

MAIN(Int $min, Int $max) sets up our main function to take two integers. Many times I don’t bother with argument types in p6, but this provides a nice feedback for users:

> perl6 collatz.pl blue red
Usage:
  collatz.pl <min> <max>

The core of it just maps the numbers from $min to $max (inclusive) to the length of the sequence (+collatz-sequence) and then says the max of the resulting list ([max]).

Personally I’m a big fan of using the sequence operator for tasks like this; it directly represents the algorithm constructing the Collatz sequence in a simple and elegant fashion. On the other hand, you should be able to memoize the recursive version for a speed increase. Maybe that would give it an edge over the sequence operator version?

Well, I was wildly wrong about that.

sub collatz-length($start) {
    given $start {
        when 1       { 1 }
        when * !%% 2 { 1 + collatz-length(3 * $_ + 1) } 
        when * %% 2  { 1 + collatz-length($_ / 2) } 
    }
}

sub MAIN($min, $max) {
    say [max] ($min..$max).map({ collatz-length($_) });        
}

This recursive version, which makes no attempt whatsoever to be efficient, is actually better than twice as fast as the sequence operator version. In retrospect, this makes perfect sense: I was worried about the recursive version making a function call for every iteration, but the sequence version has to make two, one to calculate the next iteration and the other to check and see if the ending condition has been reached.

Well, once I’d gotten this far, I thought I’d better do things correctly. I wrote two framing scripts, one for timing all the available scripts, the other for testing them to make sure they work!

my @numbers = 1..200, 10000..10200;

sub MAIN(Str $perl6, *@scripts) {
    my %results;
    for @scripts -> $script {
        my $start = now;
        qqx/$perl6 $script { @numbers }/;
        my $end = now;

        %results{$script} = $end - $start;
    }

    for %results.pairs.sort(*.value) -> (:key($script), :value($time)) {
        say "$script: $time seconds";
    }
}

This script takes as an argument a string that can be used to call a Perl 6 executable and a list of scripts to run. It runs the scripts using the specified executable, and times them using p6′s now function. It then sorts the results into order and prints them. (A similar script I won’t post here tests each of them to make sure they are returning correct results.)

In the new framework, the Collatz script has changed a bit. Instead of taking a min and a max value and finding the longest Collatz sequence generated by a number in that range, it takes a series of numbers and generates and reports the length of the sequence for each of them. Here’s the sequence operator script in its full new version:

sub collatz-length(Int $start) { 
    +($start, { when * %% 2 { $_ / 2 }; when * !%% 2 { 3 * $_ + 1 }; } ... 1);
}

sub MAIN(*@numbers) {
    for @numbers -> $n {
        say "$n: " ~ collatz-length($n.Int);
    }
}

For the rest of the scripts I will skip the MAIN sub, which is exactly the same in each of them.

Framework established, I redid the recursive version starting from the new sequence operator code.

sub collatz-length(Int $n) {
    given $n {
        when 1       { 1 }
        when * %% 2  { 1 + collatz-length($_ div 2) }
        when * !%% 2 { 1 + collatz-length(3 * $_ + 1) }
    } 
}

The sharp-eyed will notice this version is different from the first recursive version above in two significant ways. This time I made the argument Int $n, which instantly turned up a bit of a bug in all implementations thus far: because I used $_ / 2, most of the numbers in the sequence were actually rationals, not integers! This shouldn’t change the results, but is probably less efficient than using Ints. Thus the second difference about, it now uses $_ div 2 to divide by 2. This version remains a great improvement over the sequence operator version, running in 4.7 seconds instead of 13.3. Changing when * !%% 2 to a simple default shaves another .3 seconds off the running time.

Once I started wondering how much time was getting eaten up by the when statements, rewriting that bit using the ternary operator was an obvious choice.

sub collatz-length(Int $start) { 
    +($start, { $_ %% 2 ?? $_ div 2 !! 3 * $_ + 1 } ... 1);
}

Timing results: Basic sequence 13.4 seconds. Sequence with div 11.5 seconds. Sequence with div and ternary 9.7 seconds.

That made me wonder what kind of performance I could get from a handcoded loop.

sub collatz-length(Int $n is copy) {
    my $length = 1;
    while $n != 1 {
        $n = $n %% 2 ?? $n div 2 !! 3 * $n + 1;
        $length++;
    }
    $length;
}

That’s by far the least elegant of these, I think, but it gets great performance: 3 seconds.

Switching back to the recursive approach, how about using the ternary operator there?

sub collatz-length(Int $n) {
    return 1 if $n == 1;
    1 + ($n %% 2 ?? collatz-length($n div 2) !! collatz-length(3 * $n + 1));
}

This one just edges out the handcoded loop, 2.9 seconds.

Can we do better than that? How about memoization? is cached is supposed to be part of Perl 6; neither implementation has it yet, but last year’s Advent calendar has a Rakudo implementation that still works. Using the last version changed to sub collatz-length(Int $n) is cached { works nicely, but takes 3.4 seconds to execute. Apparently the overhead of caching slows it down a bit. Interestingly, the non-ternary recursive version does speed up with is cached, from 4.4 seconds to 3.6 seconds.

Okay, instead of using a generic memoization, how about hand-coding one?

sub collatz-length(Int $n) {
    return 1 if $n == 1;
    state %lengths;
    return %lengths{$n} if %lengths.exists($n);
    %lengths{$n} = 1 + ($n %% 2 ?? collatz-length($n div 2) !! collatz-length(3 * $n + 1));
}

Bingo! 2.7 seconds.

I’m sure there are lots of other interesting approaches for solving this problem, and encourage people to send them in. In the meantime, here’s my summary of results so far:

Script	Rakudo	Niecza
bin/collatz-recursive-ternary-hand-cached.pl	2.5	1.7
bin/collatz-recursive-ternary.pl	3	1.7
bin/collatz-loop.pl	3.1	1.7
bin/collatz-recursive-ternary-cached.pl	3.2	N/A
bin/collatz-recursive-default-cached.pl	3.5	N/A
bin/collatz-recursive-default.pl	4.4	1.8
bin/collatz-recursive.pl	4.9	1.9
bin/collatz-sequence-ternary.pl	9.9	3.3
bin/collatz-sequence-div.pl	11.6	3.5
bin/collatz-sequence.pl	13.5	3.8

The table was generated from timing-table-generator.pl.

Perl 6 Advent Calendar: Day 20 – Dynamic variables and DSL-y things

Thu, 20 Dec 2012 00:01:06 +0000

Today, let’s talk about DSLs.

Post from the past: a motivating example

Two years ago I wrote a blog post about Nim, a game played with piles of stones. I just put in ASCII diagrams of the actual Nim stone piles, telling myself that if I had time, I would put in fancy SVG diagrams, generated with Perl 6.

Naturally, I didn’t have time. My self-imposed deadline ran out, and I published the post with simple ASCII diagrams.

But time is ever-regenerative, and there for people who want it. So, let’s generate some fancy SVG diagrams with Perl 6.

Have bit array, want SVG

What do we need, exactly? Well, a subroutine that takes an array of piles as input and generates an SVG file would be a really good start.

Let’s take the last “image” in the post as an example:

3      OO O
4 OOOO
5 OOOO    O

For the moment, let’s ignore the numbers at the left margin; they’re just counting stones. We summarize the piles themselves as a kind of bitmap, which also forms the input to the function:

my @piles =
    [0, 0, 0, 0, 0, 0, 0, 0, 1],
    [1, 1, 1, 1, 0, 1, 1, 0, 1],
    [1, 1, 1, 1, 0, 0, 0, 0, 1];

nim-svg(@piles);

At this point, we need only create the nim-svg function itself, and make it render SVG from this bitmap. Since I’ve long since tired of outputting SVG by hand, I use the SVG module, which comes bundled with Rakudo Star.

use SVG;

sub nim-svg(@piles) {
    my $width = max map *.elems, @piles;
    my $height = @piles.elems;

    my @elements = gather for @piles.kv -> $row, @pile {
        for @pile.kv -> $column, $is_filled {
            if $is_filled {
                take 'circle' => [
                    :cx($column + 0.5),
                    :cy($row + 0.5),
                    :r(0.4)
                ];
            }
        }
    }
    
    say SVG.serialize('svg' => [ :$width, :$height, @elements ]);
}

I think you can follow the logic in there. The subroutine simply iterates over the bitmap, turning 1s into circles with appropriate coordinates.

That’s it?

Well, this will indeed generate an SVG image for us, with the stones correctly placed. But let’s look again at the input that helped create this image:

    [0, 0, 0, 0, 0, 0, 0, 0, 1],
    [1, 1, 1, 1, 0, 1, 1, 0, 1],
    [1, 1, 1, 1, 0, 0, 0, 0, 1];

Clearly, though we can discern the stones and gaps in there if we squint in a bit-aware programmer’s fashion, the input isn’t… visually attractive. (The zeroes even look like stones, even though they’re gaps!)

We can do better

Instead of using a bit array, let’s start from the desired SVG image and try to make the input look like that.

So, this is what I would prefer to write instead of a bitmask:

nim {
  _ _ _ _ _ _ _ _ o;
  o o o o _ o o _ o;
  o o o o _ _ _ _ o;
}

That’s better. That looks more like my original ASCII diagram, while still being syntactic Perl 6 code.

Making a DSL

Wikipedia talks about a DSL as a language “dedicated to a particular problem domain”. Well, the above way of specifying the input would be a DSL dedicated to solving the draw-SVG-images-of-Nim-positions domain. (Admittedly a fairly narrow domain. But I’m mostly out to show the potential of DSLs in Perl 6, not to change the world with this particular DSL.)

Now that we have the desired end state, how do we connect the wires and make the above work? Clearly we need to declare three subroutines: nim, _, o. (Yes, you can name a subroutine _, no sweat.)

sub nim(&block) {
    my @*piles;
    my @*current-pile;

    &block();
    finish-last-pile();
    
    nim-svg(@*piles);
}

sub _(@rest?) {
    unless @rest {
        finish-last-pile();
    }
    @*current-pile = 0, @rest;
    return @*current-pile;
}

sub o(@rest?) {
    unless @rest {
        finish-last-pile();
    }
    @*current-pile = 1, @rest;
    return @*current-pile;
}

Ok… explain, please?

A couple of things are going on here.

The two variables @*piles and @*current-pile are dynamic variables which means that they are visible not just in the current lexical scope, but also in all subroutines called before the current scope has finished. Notably, the two subroutines _ and o.
The two subroutines _ and o take an optional parameter. On each row, the rightmost _ or o acts as a silent “start of pile” marker, taking the time to do a bit of bookkeeping with the piles, storing away the last pile and starting on a new one.
Each row in the DSL-y input basically forms a chain of subroutine calls. We take this into account by both incrementally building the @*current-pile array at each step, all the while returning it as (possible) input for the next subroutine call in the chain.

And that’s it. Oh yeah, we need the bookkeeping routine finish-last-pile, too:

sub finish-last-pile() {
    if @*current-pile {
        push @*piles, [@*current-pile];
    }
    @*current-pile = ();
}

So, it works?

Now, the whole thing works. We can turn this DSL-y input:

nim {
  _ _ _ _ _ _ _ _ o;
  o o o o _ o o _ o;
  o o o o _ _ _ _ o;
}

…into this SVG output:

<svg
  xmlns="http://www.w3.org/2000/svg"
  xmlns:svg="http://www.w3.org/2000/svg"
  xmlns:xlink="http://www.w3.org/1999/xlink"
  width="9" height="3">

  <circle cx="8.5" cy="0.5" r="0.4" />
  <circle cx="0.5" cy="1.5" r="0.4" />
  <circle cx="1.5" cy="1.5" r="0.4" />
  <circle cx="2.5" cy="1.5" r="0.4" />
  <circle cx="3.5" cy="1.5" r="0.4" />
  <circle cx="5.5" cy="1.5" r="0.4" />
  <circle cx="6.5" cy="1.5" r="0.4" />
  <circle cx="8.5" cy="1.5" r="0.4" />
  <circle cx="0.5" cy="2.5" r="0.4" />
  <circle cx="1.5" cy="2.5" r="0.4" />
  <circle cx="2.5" cy="2.5" r="0.4" />
  <circle cx="3.5" cy="2.5" r="0.4" />
  <circle cx="8.5" cy="2.5" r="0.4" />
</svg>

Yay!

Summary

The principles I used in this post are fairly easy to generalize. Start from your desired DSL, and create the subroutines to make it happen. Have dynamic variables handle the communication between separate subroutines.

DSLs are nice because they allow us to shape the code we’re writing around the problem we’re solving. Using relatively little “adapter code”, we’re left to focus on describing and solving problems in a natural way, making the programming language rise to our needs instead of lowering ourselves down to its needs.

perl6.announce: Parrot 4.11.0 "All together - Happy Birthday Lovebird" Released! by Reini Urban

Wed, 19 Dec 2012 01:49:52 +0000

On behalf of the Parrot team, I'm proud to announce Parrot 4.11.0, also known
as "All together - Happy Birthday Lovebird".
Parrot (http://parrot.org/) is a virtual machine aimed at running all
dynamic languages.

Parrot 4.11.0 is available on Parrot's FTP site
(ftp://ftp.parrot.org/pub/parrot/releases/devel/4.11.0/),
or by following the download instructions at http://parrot.org/download.
For those who would like to develop on Parrot, or help develop Parrot
itself, we recommend using
Git to retrieve the source code to get the latest and best Parrot code.

Threads are considered experimental. There are a known GC bugs when
running large threads
on darwin and sometimes linux. There are no known problems when
running large single-threaded programs.
Reading from shared variables is transparent, writing must be done via
scheduled updater subs
on the owner thread.
You can use the API without using native OS threads by configuring
--without-threads.

Parrot 4.11.0 News:
- Core
+ Parrot now supports fast and lightweight OS threads, based
on Nat Tucks's
initial GSoC work on green threads and then Stefan Seifert's extension
to true parallel OS threads as hybrid threads.
See http://wknight8111.blogspot.co.at/2010/08/gsoc-threads-chandons-results.html
and http://niner.name/Hybrid_Threads_for_the_Parrot_VM.pdf
Summary:
http://perl6advent.wordpress.com/2012/12/11/day-11-parrot-threads/
+ New parrot cmdline option --numthreads
+ Export new dynpmc_class_<ClassName> types [GH #870].
+ Changed core_ops library format and ops2c to contain the
PBC_COMPAT only,
not the parrot version triple. Removed the duplicate
PARROT_DYNOP_CORE_INIT
cpp definition in include/parrot/oplib/core_ops.h, use only
PARROT_CORE_OPLIB_INIT from config.h
+ Store PBC_MAJOR and PBC_MINOR in the config hash, stored by
auto::revision
+ Added getprotobyname method to the Socket PMC
- Documentation
+ Marked unused API parameters
+ Install man pages for all binaries
- Languages
+ nqp-rx tests updated from perl6/nqp-rx.git, and enabled to
run on make test
- Platforms
+ More compilation fixes on Windows without working mt.exe or
ranlib=rem,
enable smoking without git binary.
- Tools
+ Re-enabled pbc_dump -h --header-only
+ Improved pmc2c, added more UNUSED(arg) macros [GH #836].
Parrot is now almost
-Wunused-parameters safe.
- Tests
+ failing Pod::Simple pod_todo.t tests are now skipped
+ added nqp-rx tests
+ removed duplicate ext/nqp-rx/t/p6regex tests
+ rewrote and enabled native_pbc tests [GH #394],
renamed native_pbc testfiles

The SHA256 message digests for the downloadable tarballs are:
46de72022dfe521fb8c5d95453e1d091d4d20d8f7ca4f2ed0cd1420b682f20dc
parrot-4.11.0.tar.gz
0632ad0ca9557df391088bddbbc2608a4503adb5679dcbe62c065a73d4ce49de
parrot-4.11.0.tar.bz2

Many thanks to all our contributors for making this possible, and our sponsors
for supporting this project. Our next supported release is 15 January 2013.

Enjoy!
--
Reini Urban
http://cpanel.net/ http://www.perl-compiler.org/

Perl 6 Advent Calendar: Day 19 – Gather and/or Coroutines

Wed, 19 Dec 2012 00:01:43 +0000

Today I’ll write about coroutines, gather-take and why they are as much fun as one another. But since it’s all about manipulating control flow, I took the liberty to reorganize the control flow of this advent post, so coroutines will finally appear somewhere at the end of it. In the meantime I’ll introduce the backstory, the problems that coroutines solved and how it looks from the Perl 6 kitchen.

LWP::Simple is all fun and games, but sometimes you can’t afford to wait for the result to come. It would make sense to say “fetch me this webpage and drop me a note when you’re done with it”. That’s non trivial though; LWP::Simple is a black box, which we tell “get() this, get() that” and it gives us the result back. There is no possible way to intercept the internal data it sends there and around. Or is there?

If you look at Perl 5′s AnyEvent::HTTP, you’ll see that it reimplemented the entire HTTP client to have it non-blocking. Let’s see if we can do better than that.

First thing, where does LWP::Simple actually block? Behind our backs it uses the built-in IO::Socket::INET class. When it wants data from it, it calls .read() or .recv() and patiently waits until they’re done. If only we could somehow make it not rely on those two directly, hmm…

„I know!”, a gemstone-fascinated person would say, „We can monkey-patch IO::Socket::INET”. And then we have two problems. No, we’ll go the other way, and follow the glorious path of Dependency Injection.

That sounds a bit scary. I’ve heard about as many definitions of Dependency Injection as many people I know. The general idea is to not create objects inside other objects directly; it should be possible to supply them from the outside. I like to compare it to elimination of „magic constants”. No one likes those; if you think of classes as another kind of magic constants which may appear in somebody else’s code, this is pretty much what this is about. In our case it looks like this:

# LWP::Simple make_request
my IO::Socket::INET $sock .= new(:$host, :$port);

There we go. “IO::Socket::INET” is the magic constant here; if you want to use a different thing, you’re doomed. Let’s mangle it for a bit and allow the socket class to come from the outside.

We’ll add an attribute to LWP::Simple, let’s call it $!socketclass

has $.socketclass = IO::Socket::INET;

If we don’t supply any, it will just fallback to IO::Socket::INET, which is a sensible default. Then, instead of the previous .new() call, we do

my $sock = $!socketclass.new(:$host, :$port);

The actual patch (https://github.com/tadzik/perl6-lwp-simple/commit/93c182ac2) is a bit more complicated, as LWP::Simple supports calling get() not only on constructed objects but also on type objects, which have no attributes set, but we only care about the part shown above. We have an attribute $!socketclass, which defaults to IO::Socket::INET but we’re free to supply another class – dependency-inject it. Cool! So in the end it’ll look like this:

class Fakesocket is IO::Socket::INET {
    method recv($) {
        note 'We intercepted recv()';
        callsame;
    }

    method read($) {
        note 'We intercepted read()';
        callsame;
    }
}

# later
my $lwp = LWP::Simple.new(socketclass => Fakesocket);

And so our $lwp is a fine-crafted LWP::Simple which could, theorically, give the control flow back to us while it waits for read() and recv() to finish. So, how about we put theory into practice?

Here start the actual coroutines, sorry for being late :)

What do we really need in our modified recv() and read()? We need a way to say „yeah, if you could just stop executing and give time to someone else, that would be great.” Oh no, but we have no threads! Luckily, we don’t need any. Remember lazy lists?

my @a := gather { for 1..* -> $n { take $n } }

So on one hand we run an infinite for loop, and on the other we have a way to say „give back what you’ve come up with, I’ll catch up with you later”. That’s what take() does: it temporarily jumps out of the gather block, and is ready to get back to it whenever you want it. Do I hear the sound of puzzles clicking together? That’s exactly what we need! Jump out of the execution flow and wait until we’re asked to continue.

class Fakesocket is IO::Socket::INET {
    method recv($) {
        take 1;
        callsame;
    }

    method read($) {
        take 1;
        callsame;
    }
}

# later
my @a := gather {
    $lwp.get("http://jigsaw.w3.org/HTTP/300/301.html");
    take "done";
}

# give time to LWP::Simple, piece by piece
while ~@a.shift ne "done" {
    say "The coroutine is still running"
}
say "Yay, done!";

There we go! We just turned LWP::Simple into a non-blocking beast, using almost no black magic at all! Ain’t that cool.

We now know enough to create some syntactic sugar around it all. Everyone likes sugar.

module Coroutines;
my @coroutines;
enum CoroStatus <still_going done>;

sub async(&coroutine) is export {
    @coroutines.push($(gather {
        &coroutine();
        take CoroStatus::done;
    }));
}

#= must be called from inside a coroutine
sub yield is export {
    take CoroStatus::still_going;
}

#= should be called from mainline code
sub schedule is export {
    return unless +@coroutines;
    my $r = @coroutines.shift;
    if $r.shift ~~ CoroStatus::still_going {
        @coroutines.push($r);
    }
}

We maintain a list of coroutines currently running. Our async() sub just puts a block of code in the execution queue. Then every call to yield() will make it jump back to the mainline code. schedule(), on the other hand, will pick the first available coroutine to be run and will give it some time to do whatever it wants.

Now, let us wait for the beginning of the post to catch up.

Perl 6 Advent Calendar: Day 18 – Formulas: resistance is futile

Tue, 18 Dec 2012 00:01:24 +0000

Today, Perl turns 25: happy birthday Perl! There’s too much to say about this language, its philosophy, its culture, … So here, I would just thank all people who make Perl a success, for such a long time.

Introduction

A formula is “an entity constructed using the symbols and formation rules of a given language“, according to Wikipedia as of this writing. These words sound really familiar for any Perl 6 users who have already played with grammars, however this is not the purpose of this article. Instead, the aim is to demonstrate how the Perl 6 language can be easily extended in order to use formulas literally in the code.

There are many domains, like Mathematics, Physics, finance, etc., that use their own specific languages. When writing programs for such a domain, it could be less error-prone and simpler to use its specific language instead of using a specific API. For example, someone who has knowledge in electronic may find the formula below:

4.7kΩ ± 5%

far more understandable than the following piece of code:

my $factory MeasureFactory.getSharedInstance();
my $resistance = $factory.createMeasure(value     => 4700,
                                        unit      => Unit::ohm,
                                        precision => 5);

The formula 4.7kΩ ± 5% will be used all along this article as an example.

Symbol `k`: return a modified value

Let’s start with the simplest symbol: k. Basically this is just a multiplier placed after a numeric value. To make the Perl 6 language support this new operator, there’s no need to know much about Perl 6 guts: operators are just funny looking sub-routines:

sub postfix:<k> ($a) is tighter(&infix:<*>) { $a * 1000 }

This just makes 4.7k return 4.7 * 1000, for example. To be a little bit picky, such kind of multiplier should not be used without a unit (ex. Ω) and not be coupled to another multiplier (ex. μ). This would have made this article a little bit more complex, so this is left as an exercise to the reader :) Regarding the tighter trait, it is already well explained in three other articles.

Symbols `%`: return a closure

The next symbol is %: it is commonly used to compute a ratio of something, that’s why 5% shouldn’t naively be transformed into 0.05. Instead, it creates a closure that computes the given percent of whatever you want:

sub postfix:<%> ($a) is tighter(&infix:<*>) { * * $a / 100 }

It’s now possible to write $f = 5%; $f(42) or 5%(42) directly, and this returns 2.1. It is worth saying this doesn’t conflict with the infix:<%> operator (modulo), that is, 5 % 42 still returns 5.

Symbol `Ω`: create a new `Measure` object

Let’s go on with the Ω symbol. One possibility is to tie the unit and the value in the same object, as in the Measure class defined below. The ACCEPTS method is explained later but the idea in this case is that two Measure objects with two different units can’t match together:

enum Unit <volt ampere ohm>;

class Measure {
    has Unit $.unit;
    has $.value;

    method ACCEPTS (Measure:D $a) {
        $!unit == $a.unit && $!value.ACCEPTS($a.value);
    }
}

Then, one operator per unit can be defined in order to hide the underlying API, that is, to allow 4.7kΩ as an equivalent of Measure.new(value => 4.7k, unit => ohm):

sub postfix:<V> (Real:D $a) is looser(&postfix:<k>) {
    Measure.new(value => $a, unit => volt)
}
sub postfix:<A> (Real:D $a) is looser(&postfix:<k>) {
    Measure.new(value => $a, unit => ampere)
}
sub postfix:<Ω> (Real:D $a) is looser(&postfix:<k>) {
     Measure.new(value => $a, unit => ohm)
}

Regarding the ACCEPTS method, it is used by ~~, the smartmatch operator, to check if the left operand can match the right operand, the one with the ACCEPTS method. In other terms, $a ~~ $b is equivalent to $b.ACCEPTS($a). Typically, this allows the intuitive comparison between two different types, like scalars and containers for example.

In this example, this method is overloaded to ensure two Measure objects can match only if they have the same unit and if their values match. That means 4kΩ ~~ 4.0kΩ is True whereas 4kΩ ~~ 4kV is False. Actually, there are many units that can mix altogether, typically currencies (¥€$) and the ones derived from the International System of Unit. But as usual, when something is a little bit more complex, it is left as an exercise to the reader ;)

Symbol `±`: create a `Range` object

There’s only one symbol left so far: ±. In the example, it is used to indicate the tolerance of the resistance. This tolerance could be either absolute (expressed in Ω) or relative (expressed in %), thus the new infix:<±> operator has several signatures and have to be declared with a multi keyword. In both cases, the value is a new Range objects with the right bounds:

multi sub infix:<±> (Measure:D $a, Measure:D $b) is looser(&postfix:<Ω>) {
    die if $a.unit != $b.unit;
    Measure.new(value => Range.new($a.value - $b.value,
                                   $a.value + $b.value),
                unit => $a.unit);
}

multi sub infix:<±> (Measure:D $a, Callable:D $b) is looser(&postfix:<Ω>) {
    Measure.new(value => Range.new($a.value - $b($a.value),
                                   $a.value + $b($a.value)),
                unit => $a.unit);
}

Actually, any Callable object could be used in the second variant, not only the closures created by the % operators.

So far, so good! It’s time to check in the Perl6 REPL interface if everything works fine:

> 4.7kΩ ± 1kΩ
Measure.new(unit => Unit::ohm, value => 3700/1..5700/1)

> 4.7kΩ ± 5%
Measure.new(unit => Unit::ohm, value => 4465/1..4935/1)

It looks good, so all the code above ought to be moved into a dedicated module in order to be re-used at will. Then, a customer could load it and write literally:

my $resistance = 4321Ω;
die "resistance is futile" if !($resistance ~~ 4.7kΩ ± 5%);

As of this writing, this works both in Niecza and Rakudo, the two most advanced implementations of Perl 6.

Symbols that aren’t operators

Symbols in a formula are not always operators, they can be symbolic constants too, like π. In many languages, constants are just read-only variables, which sounds definitely weird: a variable isn’t supposed to be … variable? In Perl 6, a constant can be a read-only variable too (hmm) or a read-only term (this sounds better). For example, to define the constant term φ:

constant φ = (1 + sqrt(5)) / 2;

Conclusion

In this article the Perl 6 language was slightly extended with several new symbols in order to embed simple formulas. Although it is possible to go further by changing the Perl 6 grammar in order to embed more specific languages, that is, languages that don’t have the same grammar rules. Indeed, there are already two such languages supported by Perl 6: regexp and quotes. The same way, Niecza use a custom language to connect its portable parts to the unportable.

Bonus: How to type these exotic symbols?

Most of the Unicode symbols can be type in Xorg — the most used interface system on Linux — thanks to the Compose key, also named Multi key. When this special key is pressed, all the following key-strokes are somewhat merged in order to compose a symbol.

There’s plenty of documentation about this support elsewhere on Internet, so only the minimal information is provided here. First, to map the Compose key to the Caps Lock key, write in a X terminal:

sh> setxkbmap -option compose:caps

Some compositions are likely already defined, for instance <caps> followed by + then - should now produce ±, but both Ω and φ are likely not defined. One solution is to write a
~/.XCompose file with the following content:

include "%L" # Don't discard the current locale setting.

<Multi_key> <o> <h> <m>      : "Ω"  U03A9
<Multi_key> <O> <underscore> : "Ω"  U03A9
<Multi_key> <underscore> <O> : "Ω"  U03A9

<Multi_key> <p> <h> <y> : "φ"  U03C6
<Multi_key> <o> <bar>   : "φ"  U03C6
<Multi_key> <bar> <o>   : "φ"  U03C6

This takes effect for each newly started applications. Feel free to leave a comment if you know how to add such a support on other
systems.

Perl 6 Advent Calendar: Day 17 – Perl 6 from 30,000 feet

Mon, 17 Dec 2012 00:00:25 +0000

Many people have heard of Perl 6, especially in the greater Perl community. However, Perl 6 has a complicated ecosystem which can be a littled daunting, so as a newcomer to the Perl 6 community myself, I thought I would share what I’ve learned.

How do I install Perl 6?

It’s simple; you can just download one of the existing implementations of the language (as Perl 6 is a specification), build it, and install it! There are several implementations out there right now, in various states of completion. Rakudo is an implementation that targets Parrot, and is the implementation that I will discuss most in this post. Niecza is another implementation that targets the CLR (the .NET runtime). For more information on these implementations and on other implementations, please see Perl 6 Compilers. Perl 6 is an ever-evolving language, and any compiler that passes the official test suite can be considered a Perl 6 implementation.

You mentioned “Parrot”; what’s that?

Parrot is a virtual machine that is designed to run dynamically typed languages. Along with the virtual machine, it includes tools for generating virtual machine code from intermediate languages (named PIR and PASM), as well as a suite of tools to make writing compilers easier.

What is Rakudo written in?

Rakudo itself is written primarly in Perl 6, with some bits of C for some of the lower-level operations, like binding method arguments and adding additional opcodes to the Parrot VM. It may seem strange to implement a Perl 6 compiler in Perl 6 itself; Rakudo uses NQP for building itself.

What’s NQP?

NQP (or Not Quite Perl 6) is an implementation of Perl 6 that is focused on creating compilers for the Parrot Compiler Toolkit. It is currently focused on targetting Parrot, but in the future, it may support various compilation targets, so you will be able to use Rakudo to compile your Perl 6 programs to Parrot opcodes, a JVM class file, or perhaps Javascript so you can run it in the browser. NQP is written in NQP, and uses a pre-compiled version of NQP to compile itself.

I hope that this information was useful to you, dear reader, and that it helps to clarify the different pieces of the Perl 6 ecosystem. As I learn more about each piece, I intend to write blog posts that will hopefully help others to get started contributing to Perl 6!

-Rob

Perl 6 Advent Calendar: Day 16 – Operator precedence

Sun, 16 Dec 2012 00:01:04 +0000

All the precedence men

As I was taking a walk today, I realized one of the reasons why I like Perl. Five as well as six. I often hear praise such as “Perl fits the way I think”. And I have that feeling too sometimes.

If I were the president (or prime minister, as I’m Swedish), and had a bunch of advisers, maybe some of them would be yes-men, trying to give me advice that they think I will want to hear, instead of advice that would be useful to me. Some languages are like that, presenting us with an incomplete subset of the necessary tools. The Perl languages, if they were advisers, wouldn’t be yes-men. They’d give me an accurate view of the world, even if that view would be a bit messy and hairy sometimes.

Which, I guess, is why Perl five and six are so often used in handling messy data and turning it into something useful.

To give a few specific examples:

Perl 5 takes quotes and quoting very seriously. Not just strings but lists of strings, too. (See the qw keyword.) Perl 6 does the same, but takes quoting further. See see the recent post on quoting.
jnthn shows in yesterday’s advent post that Perl 6 takes compiler phases seriously, and allows us to bundle together code that belongs together conceptually but not temporally. We need to do this because the world is gnarly and running a program happens in phases.
Grammars in Perl 6 are not just powerful, but in some sense honest, too. They don’t oversimplify the task for the programmer, because then they would also limit the expressibility. Even though grammars are complicated and intricate, they should be, because they describe a process (parsing) that is complicated and intricate.

Operators

Perl is known for its many operators. Some would describe it as an “operator-oriented” language. Where many other language will try to guess how you want your operators to behave on your values, or perhaps demand that you pre-declare all your types so that there’ll be no doubt, Perl 6 carries much of the typing information in its operators:

my $a = 5;
my $b = 6;

say $a + $b;      # 11 (numeric addition)
say $a * $b;      # 30 (numeric multiplication)

say $a ~ $b;      # "56" (string concatenation)
say $a x $b;      # "555555" (string repetition)

say $a || $b;     # 5 (boolean disjunction)
say $a && $b;     # 6 (boolean conjunction)

Other languages will want to bunch together some of these for us, using the + operator for both numeric addition and string concatenation, for example. Not so Perl. You’re meant to choose yourself, because the choice matters. In return, Perl will care a little less about the types of the operands, and just deliver the appropriate result for you.

“The appropriate result” is most often a number if you used a numeric operator, and a string if you used a string operator. But sometimes it’s more subtle than that. Note that the boolean operators above actually preserved the numbers 5 and 6 for us, even though internally it treated them both as true values. In C, if we do the same, C will unhelpfully “flatten” these results down to the value 1, its spelling of the value true. Perl knows that truthiness comes in many flavors, and retains the particular flavor for you.

Operator precedence

“All operators are equal, but some operators are more equal than others.” It is when we combine operators that we realize that the operators have different “tightness”.

say 2 * 3 + 1;      # 7, because (2 * 3) + 1
say 1 + 2 * 3;      # 7, because 1 + (2 * 3), not 9

We can always be 100% explicit and surround enough of our operations with parentheses… but when we don’t, the operators seem to order themselves in some order, which is not just simple left-to-right evaluation. This ordering between operators is what we refer to as “precedence”.

No doubt you were taught in math class in school that multiplications should be evaluated before additions in the way we see above. It’s as if factors group together closer than terms do. The fact that this difference in precedence is useful is backed up by centuries of algebra notation. Most programming languages, Perl 6 included, incorporates this into the language.

By the way, this difference in precedence is found between other pairs of operators, even outside the realm of mathematics:

      Additive (loose)    Multiplicative (tight)
      ================    ======================
number      +                       *
string      ~                       x
bool        ||                      &&

It turns out that they make as much sense for other types as they do for numbers. And group theory bears this out: these other operators can be seen as a kind of addition and multiplication, if we squint.

Operator precedence parser

Deep in the bowels of the Perl 6 parser sits a smaller parser which is very good at parsing expressions. The bigger parser which parses your Perl 6 program is a really good recursive-descent parser. It works great for creating syntax trees out of the larger program structure. It works less well on the level of expressions. Essentially, what trips up a recursive-descent parser is that it always has to create AST nodes for all the possible precedence levels, whether they’re present or not.

So this smaller parser is an operator-table parser. It knows what to do with each type of operator (prefix, infix, postfix…), and kind of weaves all the terms and operators into a syntax tree. Only the precedence levels actually used show up in the tree.

The optable parser works by comparing each new operator to the top operator on a stack of operators. So when it sees an expression like this:

$x ** 2 + 3 * $x - 5

it will first compare ** against + and decide that the former is tighter, and thus $x ** 2 should be put together into a small tree. Later, it compares + against *, and decides to turn 3 * $x into a small tree. It goes on like this, eventually ending up with this tree structure:

infix:<->
 +-- infix:<+>
      +-- infix:<**>
      |    +-- term:<$x>
      |    +-- term:<2>
      +-- infix:<*>
           +-- term:<3>
           +-- term:<$x>

Because leaf nodes are evaluated first and the root node last, this tree structure determines the order of evaluation for the expression. The order ends up being the same as if the expression had these parentheses:

(($x ** 2) + (3 * $x)) - 5

Which, again, is what we’ve learned to expect.

Associativity

Another factor also governs how these invisible parentheses are to be distributed: operator associativity. It’s the concern of how the operator combines with multiple copies of itself, or other sufficiently similar operators on the same precedence level.

Some examples serve to explain the difference:

$x = $y = $z;     # becomes $x = ($y = $z)
$x / $y / $z;     # becomes ($x / $y) / $z

In both of these cases, we may look at the way the parentheses are doled out, and say “well, of course”. Of course we must first assign to $y and only then to $x. And of course we first divide by $y and only then by $z. So operators naturally have different associativity.

The optable parser compares not just the precedence of two operators but also, when needed, their associativity. And it puts the parentheses in the right place, just as above.

User-defined operators

Now we come back to Perl not being a yes-man, and working hard to give you the appropriate tools for the job.

Perl 6 allows you to define operators. See my post from last year on the details of how. But it also allows you to specify precedence and associativity of each new operator.

As you specify a new operator, a new Perl 6 parser is automatically constructed for you behind the scenes, which contains your new operator. In this sense, the optable parser is open and extensible. And Perl 6 gives you exactly the same tools for talking about precedence and associativity as the compiler itself uses internally.

Perl treats you like a grown-up, and expects you to make good decisions based on a thorough understanding of the problem space. I like that.

Perl 6 Advent Calendar: Day 15 – Phasers set to stun

Sat, 15 Dec 2012 13:28:07 +0000

When writing programs, it’s important not only to separate the concerns that need separating, but also to try and keep related things close to each other. This gives the program a sense of cohesion, and helps to avoid the inevitable problems that arise when updating one part of a program demands an update in another far-away part. One especially tricky problem can be when the things we want to do are distributed over time. This can cause us to move related things apart in order to get them to happen at the times we want.

Phasers in Perl 6 help you keep related concepts together in your code, while also indicating that certain aspects of them should happen at different points during the lifetime of the current program, invocation or loop construct. Let’s take a look at some of them.

ENTER and LEAVE

One of the things I had most fun writing in Perl 6 recently was the debugger. There are various things that need a little care. For example, the debugger needs to look out for exceptions and, when they are thrown, give the user a prompt to let them debug why the exception was thrown. However, there is also a feature where, at the prompt, you can evaluate an expression. The debugger shouldn’t re-enter itself if this expression throws, so we need to keep track of if we’re already showing the prompt. This meant setting and clearing a flag. Thing is, the prompt method is relatively lengthy; it has a given/when to identify the various different commands. I could, of course, have set the prompt flag at the start and cleared it at the end. But that would have spread out the concern of maintaining the flag. Here’s what I did instead:

method issue_prompt($ctx, $cur_file) {
    ENTER $in_prompt = True;
    LEAVE $in_prompt = False;

    # Lots of stuff here
}

This ensures the flag is set when we enter the method, cleared when we leave the method – and lets me keep the two together.

INIT and END

We’re writing a small utility and want to log what happens as we run it. Time wise, we want to:

Open the log file at the start of the program, creating it if needed and overwriting an existing one otherwise
Write log entries at various points during the program’s execution
Close the log file at the end

Those three actions are fairly spread out in time, but we’d like to collect them together. This time, the INIT and END phasers come to the rescue.

sub log($msg) {
    my $fh = INIT open("logfile", :w);
    $fh.say($msg);
    END $fh.close;
}

Here, we use INIT to perform an action at program start time. It turns out that INIT also keeps around the value produced by the expression following it, meaning it can be used as an r-value. This means we have the file handle available to us, and can write to it during the program. Then, at the END of the program, we close the file handle. All of these have block forms, should you wish to do something more involved:

sub log($msg) {
    my $fh = INIT open("logfile", :w);
    $fh.say($msg);
    END {
        $fh.say("Ran in {now - INIT now} seconds");
        $fh.close;
    }
}

Note the second use of INIT in this example, to compute and remember the program start time so we can use it in the subtraction later on.

FIRST, NEXT and LAST

These phasers work with loops. They fire the first time the loop body executes, at the end of every loop body execution, and after the last loop body execution. FIRST and LAST are especially powerful in so far as they let us move code that wants to special-case the first and last time the loop body runs inside of the loop construct itself. This makes the relationship between these bits of code and the loop especially clear, and lessens the chance somebody moves or copies the loop and forgets the related bits it has.

As an example, let’s imagine we are rendering a table of scores from a game. We want to write a header row, and also do a little ASCII art to denote the start and end of the table. Furthermore, we’d like to keep track of the best score each time around the loop, and then at the end print out the best score. Here’s how we could write it.

for %scores.kv -> $player, $score {
    FIRST say "Score\tPlayer";
    FIRST say "-----\t------";
    LAST  say "-----\t------";

    NEXT (state $best_score) max= $score;
    LAST say "BEST SCORE: $best_score";

    say "$score\t$player";
}

Notice how we keep the header/footer code together, as well as being able to keep the best score tracking code together. It’s also all inside the loop, making its relationship to the loop clear. Note how the state variable also comes in useful here. It too is a construct that lets us keep a variable scoped inside a block even if its usage spans multiple invocations of the block.

KEEP and UNDO

These are variants of LEAVE that trigger conditional on the block being successful (KEEP) or not (UNDO). A successful block completes without unhandled exceptions and returns a defined value. An unsuccessful block exits due to an exception or because it returns an undefined value. Say we were processing a bunch of files and want to build up arrays of successful files and failed files. We could write something like:

sub process($file) {
    KEEP push @success, $file;
    UNDO push @failure, $file;

    my $fh = open($file);
    # ...
}

There are probably a bunch of transaction-like constructs that can also be very neatly implemented with these two.

And there’s more!

While I’ve covered a bunch of the phasers here, there are some others. For example, there’s also BEGIN, which lets you do some computation at compile time. Hopefully, though, this set of examples gives you some inspiration in how phasers can be used effectively, as well as a better grasp of the motivation for them. Bringing related things together and setting unrelated things apart is something we need to think carefully about every day as developers, and phasers help us keep related concerns together, even if they should take place at different phases of our program’s execution.

Konrad Borowski: Perl 6 changes - 2012W50

glitchmr@myopera.com (Konrad Borowski) — Sat, 15 Dec 2012 08:00:00 +0000

So, this is yet another article in Perl 6 Changes series. (I know, those introductions are really boring, but would you like to see Lorem Ipsum Dolor instead)? As in previous week, Rakudo mainly improves Perl 6 parsing.

By the way, if you don’t know about Perl 6 Coding Contest, check it out. The challenges are rather easy if you had any programming language experience (if you don’t know Perl 6, check out Using Perl 6 book). If you still have questions, people in #perl6 can help you.

id='incompatible_changes'>Incompatible changes>

Rakudo Perl>

`uniq` has `===` semantics. It could break your programs if you for example were using `uniq` to remove identical arrays (previously they were converted to strings which did what you wanted, as long the strings in array didn’t contained space or you didn’t have array in array.

id='new_features'>New features>

Rakudo Perl>

“Two terms in a row” error message.

Information about what was expected, if possible.

$ perl6 -e '2 2'
===SORRY!===
Two terms in a row
at -e:1
------> 2 ⏏2
    expecting any of:
        postfix
        infix or meta-infix
        infix stopper
        statement end
        statement modifier
        statement modifier loop

for 1..10 { say so 9999 == any(1..2000) } is 30 times faster (huge junctions are faster).
Added Texas versions of the Set and Bag operators, used in Bags and Sets article in Perl 6 Advent Calendar.
Rakudo Perl now informs about obsolete use of . to concatenate strings.
Implemented sub term:<foo>. Now 2 + 2 == 5 can be true for sufficiently huge values of 2 (not like it should be).
qww now works.

Niecza Perl>

`slurp` now works without arguments.

Carl Masak: The 2012 Perl 6 Coding Contest

Fri, 14 Dec 2012 20:21:30 +0000

For the third year in a row, let's write some Perl 6 contest code!

The 2012 Perl 6 Coding Contest

Perl 6 has had a great year, with its two major implementations growing increasingly mature. Sometimes it's easy to forget all the progress that happens each month. Features and performance have improved all through the year. I think that will show in this year's entries.

So here's what's happening:

do five tasks, and you might win Amazon books worth €100!

Addendum: Thanks to an anonymous donor, we're proud to announce a second prize as well: Amazon books worth 100 USD! (Redeemable within a year of winning.)

The contest starts now, today on 2012-12-14. It ends five weeks later, on 2013-01-18. Registration is open for two weeks, starting now. Just send an email to cmasak@gmail.com — saying "sign me up!" or even sending your Amazon wishlist so I'll know which books to buy you if you win.

You might be curious about what you need to do to win. Here's an extract from the file rules.md:

Since "code quality" is a slightly subjective measure, let us provide a few
hints of what we'll be looking for:

* Correctness.
* Readability.
* Consistency.
* Clarity of intent.
* Algorithmic efficiency.
* Idiomatic use of Perl 6.
* Brevity.

In short, what we're looking for is top-quality code. That's how you win.

Here are the five tasks. Write Perl 6 programs to...

Tell knights from knaves based on what they say. (more)
Generate rectangle haikus. (more)
Arrange wire crossings to rearrange wires. (more)
Simulate rain in a world of cubes. (more)
Distribute weights in bags evenly. (more)

I've chosen the problems so that they're easy to explain, but allow contestants quite a bit of freedom to play around with various solutions.

This time, the whole contest is running as a GitHub repository. Easier than hosting zip files.

Now sign up — you've nothing to lose, and books to win! Also, this is your chance to finally take a look at Perl 6. 哈哈

(Looking for previous years' tasks and solutions? Here they are: 2010 tasks and 2010 solutions. 2011 tasks and 2011 solutions.)

Perl 6 Advent Calendar: Day 10 – Don’t quote me on it…

Mon, 10 Dec 2012 00:01:34 +0000

In many areas, Perl 6 provides you with a range of sane defaults for the common cases along with the power to do something a little more interesting when you need it. Quoting is no exception.

The Basics

The two most common quoting constructs are the single and double quotes. Single quotes are simplest: they let you quote a string and just about the only “magic” they provide is being able to stick a backslash before a single quote, which escapes it. Since backslash has this special meaning, you can write an explicit backslash with \\. However, you don’t even need to do that, since any other backslashes just pass on straight through. Here’s some examples.

> say 'Everybody loves Magical Trevor'
Everybody loves Magical Trevor
> say 'Oh wow, it\'s backslashed!'
Oh wow, it's backslashed!
> say 'You can include a \\ like this'
You can include a \ like this
> say 'Nothing like \n is available'
Nothing like \n is available
> say 'And a \ on its own is no problem'
And a \ on its own is no problem

Double quotes are, naturally, twice as powerful. :-) They support a range of backslash escapes, but more importantly they allow for interpolation. This means that variables and closures can be placed within them, saving you from having to use the concatenation operator or other string formatting constructs so often. Here are some simple examples.

> say "Ooh look!\nLine breaks!"
Ooh look!
Line breaks!
> my $who = 'Ninochka'; say "Hello, dear $who"
Hello, dear Ninochka
> say "Hello, { prompt 'Enter your name: ' }!"
Enter your name: Jonathan
Hello, Jonathan!

The second example shows the interpolation of a scalar, and the third shows how closures can be placed inside double quoted strings also. The value the closure produces will be stringified and interpolated into the string. But what about all the other sigils besides “$”? The rule is that you can interpolate all of them, but only if they are followed by some kind of postcircumfix (that is, an array or hash subscript, parentheses to make an invocation, or a method call). In fact, you can put all of these on a scalar too.

> my @beer = <Chimay Hobgoblin Yeti>;
Chimay Hobgoblin Yeti
> say "First up, a @beer[0]"
First up, a Chimay
> say "Then @beer[1,2].join(' and ')!"
Then Hobgoblin and Yeti!
> say "Tu je &prompt('Ktore pivo chces? ')"
Ktore pivo chces? Starobrno
Tu je Starobrno

Here you can see interpolation of an array element, a slice that we then call a method on and even a function call. The postcircumfix rule happily means that we don’t go screwing up your email address any more.

> say "Please spam me at blackhole@jnthn.net"
Please spam me at blackhole@jnthn.net

Choose Your Own Delimiters

The single and double quotes are suitable for a bunch of cases, but what if you want to use a bunch of single or double quotes inside the string? Escaping them would rather suck. Thing is, you could probably make that argument about any choice of quoting characters. So instead of making the choice for you, Perl 6 lets you pick. The q and qq quote constructs expect to be followed by a delimiter. If it’s something with a matching closer, it will look for that (for example, if you use an opening curly then your string is terminated by a closing curly; note that there’s only a finite set of these, and no, it doesn’t include having a comet be terminated by a snowman). Otherwise it looks for the same thing to terminate the string. Note you can also use multi-character openers and closers too (but only by repeating the same character). Otherwise, the q gives you the same semantics as single quotes, and qq gives you the same semantics as double quotes.

> say q{C'est la vie}
C'est la vie
> say q{{Unmatched } and { are { OK } in { here}}
Unmatched } and { are { OK } in { here
> say qq!Lottery results: {(1..49).roll(6).sort}!
Lottery results: 12 13 26 34 36 46

Heredocs

All of the quoting constructs demonstrated so far allow you to include multiple lines of content. However, for that there’s usually a better way: here documents. There can be started with either q or qq, and then with the :to adverb being used to specify the string we expect to find, on a line of its own, at the end of the quoted text. Let’s see how this works, illustrated by a touching story.

print q:to/THE END/
    Once upon a time, there was a pub. The pub had
    lots of awesome beer. One day, a Perl workshop
    was held near to the pub. The hackers drank
    the pub dry. The pub owner could finally afford
    a vacation.
    THE END

The output of this script is as follows:

Once upon a time, there was a pub. The pub had
lots of awesome beer. One day, a Perl workshop
was held near to the pub. The hackers drank
the pub dry. The pub owner could finally afford
a vacation.

Notice how the text is not indented like in the program source. Heredocs remove indentation automatically, up to the indentation level of the terminator. If we’d used qq, we could have interpolated things into the heredoc. Note that this is all implemented by using the indent method on strings, but if your string doesn’t do any interpolation we do the call to indent at compile time as an optimization.

You can also have multiple heredocs, and even call methods on the data that will be located in the heredoc (note the call to lines in the following program).

my ($input, @searches) = q:to/INPUT/, q:to/SEARCHES/.lines;
    Once upon a time, there was a pub. The pub had
    lots of awesome beer. One day, a Perl workshop
    was held near to the pub. The hackers drank
    the pub dry. The pub owner could finally afford
    a vacation.
    INPUT
    beer
    masak
    vacation
    whisky
    SEARCHES

for @searches -> $s {
    say $input ~~ /$s/
        ?? "Found $s"
        !! "Didn't find $s";
}

The output of this program is:

Found beer
Didn't find masak
Found vacation
Didn't find whisky

Quote Adverbs for Custom Quoting Constructs

The single and double quote semantics, also available through q and qq, cover most cases. But what if you have a situation where you want to, say, interpolate closures but not scalars? This is where quote adverbs come in. They allow you to turn certain quoting features on and off. Here’s an example.

> say qq:!s"It costs $10 to {<eat nom>.pick} here."
It costs $10 to eat here.

Here, we use the semantics of qq, but then turn off scalar interpolation. This means we can write the price without worrying about it trying to interpolate the 11th capture of the last regex. Note that this is just using the standard colonpair syntax. If you want to start from a quote construct that supports basically nothing, and then just turn on some options, you can use the Q construct.

> say Q{$*OS\n&sin(3)}
$*OS\n&sin(3)
> say Q:s{$*OS\n&sin(3)}
MSWin32\n&sin(3)
> say Q:s:b{$*OS\n&sin(3)}
MSWin32
&sin(3)
> say Q:s:b:f{$*OS\n&sin(3)}
MSWin32
0.141120008059867

Here we start with a featureless quoting construct, then turn on extra features: first scalar interpolation, then backslash escapes, then function interpolation. Note that we could have chosen any delimiter we wished too.

Quote Constructs are Languages

Finally, it’s worth mentioning that when the parser enters a quoting construct, really it is switching to parsing a different language. When we build up quoting constructs from adverbs, really this is just mixing extra roles into the base quoting language to turn on extra features. For the curious, here’s how Rakudo does it. Whenever we hit a closure or some other interpolation, the language is temporarily switched back to the main language. This is why you can do things like:

> say "Hello, { prompt "Enter your name: " }!"
Enter your name: Jonathan
Hello, Jonathan!

And the parser doesn’t get terribly confused about the fact that the closure being interpolated contains another double quoted string. That is, we’re parsing the main language, then slip into a quoting language, then recurse into the main language again, and finally recurse into the quoting language again to parse the string in the closure in the string in the program. It’s like the Perl 6 parser wants to give us all matryoshka dolls for Christmas. :-)

Konrad Borowski: Perl 6 changes - 2012W49

glitchmr@myopera.com (Konrad Borowski) — Sat, 08 Dec 2012 08:00:00 +0000

To begin with, I’ve made a post for Perl 6 Advent Calendar. If you were wondering why it was late, I will tell you - it’s actually early. Somebody else has claimed article for eighth day and forgot to upload it, so somebody had to put replacement article. I’ve moved my article from 12th day to 8th day, just so Perl 6 Advent Calendar could be continued. Moritz, to protect against that, put the just-in-case article about exceptions (it’s exceptional), if that would happen again.

Enough talking about Perl 6 Advent Calendar, as obviously this blog post isn’t about it. Every week, I describe changes in Perl 6 and its implementations. This is yet another article about changes.

Jonathan Worthington lately was working on improving syntax errors messages in Rakudo Perl. Now they are better and more accurate, just like they’re already in STD.pm6 or Niecza.

id='new_features'>New features>

Rakudo Perl>

Syntax errors now provide context using ⏏ symbol, just like STD.pm6 does.

Rakudo is now more accurate about error locations and doesn’t inform about errors on line 1 (like missing semicolon) that actually are on line 777.

Syntax errors now have colors.

Niecza Perl>

A `.join` method isn’t O(N⁴) anymore.

A `~` infix operator is faster.

Numeric ranges are faster.

Perl 6 Advent Calendar: Perl 6 Advent Calendar 2012: Table of Contents

Sat, 01 Dec 2012 11:59:59 +0000

This post serves as a table of contents for the 2012 Perl 6 advent calendar. Links to new posts will appear here during the course of this month.

See also: table of contents for 2011, 2010, 2009.

Konrad Borowski: Perl 6 changes - 2012W48

glitchmr@myopera.com (Konrad Borowski) — Sat, 01 Dec 2012 08:00:00 +0000

Rakudo Star 2012.11 was released with new impressive improvements (check my previous posts for more information). If you’re reading this, please check out the Perl 6 Advent Calendar - new posts are added daily.

id='incompatible_changes'>Incompatible changes>

Rakudo Star (post Rakudo Star 2012.11)>

Removed `~/.perl6/lib` from `@*INC`.

id='new_features'>New features>

Perl 6 specification>

`Str.indent` ignores empty lines.

Rakudo Perl (in Rakudo Star 2012.11)>

Unknown metacharacters in regexp throw `X::Syntax::Regex::UnrecognizedMetachar`.

Null regexp throws `X::Syntax::Regex::NullRegex`.

Unrecognized backslash sequence throws `X::Backslash::UnrecognizedSequence`.

Non-variable dollar throws `X::Backslash::NonVariableDollar`.

Rakudo Perl (post Rakudo Star 2012.11)>

`--doc` option now works.

`Str.indent` ignores empty lines.

Added `Signature.ACCEPTS`.

Niecza Perl>

`make install` now works.

Konrad Borowski: -2 6

glitchmr@myopera.com (Konrad Borowski) — Fri, 30 Nov 2012 08:00:00 +0000

You know, -2 is highly connected to Perl 6. It’s valid syntax for -(2).

rakudo.org: Rakudo Star 2012.11 released

Wed, 28 Nov 2012 19:12:18 +0000

On behalf of the Rakudo and Perl 6 development teams, I’m happy to announce the November 2012 release of “Rakudo Star”, a useful and usable distribution of Perl 6. The tarball for the November 2012 release is available for download. A Windows .MSI version of Rakudo star will usually appear in the downloads area shortly after the tarball release.

In the Perl 6 world, we make a distinction between the language (“Perl 6″) and specific implementations of the language such as “Rakudo Perl”. This Star release includes release 2012.11of the Rakudo Perl 6 compiler, version 4.6.0 of the Parrot Virtual Machine, and various modules, documentation, and other resources collected from the Perl 6 community.

Some of the new features added to this release include:

heredocs
quote adverbs (like q:w//)
implemented precedence related traits (equiv, looser, tighter, assoc)
Perl 6 grammar NFAs are pre-computed, saving some work on each invocation; this shaved around 10% off the time needed to run the spectests
regexes and quotes have better support for user-selected delimiters
FIRST/NEXT/LAST can now be used in all types of loop (previously limited to for)
several fixes related to module precompilation. This should make working with larger code bases much less painful.

This release also contains a range of performance improvements, bug fixes, improvements to error reporting and better failure modes.

The following features have been deprecated or modified from previous releases due to changes in the Perl 6 specification, and are being removed or changed as follows:

at present, a reference to an &foo that does not exist evalutes to Nil. This will become a CHECK-time failure, in line with STD.
Unary hyper ops currently descend into nested arrays and hashes. This will change to make them equivalent to a one-level map.

~/.perl6/lib will go away from the default include path (@*INC).
Instead %*CUSTOM_LIB now holds paths to four library locations:

perl    Rakudo installs its core modules here
vendor  OS-level package managers should install their modules here
site    for local module installations (e.g. with panda or ufo)
home    like site, but always under the user's home directory.
        fallback if site isn't writable.

The Str.ucfirst builtin is deprecated; it will be replaced by Str.tc.
Leading whitespace in rules and under :sigspace will no longer be converted to <.ws> . For existing regexes that expect this conversion, add a <?> in front of leading whitespace to make it meta again.
The ?-quantifier on captures in regexes currently binds the capture slot to a List containing either zero or one Match objects; i.e., it is equivalent to “** 0..1″. In the future, the ?-quantifier will bind the slot directly to a captured Match or to Nil. Existing code can manage the transition by changing existing ?-quantifiers to use “** 0..1″, which will continue to return a List of matches.

There are some key features of Perl 6 that Rakudo Star does not yet handle appropriately, although they will appear in upcoming releases. Some of the not-quite-there features include:

advanced macros
threads and concurrency
Unicode strings at levels other than codepoints
interactive readline that understands Unicode
non-blocking I/O
much of Synopsis 9

There is an online resource at http://perl6.org/compilers/features that lists the known implemented and missing features of Rakudo and other Perl 6 implementations.

In many places we’ve tried to make Rakudo smart enough to inform the programmer that a given feature isn’t implemented, but there are many that we’ve missed. Bug reports about missing and broken features are welcomed at .

See http://perl6.org/ for links to much more information about Perl 6, including documentation, example code, tutorials, reference materials, specification documents, and other supporting resources. A draft of a Perl 6 book is available as docs/UsingPerl6-draft.pdf in the release tarball.

perl6.announce: Announce: Niecza Perl 6 v24 by Stefan O'Rear

Mon, 26 Nov 2012 23:18:27 +0000

Announce: Niecza Perl 6 v24

This is release number 24 of Niecza Perl 6, marking 24 months since the first
public release. How time flies.

You can obtain a build of Niecza from [1]. This build contains a
working compiler as a set of .exe and .dll files suitable for use with
Mono or Microsoft .NET. If you wish to follow latest developments,
you can obtain the source from [2]; however, you still need a
binary for bootstrapping.

Niecza is a Perl 6 compiler project studying questions about the
efficient implementability of Perl 6 features. It currently targets
the Common Language Runtime; both Mono and Microsoft .NET are known to
work. On Windows, Cygwin is required for source builds only; see the
README for details.

List of changes

[Bug fixes]

.classify and .categorize now return hashes. (Solomon Foster)

qx// is now sandboxed. (Solomon Foster)

[Features]

Added special functions gamma, expm1, log1p, erf, lgamma. (Solomon Foster,
using library code by John D. Cook)

Export $*CUSTOM-LIB for panda's use. (Solomon Foster)

Added Parcel.Array and List.Array. (Solomon Foster)

Postfix operaters which mimic methods can now be used. (Larry Wall)

Added functionality for a "precompiled system modules" folder, eliminating
the need for each user to separately compile the setting in a multi-user
install.

Added line editing to the REPL.

Added an install target for *nixes.

[Internal]

Niecza has been significantly refactored to use a single instance of the
kernel during compilation, opening the door to direct object sharing
between the compiler and the runtime and significantly simplifying the
bootstrap procedure.

Getting involved

Contact sorear in irc.freenode.net #perl6 or via the sender address of
this mailing.

[1] https://github.com/downloads/sorear/niecza/niecza-24.zip
[2] https://github.com/sorear/niecza

Konrad Borowski: Perl 6 changes - 2012W47

glitchmr@myopera.com (Konrad Borowski) — Sat, 24 Nov 2012 08:00:00 +0000

So, well, now that my code compiles (well, C code, not Perl 6 code :-)) I’ve time to describe Perl 6 changes, instead of simply doing what XKCD says - I shouldn’t waste time.

Rakudo Perl 2012.11 is now released. The change mentioned in this article is already in.

id='new_features'>New features>

Perl 6 specification>

`NativeCall` module is now part of specification.

Calls to `trait_mod` routines are evaluated in sink context.

STD.pm6>

`-->` (return type) acts more like a normal subroutine parameter.

Rakudo Perl>

`:($a, $b) := \(1, 2)` updates `$a` and `$b` instead of doing nothing.

Jan Ingvoldstad: No such pipe, or this pipe has been deleted

Fri, 23 Nov 2012 08:42:40 +0000

This data comes from pipes.yahoo.com but the Pipe does not exist or has been deleted.

(lue): The Rakudo Codebase: Visualized! (Partially)

Thu, 22 Nov 2012 08:59:05 +0000

Yesterday I stumbled upon this old perl6-compiler mailing list message which inspired me to actually try to split up the compilation of CORE.setting into smaller, saner pieces. I’ve started work on this already (so far having just modified the Makefile).

The one and only response to that post suggested using the stub syntax to resolve missing class issues. Sadly, that does not work, because perl6 needs to see the stubbed class in full sometime later in the file.

So, the best option I’ve seen is to include the files containing needed classes and roles into files that need them (through either use or require). This requires knowing what to include (as including everything everywhere ruins the point), and I’ve started to try to grasp what’s going on in the rakudo codebase, specifically src/core, in terms of dependencies.

Here’s what I have so far. It’s right now just one, single script. This single script graphs the inheritance chain of all the classes and roles in src/core (what classes and roles “is” and “does”). You need the modules Term::ANSIColor, Term::ProgressBar, and IO::Capture::Simple (at this time pulled in, but apparently unused, by Term::ProgressBar). You also need ack and graphviz (which is not used in the script, but needed to process the resulting core.dot file).

Wanna see the graph for nom commit bf472b0, the latest as of this writing? Here you go (click to embiggen, clearly):

The arrows point towards what a particular box is inheriting, and the one or two white boxes mean that the class or role doesn’t exist within the src/core files. Yellow denotes classes, red denotes roles. And each box contains the name of the class/role, and the file it was found in and the line number of its definition.

This, however, doesn’t even begin to deal with all the dependencies abound in the src/core files. See all those abandoned colored blocks? Those aren’t inherited by anything in src/core, but they most certainly are utilized (e.g. the Pod::Config block, though not inherited, could easily be utilized as a variable type in the Pod code). I have a lot more work to do before I have a clear picture on how to separate CORE.setting into smaller pieces. (I might even need to utilize STD.pm6 or similar to do all the parsing I’ll eventually be doing!)

Just to finish off, Here are a couple more graphs of the same data. The first graph above was generated with GraphViz’s “dot”. This one (generated by their “twopi” program) I think looks hilarious:

And this picture was generated by GraphViz’s “fdp”. I think this one is the easiest to follow.

Happy viewing!

perl6.announce: Parrot 4.10.0 "Red-eared Parakeet" Released by Reini Urban

Wed, 21 Nov 2012 17:27:35 +0000

On behalf of the Parrot team, I'm proud to announce Parrot 4.10.0, also known
as "Red-eared Parakeet". Parrot (http://parrot.org/) is a virtual machine aimed
at running all dynamic languages.

Parrot 4.10.0 is available on Parrot's FTP site
(ftp://ftp.parrot.org/pub/parrot/releases/devel/4.10.0/), or by following the
download instructions at http://parrot.org/download. For those who would like
to develop on Parrot, or help develop Parrot itself, we recommend using Git to
retrieve the source code to get the latest and best Parrot code.

Parrot 4.10.0 News:
- Core
+ Add PMCNULL check to result_info op
+ Fixed record separator bug on io handles
- Build
+ Add -fpermissive so that nqp builds under gcc 4.7.2
+ Fixed warnings for old ICU 4.4
- Platforms
+ More VMS fixes, but not yet done
- Branches
+ The threads branch is ready to be merged, but nqp needs some changes,
so it will be merged with 4.11.0.
- Examples
+ Parrot mini-web server fixes

The SHA256 message digests for the downloadable tarballs are:
82f38e6e596ab101c52313ad3e3560d2c8a72985362b54d9898619ab17a6f17d
parrot-4.10.0.tar.gz
7d6a592c386d25bcee429a269cc571ae81c5c080b15d14a884401a27fe3f5d4b
parrot-4.10.0.tar.bz2

Many thanks to all our contributors for making this possible, and our sponsors
for supporting this project. Our next scheduled release is 18 December 2012.

The release is a day late. I apologize for the lateness.
--
Reini Urban
http://cpanel.net/ http://www.perl-compiler.org/

Konrad Borowski: Perl 6 changes - 2012W46

glitchmr@myopera.com (Konrad Borowski) — Sat, 17 Nov 2012 08:00:00 +0000

I’m already aware that my blog looks spammy because of all those Perl 6 changes posts. I would like to remove all that garbage, so I’m thinking about separate site for those posts (of course with proper redirects of my old URLs).

id='deprecations'>Deprecations>

Rakudo Perl>

In Rakudo Perl 2012.12, `&foo` when `sub foo` isn’t declared will make CHECK-time exception instead of returning `Nil`.

id='new_features'>New features>

STD.pm6>

When `<infix><prefix><space>` sequence appears, user gets warning. This helps avoiding code like `$a ~ $b` which actually means `$a (~$b)`.

Empty infix operator (`sub infix:[""]`) isn’t allowed anymore.

Rakudo Perl>

BOM (Byte Order Mark) at beginning of file is ignored.

`--optimize` isn’t passed by default to Parrot when `--parrot-option` is specified.

Niecza Perl>

Niecza doesn’t depend on global `Top` variable anymore allowing multiple Niecza interpreters running at once. It was being removed for two months.

Konrad Borowski: Perl 6 changes - 2012W45

glitchmr@myopera.com (Konrad Borowski) — Sun, 11 Nov 2012 08:00:00 +0000

It’s another late Perl 6 changes article. Today the fun parts are adverbs, adverbs and adverbs. Oh, and blocks. And fixing bugs. Have fun playing with new features.

id='new_features'>New features>

Rakudo Perl>

`INIT {}` block is now rvalue.

`our ($x, $y)` works correctly.

`take` and `take-rw` have correct return values.

`SET_FILE_LINE` is now called only on exceptions that support it.

`FIRST`, `NEXT` and `LAST` blocks are supported on various kinds of loops, previously it only worked for `for` loops.

Implemented operator adverbs.

Associative objects now support `:exists` and `:delete` adverbs.

`q:c[]` (string with interpolated closures) now works.

Implemented slice adverbs, such as `:p`, `:kv`, `:k` or `:v`.

Jonathan Worthington (6guts): Lots of improvements coming in the Rakudo November release

Wed, 07 Nov 2012 23:27:00 +0000

The November release is still a couple of weeks off, but it’s already looking like one of the most exciting ones in a while. Here’s a rundown of the major improvements you can expect.

User Defined Operator Improvements

The way Rakudo handles parsing of user-defined operators has been almost completely reworked. The original implementation dated back quite a long way (we’re talking largely unchanged since probably 2010 or so), and as you might imagine, we’ve the tools to do a lot better now. The most significant change is that we now parse them using a lexically scoped derived language. This is done by mixing in to the current cursor’s grammar – something we couldn’t have done some years ago when NQP lacked roles! This means that the additions to the grammar don’t leak outside of the scope where the user-defined operators are defined – unless they are explicitly imported into another scope, of course.

{
    sub postfix:<!>($n) { [*] 1..$n };
    say 10!; # 3628800
}
say 10!; # fails to parse

Before, things were rather more leaky and thus not to spec. Perl 6 may allow lots of language extensibility, but the language design takes great care to limit its scope. Now Rakudo does much better here. As well as making things more correct, the nasty bug with pre-compilation of modules containing user defined operators is gone. And, last but not least, the precedence and associativity traits are now implemented, so user defined operators can pick their precedence level.

sub infix:<multiplied-with>($a, $b) is equiv(&infix:<*>) {
    $a * $b
}
sub infix:<to-the-power>($a, $b) is equiv(&infix:<**>) {
    $a ** $b
}
say 2 multiplied-with 3 to-the-power 2;

Quote Adverbs and Heredocs

The other thing that’s had a big do-over is quote parsing. This meant a lot of quoting related things that hadn’t been easy to attack before became very possible. So possible, in fact, that they’ve been done. Thus, we now support heredocs:

say q:to/DRINKS/
    Yeti
    Leffe
    Tokyo
    DRINKS

This outputs:

Yeti
Leffe
Tokyo

Notice how it trims whitespace at the start of the string to the same level of degree of indentation as the end marker, so you can have heredocs indented to the same level your code is, if you wish. The :to syntax is an example of a quote adverb, and we now support those generally too. For example, a double-quoted string normally interpolates a bunch of different things: scalars, arrays, function calls, closures, etc. But what if you want a quoting construct that only interpolates scalars? You do something like:

my $value = 42;
say Q:s'{ "value": $value }';

Alternatively, you could take a quoting construct that normally interpolates everything and just switch off the closure interpolation:

my $value = 42;
say qq:!c'{ "value": $value }';

Last but not least, the shell words syntax now works. This lets you do quoting where things are broken up into a list by whitespace, but you can quote individual parts to prevent them
getting split, and also do interpolation.

say << Hobgoblin 'Punk IPA' >>.perl; # ("Hobgoblin", "Punk IPA")

Sadly, I’ve got too bad a headache today to enjoy any of the listed beers. But hey, at least now you know some good ones to try… :-)

Operator Adverbs

Well, weren’t these some fun to implement. STD has parsed them for a long while. You’d think that’d make it easy to steal from, but no. Before you can do that, you have to figure out that not only does it parse adverbs as if they were infixes, but then pushes them onto the op stack in the operator precedence parser and does a reduce such that they come out looking like postfixes, which in turn need an AST transform to turn the postfix inside-out so what was originally parsed as a fake infix becomes a named argument to the thing preceding it. I’m still not sure if this is beautiful or horrifying, but now it’s in place. This means we can at last handle the adverb-y syntax for doing things like hash key existence checks or deletion:

my %h = a => 0;
say %h<a>:exists; # True
say %h<a>:delete; # 0
say %h<a>:exists; # False

Macros

Recent work by masak++ has got macros to the point where they’re powerful enough to be potentially useful now. See his blog post for the sometimes-gory details, illustrated with exciting adventures involving wine barrels and questionably pluralized bits of Russian landscape. Really.

NFA Precomputation

This is decidedly a guts change, but worth a mention because it’s probably the main performance improvement we’ll see in the November release: the transitive NFAs that are used by the parser are now computed at the time we compile the Perl 6 grammar, not on-demand as we start parsing. They are then serialized, so at startup we just deserialize them and dig into the parsing. They’re not entirely cheap to construct, and so this saves a bit of work per invocation of the compiler. In the spectests, it made an almost 10% difference to the time they take to run (many test files are quite small, and we run over 700 test files, so reducing invocation overhead adds up).

By the way, if you’re confused on what these transitive NFAs are about, let me try and explain a little. In order to decide which of a bunch of protoregexes or which branch of an alternation to take, the declarative parts of a grammar are analyzed to build NFAs: state machines that can efficiently decide which branches are possible then rank them by token length. It’s not only important for a correct parse (to get longest token matching semantics correct), but also important algorithmically. If you look at the way Perl 6 grammars work, the naive view is that it’s recursive descent. That makes it nice to write, but if it really was evaluated that way, you’d end up trying loads of paths that didn’t work out before hitting upon the correct one. The NFA is used to trim the set of possible paths through the grammar down to the ones that are actually worth following, and their construction is transitive, so that we can avoid following call chains several levels deep that would be fruitless. If you’ve ever used the debugger to explore a grammar, and wondered how we seem to magically jump to the correct alternative so often, well, now you know. :-)

Other Things

There are a few other bits and pieces: INIT phasers now work as r-values, you can use the FIRST/NEXT/LAST phasers in all the looping constructs now (previously, they worked only in for loops), version control markers left behind in code are gracefully detected for what they are, and a bunch of proto subs in the setting have been given narrower signatures which makes various things you’d expect to work when passed to, say, map, actually do so.

Oh, and whatever else we manage to get around to in the next couple of weeks. :-)

Konrad Borowski: Perl 6 changes - 2012W44

glitchmr@myopera.com (Konrad Borowski) — Sun, 04 Nov 2012 07:00:00 +0000

I apologize for being late, Gabor Szabo. I simply forgot about writing this. This week, the main attraction is proper quote constructs support.

Incompatible changes

Rakudo Perl

IO::Path.dir was removed. You’re expected to use IO::Path.directory instead. Also, named parameter in IO::Path constructor was renamed from :dir to :directory.

New features

STD.pm6

Postfix operators like .foo-bar take precedence above method calls.

Rakudo Perl

Better delimiter handling. For example, rx and m allow delimiters other than // or {}.
Quote adverbs, such as :w or :x are supported.
Implemented andthen infix operator.
You can specify input line separator on new sockets using :input-line-separator named argument.
Quotes have better error reporting.
Heredocs are implemented.
Macro arguments now carry lexical envirnoment properly.
Postfix operators like .foo-bar take precedence above method calls.
Git merge conflicts markers are now detected properly.

Niecza Perl

qw is unsafe and cannot be used in safe (sandbox) mode.
Postfix operators like .foo-bar take precedence above method calls.

Konrad Borowski: Perl 6 changes - 2012W44

glitchmr@myopera.com (Konrad Borowski) — Sun, 04 Nov 2012 07:00:00 +0000

I apologize for being late, Gabor Szabo. I simply forgot about writing this. This week, the main attraction is proper quote constructs support.

id='incompatible_changes'>Incompatible changes>

Rakudo Perl>

`IO::Path.dir` was removed. You’re expected to use `IO::Path.directory` instead. Also, named parameter in `IO::Path` constructor was renamed from `:dir` to `:directory`.

id='new_features'>New features>

STD.pm6>

Postfix operators like `.foo-bar` take precedence above method calls.

Rakudo Perl>

Better delimiter handling. For example, `rx` and `m` allow delimiters other than `//` or `{}`.

Quote adverbs, such as `:w` or `:x` are supported.

Implemented `andthen` infix operator.

You can specify input line separator on new sockets using `:input-line-separator` named argument.

Quotes have better error reporting.

Heredocs are implemented.

Macro arguments now carry lexical envirnoment properly.

Postfix operators like `.foo-bar` take precedence above method calls.

Git merge conflicts markers are now detected properly.

Niecza Perl>

`qw` is unsafe and cannot be used in safe (sandbox) mode.

Postfix operators like `.foo-bar` take precedence above method calls.

Carl Masak: Macros progress report: D2 merged

Fri, 02 Nov 2012 16:40:43 +0000

The grant that I'm currently working on, the macros grant, has now reached its D2 milestone. That is, the so-called "unquotes" work as advertised in Rakudo:

macro apply($code, $argument) {
    quasi {
        {{{$code}}}( {{{$argument}}} );
    }
}

apply sub ($t) { say $t }, "OH HAI";    # prints "OH HAI"

Macros are routines, and so they take parameters. The above apply macro takes some $code and an $argument, and calls the former with the latter. It's as if, when the macro expansion is all done, what's left in the code is the following line:

(sub ($t) { say $t })("OH HAI");    # prints "OH HAI"

Of course, we never actually see this line, and in the compiler it's never textually substituted like that, because the substitution all happens on the level of syntax trees, not on the level of text.

The new thing in this picture is the {{{ }}} thingies: the so-called unquotes. Back in my last progress report, we still didn't have unquote support in nqp. Now we do. In fact, we got unquote support already back in August. That, it turns out, was the easy bit.

Then the conceptual problems appeared. For a few months, whenever I thought about macros, my brain would melt trying to think about those problems. It took quite a while to go from unquotes existing to them being actually useful. What follows is an explanation of the problem and its solution.

The problem is one of context. By that, we mean the variable bindings seen by a piece of code. Psychologically, we expect a piece of code to see the variables in its lexical environment, that is, all the variables declared in all surrounding blocks.

my $a;

sub f {
    my $b;
    sub {
        my $c;
        # $a, $b, $c visible here
    }
}

The exciting and highly useful thing about closures is that they honor this expectation, while simultaneously being first-class values that you can pass around between parts of your program. This combination of static bindings and dynamic function values is so powerful that you can use it to emulate the object encapsulation so espoused by OO enthusiasts.

In the above case, the sub f implicitly returns its inner sub, which can be transported across the Russian tundras, stored in a dank wine cellar for 75 years before being uncorked... but when finally called, it will still remember its $a, $b, and $c bindings. That's because closures aren't just containers of statements. They also hold a reference to an OUTER block through which variable lookups can be made.

(And in the above case, $b and $c are properly encapsulated. $a isn't, since it's globally visible.)

We want macros to behave the same. That is, quasi blocks should behave like closures.

macro f {
    my $a = "OH HAI";
    quasi {
        say $a;
    }
}

my $a = "B... BOOOOM!";
f;      # OH HAI

It's the same principle: after the f; call has been conceptually replaced by say $a; this code should still remember its context, its origins, namely the macro body. The fact that say $a; doesn't print "B... BOOOOM!", from the variable in the mainline scope, is part of what's called hygiene. Hygiene means that just like with closures, bindings inside are isolated from bindings outside by default.

(The term "hygiene" is often conflated in people's minds with the term "AST-based macros". The two are not the same. AST-based macros are necessary but not sufficient for hygiene. End of rant.)

But wait a minute. These two situations are obviously very similar. In the case of the closure, we know that the closure must keep an OUTER reference to remember its context. What is it in the macro case that remembers the context?

The quasi construct generates an AST, a syntax tree, that then gets spliced into the mainline code where the f; call used to be. This AST must be the vessel for the context information. So, just like a closure is a bunch of statements plus a context, an AST object must be a tree plus the context information. If the AST didn't have a context, the above macro expansion couldn't be hygienic.

We must perform unholy surgery on the block that eventually results from the quasi AST. The block will naturally have mainline context, but we want to recontext it to have macro context. So in the Rakudo macro expansion code, there is some code that transplants the context from the AST object to the new block. It involves a Rakudo-specific op called perl6_get_outer_ctx. It's only used for this code path.

This much was clear already when I was merging D1. Now for the new complications.

Macro expansions consist of two stages of substitution, and this is what makes them useful:

Unquotes are replaced by ASTs, typically arguments originating from the outisde of the macro.
The macro invocation is replaced by the AST returned from applying the macro to its arguments.

When implementing D1, I sorted out my thoughts by writing lots of ruminating gists. During this phase of the work, I've composed fewer gists, but an unexpected thing happened: the more time passed, the more I realized how much I had misnamed the variables in the macro code I had contributed to Rakudo.

It wasn't that I was careless about naming when I first wrote that code. Instead, my understanding of the macro domain had shifted so much that the choices of names I had made started to feel wrong. Today I landed a long-awaited refactor which not only unified the three macro-invocation code paths, but also fixed all the now slighty-off variable names. Quite a relief.

Here's part of what changed. During D1, a lot of AST objects in source ended up being called quasi ASTs. Nowadays, the following distinction is made:

quasi ASTs are what quasi blocks generate. Naturally.
argument ASTs are the things that the parser generates as it parses the macro arguments, just before it invokes the macro.
macro ASTs are what's returned from a macro, to be spliced back into the mainline code.

There is overlap. A macro AST has been generated as either a quasi AST or an argument AST at some point or other. But the focus here is where the ASTs are coming from. And it turns out that matters a lot. Quasi ASTs and argument ASTs are quite different. Hence the need for precision.

By the way, there is possibly a fourth kind of AST, one that we don't have yet, but that is totally possible once people start building macro libraries and stuff:

synthetic ASTs, syntax trees built up programmatically from individual AST nodes or smaller ASTs.

Don't know yet if that's going to become a reality. Until it does, quasi blocks fill much the same role.

Once we had unquotes working in Rakudo, the one glaring omission was that the unquotes didn't behave hygienically. Which was a shame because, again, people really expect hygiene to work:

macro test($value) {
    my $a = "B... BOOOOM!";
    quasi {
        say {{{$value}}};
    }
}

my $a = "OH HAI";
test $a;    # OH HAI nowadays, used to B... BOOOOM!

Just as the quasi AST should remember its own original context, so should the argument AST that ends up in $value. It used not to, and so the context it got was the quasi's, resulting in "B... BOOOOM!" above. A little ironic that it was the successful recontexting of the quasi AST that messed things up for the argument AST.

For months I struggled with the problem of how to recontext the argument ASTs. I developed a solution in a branch, which finally worked as it should, except that it still didn't recontext the ASTs properly! Argh!

My plan of attack had been to set the context at the time of unquote evaluation, as the quasi is evaluated when running the macro. The other day, jnthn pointed out that this approach may be overly complicated: maybe the context could be set at the time of argument collection, just before calling the macro. This is definitely simpler. Not least because at this point, the parser actually is in the context it wants to set! And in particular, no block surgery was needed this time.

I tried it. It worked. This solution almost feels too simple, and I'm not sure yet it will let us do all the things we want to do. But all the tests pass, and I have hammered this solution with tricky situations that might break, and it's holding up so far. So, we now have hygienic macros with unquotes in Rakudo.

Here are the macro-related gists that I wrote during this period. They are in various states of obsolescence at this point, but still potentially informative:

The other artifacts that have emerged since D1 are as follows:

A new spectest file. Also, thanks to a suggestion by moritz++, the macro spectest files are now much better named.
A number of commits to the nom branch of Rakudo:
A number of commits to the nqp project:
Two more deliveries of the macros talk: one at French Perl Workshop in Strasbourg, and one at YAPC::Europe in Frankfurt.

And once again, it's time to glance at what's ahead in the grant work. D3 promises to deliver hygiene. As explained above, D2 already provides this; I actually could have declared the milestone D2 finished at the point I got unquotes working in Rakudo (in August), but it felt slightly disingenuous to do so, because unquotes aren't really useful until they're fully hygienic. Anyway, half of D3 ends up being already done. What still needs to be implemented is the COMPILING:: pseudopackage, which gives the macro author several ways to opt out of hygiene. This is sometimes very powerful, even if it makes sense for it not to be the default.

My grant reports have been sparse lately. I'm hopeful that the wait until the next one won't be as long.

rakudo.org: Rakudo Star 2012.10 released

Mon, 29 Oct 2012 22:24:02 +0000

On behalf of the Rakudo and Perl 6 development teams, I’m happy to announce the October 2012 release of “Rakudo Star”, a useful and usable distribution of Perl 6. The tarball for the October 2012 release is available from <http://github.com/rakudo/star/downloads>. A Windows .MSI version of Rakudo star will usually appear in the downloads area shortly after the tarball release.

In the Perl 6 world, we make a distinction between the language (“Perl 6″) and specific implementations of the language such as “Rakudo Perl”. This Star release includes release 2012.10 of the Rakudo Perl 6 compiler, version 4.6.0 of the Parrot Virtual Machine, and various modules, documentation, and other resources collected from the Perl 6 community.

Some of the new features added to this release include:

* The method case of the handles trait

* The <-> lambda, which defaults parameters to rw

* The :dba modifier in regexes

* The radix form :60[24, 59, 59]

* Improved coverage of Perl 5 regexes when the P5 modifier is used

* Correct parsing of nested quote delimiters

* Attributes in scope are now visible inside of an eval

This release also contains a range of performance improvements, bug fixes, improvements to error reporting and better failure modes.

The following features have been deprecated or modified from previous releases due to changes in the Perl 6 specification, and are being removed or changed as follows:

* Unary hyper ops currently descend into nested arrays and hashes.
This will change to make them equivalent to a one-level map.

* ~/.perl6/lib will go away from the default include path (@*INC).
Instead %*CUSTOM_LIB now holds paths to four library locations:
perl    Rakudo installs its core modules here
vendor OS-level package managers should install their modules here
site    for local module installations (e.g. with panda or ufo)
home    like site, but always under the user’s home directory.
fallback if site isn’t writable.

* protos for built-in routines are now mostly as generic as possible,
and will be changed to be specific to the arity of the routine.
For example ‘proto sub chr(|) {*}’ will become ‘proto sub chr($) {*}’
This affects everybody who adds multis with unusual arity to built-in
subs.

* Parameters preceded by a | or \ can no longer have a sigil.

* IO::Path.dir (which returns the directory part of the path) has been
renamed to IO::Path.directory. IO::Path.dir will be removed or
re-purposed.

* The Str.ucfirst builtin is deprecated; it will be replaced by Str.tc.

* Leading whitespace in rules and under :sigspace will no longer be
converted to <.ws> . For existing regexes that expect this conversion,
add a <?> in front of leading whitespace to make it meta again.

* The ?-quantifier on captures in regexes currently binds the capture
slot to a List containing either zero or one Match objects; i.e., it
is equivalent to “** 0..1″. In the future, the ?-quantifier will
bind the slot directly to a captured Match or to Nil. Existing code
can manage the transition by changing existing ?-quantifiers to
use “** 0..1″, which will continue to return a List of matches.

There are some key features of Perl 6 that Rakudo Star does not yet handle appropriately, although they will appear in upcoming releases. Some of the not-quite-there features include:

* advanced macros
* threads and concurrency
* Unicode strings at levels other than codepoints
* interactive readline that understands Unicode
* non-blocking I/O
* much of Synopsis 9

There is an online resource at http://perl6.org/compilers/features that lists the known implemented and missing features of Rakudo and other Perl 6 implementations.

In many places we’ve tried to make Rakudo smart enough to inform the programmer that a given feature isn’t implemented, but there are many that we’ve missed. Bug reports about missing and broken features are welcomed at <rakudobug@perl.org>.

See http://perl6.org/ for links to much more information about Perl 6, including documentation, example code, tutorials, reference materials, specification documents, and other supporting resources. A draft of a Perl 6 book is available as <docs/UsingPerl6-draft.pdf> in the release tarball.

The development team thanks all of the contributors and sponsors for making Rakudo Star possible. If you would like to contribute, see <http://rakudo.org/how-to-help>, ask on the perl6-compiler@perl.org mailing list, or join us on IRC #perl6 on freenode.

Konrad Borowski: Perl 6 changes - 2012W43

glitchmr@myopera.com (Konrad Borowski) — Sat, 27 Oct 2012 07:00:00 +0000

Rakudo Star 2012.10 will be released soon, but before that will happen, I’ll list changes in Perl 6 specifications and implementations. Not many changes, but many of those are very important. Personally, I find custom operators precedences, << >> quotes and prototypes in core subroutines very impressive improvements - it was all done in one week.

id='new_features'>New features>

Rakudo Perl>

Custom operators can now have precedences.

`<< >>` quotes now support interpolation.

Perl 6 grammars are pre-computed.

`eval` now sees custom operators.

Not-found dynamic values make better error messages.

`${variable}` now makes better error message.

`read` now exists again in `IO::Socket`.

Many core subroutines now have prototypes. This allows doing stuff like `@array.map: &sin`.

Carl Masak: Sweet ports

Fri, 26 Oct 2012 20:19:37 +0000

I had invited jnthn over because I had invested in a bottle of nice port wine that I really wanted to try, but one whole bottle was too much for me. For both of us, during the course of an afternoon, it was about right.

So that was the premise. Because we both like bad puns — jnthn admittedly more than I do — we decided to make it a "porting hackathon", where we would port some piece of software to Perl 6 while sipping the sweet beverage from Portugal.

Which piece of software did we end up porting?

JSON::Path from CPAN. It's a module that does for JSON what XPath did for XML.

Well, actually JSON::Path is not very tied to the JSON format at all. It just expects your data to be a hierarchy of arrays, hashes, and scalar values, just like JSON.

Here are some JsonPath examples, just to give you a taste of it:

$.class.student[0].name           name of first student in class hash
$['class']['student'][0]['name']  same, with an alternate notation
$.beers[*]                        all beers
$.beers[*].name                   names of all beers
$..author                         recursively find all 'author' entries

See here for a more detailed specification.

Any nice insights along the way?

Yes, we found a pattern which we really liked. Not sure what it's called, or if it has a name. We're certainly not the first to come across it — it's probably well-known in FP circles. But as soon as we came up with the idea, the rest of the design basically fell into place.

Perhaps the easiest way to explain the pattern is to say this: our action methods make little subroutines.

method command:sym<.>($/) {
    my $key = ~$<ident>;
    make sub ($next, $current, @path) {
        $next($current{$key}, [@path, "['$key']"]);
    }
}

This is especially apt, because the JsonPath language is all about how to find data in a hierarchical data structure. So each little subroutine outlines how one particular piece of syntax digs further down into the data structure. In the above case, it's saying that a JsonPath fragment such as

.foo

will be translated to Perl 6 code such as

$current{'foo'}

All the other action methods also return little anonymous subs like this one. To make it all work, the grammar is structured so that the fragments end up nesting around each other from left to right:

token commandtree {
    <command> <commandtree>?
}

The resulting AST comes out looking like a Matryoshka doll of commandtree nodes. Or a chain where each link is generated by its own specific rule. The links of the chain are forged together by the commandtree action method, that uses assuming to pre-set the $next parameter of the anonymous functions:

method commandtree($/) {
    make $<command>.ast.assuming(
        $<commandtree>
            ?? $<commandtree>[0].ast
            !! #`[create a function that returns the result];
}

And that's it. Long ago in the mists of computer history, AI researchers must have felt that intelligence would sprout from depths of LISP because they knew about patterns like this one: how to build up complex behavior by forging together little links in a chain, each one a simple "atom" of behavior. At least that's how it felt to see this experiment work.

It also made me tweet this:

Yes, but how was the port?

Oh, it was pretty nifty. I've decided to provisionally like port wine after this. It's fruity and sweet, and goes really well with cheese, or quality chocolate.

Here's a picture of the finished bottle, along with all the tests passing:

Our finished module can be found at Github.

Jonathan Worthington (6guts): Rakudo Debugger Updates

Wed, 24 Oct 2012 15:39:29 +0000

A while ago I wrote about Rakudo getting an interactive debugger. The feedback I’ve got has been a happy mixture of “this is useful” and “I’d really like it to X” – so I’ve been working on some improvements. I showed them off at my Nordic Perl Workshop talk the weekend before last, in a fun session where I used the debugger to hunt down some problems in a small tool that had a couple of modules. It was nice not only to demonstrate the debugger itself, but also because I could show some neat Perl 6 code along the way.

So, what new things can you expect from the debugger that will be bundled in the Rakudo Star release that will be emerging in the next few days? Here’s a quick rundown.

Attributes that are in scope can now be introspected, just by typing their name. Additionally, the self keyword is recognized as a variable name by the debugger, meaning you can look at it directly (just type “self” and press enter) or even to look at public attributes (“self.name”).

Before, if you were on a line of code that was going to make a call, you would always step in to the callee on pressing enter. Now, if you type “s” and press enter, you will step over the call. Got bored of debugging the current routine and want to let it run its course, and break on the next statement after it returns? Just type “so” and press enter, and you will step out of the current routine.

Trace points are perhaps the most powerful of the new features. They enable you to add print statements to your code, without actually adding print statements to your code. A trace point is like a break point, but instead of breaking, it just evaluates an expression and logs it. Later on, you can view the log.

To go with Rakudo’s improving support for the P5 adverb on regexes, which allows the use of Perl 5 regex syntax from within Perl 6, the debugger now also supports single-stepping through those Perl 5 regexes.

There are also a range of small fixes and tweaks that avoid some of the noise you could get before. For example, fails no longer count as exception throws that cause a rt (run until throw) to break, and sigspace in rules is no longer single-stepped, just jumping straight to the next interesting atom.

What of plans for the future? There are some other ideas already in the issues queue. Beyond those, I’m planning to make a web-based API frontend to the debugger, to go alongside the command line one. This should allow a browser-based debugging interface to be built, but should also enable tools like Padre to integrate with the Rakudo Debugger.

Enjoy the updates!

Konrad Borowski: Travis - add and forget

glitchmr@myopera.com (Konrad Borowski) — Sun, 21 Oct 2012 07:00:00 +0000

So, you may have heard that Travis is nice build testing service. It is - I even use it for my JavaScript projects. But, guess what will happen if somebody uses Travis incorrectly.

The problem is about one of the most commonly languages made by bad developers for bad developers. But this article isn’t about why this language is bad - we all know it. Technically, I would use PHP instead of Brainfuck if I would have choice between PHP and Brainfuck, but it would be choice between very bad and worse.

6 months ago, Davey Shafik though - everybody uses Travis CI, why PHP source code doesn’t. He wanted to add Travis CI to his projects, but after quick look he noticed he doesn’t have test cases in his code and he was too lazy to add them. So, why not add Travis CI to some project that already has test cases so he could see how Travis CI would work.

And so he did. He decided to ask PHP to add Travis CI to its source code. Of course, the reaction is obvious.

This is awesome, thanks!

If this gets merged in and you guys are interested in pull-request testing then shoot us a note at contact@travis-ci.org. This feature is currently being rolled out manually but we’d love to turn it on for php.

Cool, Travis is awesome.

FYI looking into it. General idea is great, have to read about travis, etc first.

After general “awesome” comments (seriously, do you know other words?) the code review started. For example, David Soria Parra complained about !! usage

define('TRAVIS_CI' , !!getenv('TRAVIS_PHP_VERSION'));

Of course, version below is better. I mean, it’s longer so it’s definitely better. Yeah… is any programmer using Boolean() function anyway in JavaScript (warning: not Boolean constructor - Boolean constructor is something you shouldn’t ever use as it’s always truthy).

define('TRAVIS_CI', (boolean) getenv('TRAVIS_PHP_VERSION'));

I’ve decided to look for explicit booleans. So, I made a search query and I’ve found answer on Stack Overflow.

$test_mode_mail = $string == 'true'? true: false;

I seriously don’t know what to think about this. It got 8 upvotes. It works - but it would be seriously stupid code in any language that isn’t PHP. I mean - $string == 'true' is boolean, so why use ternary operator for it? And WHY use ternary operator for it even if it wouldn’t be ternary operator. I’m not going to mention usage of ==, instead of ===, it doesn’t really matter in this case.

My solution would be like this:

define('TRAVIS_CI', isset($_ENV['TRAVIS_PHP_VERSION']));

But good design should avoid dynamic constants.

Anyway, Davey Shafik has changed !! into (bool)… hey, hasn’t David Soria Parra said him to use (boolean)… check. Those PHP aliases attack again. Why? Why converting to boolean could be done using two casts with only different name? Why converting to float could be done using three casts with only different name - (float), (double) and (real)?

But that isn’t real problem. The Travis CI change was merged. Now README page of PHP contains the “build status: failing”. I’ve decided to click that image - every single change had either “build status: failing” or “build status: unknown” (because of failure while doing git clone (possibly GitHub went down?)). PHP wastes time of Travis CI just to get “build status: failing” again.

After some time, David Soria Parra has decided to add e-mailing about failures to php-qa@lists.php.net.

What went wrong? Well…

notifications:
    email:
        recipients:
            - php-qa@lists.php.net
        on_success: change # [always|never|change] default: change
        on_failure: always # [always|never|change] default: always

But… PHP always fails - so PHP Quality Assurance would get lots of useless messages. Well, the solution for PHP developers wasn’t to fix PHP issues, definitely not - that would be too clever for programmers that work on PHP. The fix for developers was to disable e-mail notifications.

notifications:
    email:
        recipients:
            - php-qa@lists.php.net
        on_success: never # [always|never|change] default: change
        on_failure: never # [always|never|change] default: always

Now, I wonder why simply didn’t removed notifications: section. I mean, they gave e-mail and told Travis - whatever happens, don’t e-mail. I’m going to call this brillant.

After some time, David Soria Parra has noticed that he doesn’t have to give e-mail to PHP Quality Assurance. So, he removed it.

notifications:
    email: false

Now I wonder why he didn’t removed whole section. He doesn’t want any notifications, so why have this section? So now, Travis CI does nothing and everybody ignores it. I mean, it compiles, but…

It has 1123 compilation warnings. It has 106 test failures (with 43 expected test failures), one expected SIGSEGV (wait…) and 70.4% code coverage. And it’s not about unimplemented features - those are simply SKIPed. No, it isn’t better in PHP 5.4 - I mean, PHP 5.4 has 103 test failures (with 43 expected test failures).

PHP developers - please fix every bug before implementing new features. Having codebase that constantly contains lots of test failures isn’t really fun.

If you are PHP user, I would advice you trying some other language, such as Perl (believe me or not, Perl is way better than PHP) or Python. Those languages are consistent and more structured than PHP. Annoyed by security issues? Those languages make it easier to protect against them.

If you still want to use PHP, use OOP everywhere and never use global variables (with the exception for $_GET, $_POST and $_SERVER of course). PHP is simply unusable without OOP. But still, I would recommend using other langauges. Also, use template engine and prepared SQL queries.

id='20121029_update'>2012-10-29 update>

The older version of article was mentioning following syntax.

const TRAVIS_CI = isset($_ENV['TRAVIS_PHP_VERSION']);

Turns out that it doesn’t work. Why? Because constant value isn’t constant - it depends on environment variable existing. All you get is cryptic syntax error. PHP stupidly uses its grammar to annoy developer, even if this syntax would be allowed in other sane languages. For example, you cannot do func()(), just because grammar forbids that. I haven’t heard of other language with similar problems.

PHP Parse error:  syntax error, unexpected 'isset' (T_ISSET)

Instead, the solution should be more like this.

$travis_ci = isset($_ENV['TRAVIS_PHP_VERSION']);

It’s not constant. It shouldn’t be because it isn’t real constant.

Planet Perl Six

perl6.announce: Parrot 5.4.0 "Austin Parrot" Released! by Bruce Gray

Konrad Borowski: Perl 6 changes - 2013W20

Perl 6 specification> You cannot create a set or bag from list of pairs or hash using a constructor. The bag {a => 2} was ambiguous, because it could mean either bag containing 'a' with value 2, or bag containg {a => 2} with value 1. You’re expected to use .Bag or .Set instead, if you mean that.

STD.pm6> You cannot use set or bag to mean empty set or bag. If you really meant that, you are expected to use set() or bag() instead. id='new_features'>New features>

Niecza> Set, KeySet, Bag, and KeyBag have default method. Just like in Rakudo, except with Niecza.

Jonathan Worthington (6guts): Rakudo on JVM progress update, and some questions answered

The Rakudo Port

Interop

Performance?

From Here

Konrad Borowski: Perl 6 changes - 2013W19

Niecza Perl> Pair.perl can now deal with the cases when Pair is the key. Doing ((a => 'b') => 'c').perl used to output "a" => "b" => "c", what actually meant (a => (b => 'c')). .perl now outputs correct code in that case. Implemented Hash.ACCEPTS.

Carl Masak: After the #masakism workshop

Successes

Things learned

Message

Once more?

Konrad Borowski: Perl 6 changes - 2013W18

Gabor Szabo: To merge or not to merge?

Konrad Borowski: Perl 6 changes - 2013W17

Perl 6 specification> DateTime doesn’t accept Callable timezones anymore. 5692ced333

Rakudo Perl> Renamed IO to IO::Handle. IO itself is a role. 2dca6ed950 37c9956502 DateTime doesn’t accept Callable timezones anymore. 71ea142406 id='new_features'>New features>

Rakudo Perl> Implemented Cool.lines. 0d1d1194a4

Carl Masak: t2: Rectangle haikus

Carl Masak: The #masakism workshop

Konrad Borowski: Perl 6 changes - 2013W16

Perl 6 specification> Removed IO::Dir and IO::File classes. fb25fa475854 id='new_features'>New features>

Jonathan Worthington (6guts): NQP on JVM bootstrapped, soon will land in NQP master

Getting NQP on JVM Bootstrapped

Supporting invokedynamic

What next?

Moritz Lenz (Perl 6): The REPL trick

perl6.announce: Parrot 5.3.0 "W00tstock Parrot" Released! by Bruce Gray

Konrad Borowski: Perl 6 changes - 2013W15

Perl 6 specification> Parcels are allowed to have 0 elements.

Niecza Perl> Added special cases for big_pow when either of values is -1, 0, or 1.

Konrad Borowski: Perl 6 changes - 2013W14

Perl 6 specification> Renamed weekday-of-month to day-of-week-in-month.

Rakudo Perl> Fixed signature of Pod::Block.gist, so it works properly with multis. for loops are eager now. Added Buf.perl. Fixed Buf.gist, so it works properly with multis.

Moritz Lenz (Perl 6): Rakudo's Abstract Syntax Tree

Ops and Constants

Variables and Blocks

Polymorphism and QAST::Want

Konrad Borowski: Perl 6 changes - 2013W13

Perl 6 specification> Added many IO::Path methods: .contents, .is-absolute, .is-relative, .absolute, and .relative. .path is now NOP that returns itself. For its previous behavior, use .Str. Changed Numeric.conjugate to Numeric.conj.

Rakudo Perl> +(23..2300000000) is optimized. --help reports -I and -M options.

Tadeusz Sośnierz (tadzik): Polish Perl Workshop status update

perl6.announce: Parrot 5.2.0 "Stuffed Parrot" Released! by Bruce Gray

Konrad Borowski: Perl 6 changes - 2013W12

Perl 6 specification> Added X::Range::InvalidArg.

Rakudo Perl> Removed lots of pir:: usage. Multi dispatch is now implemented in NQP, not C. Released Rakudo 2013.03.

Perl 6 Maven: Reading from a file in Perl 6

Konrad Borowski: Perl 6 changes - 2013W11

Niecza Perl> Empty.gist returns nothing.

Perl 6 Maven: Getting started with Rakudo * (2013.01) on Windows

Konrad Borowski: Perl 6 changes - 2013W10

Perl 6 specification> Rat.perl returns decimal number when denominator is normalizable to a power of 10. Added FALLBACK methods. Included Failure in smartmatch passthroughs. Loosened relationship between Nil and (). Added List.permutations. rat64 type has unsigned denominator.

STD.pm6> Nil isn’t suggested for empty list in error messages anymore.

Niecza Perl> Rat.perl returns decimal number when denominator is normalizable to a power of 10. CommonEnum.gist returns (Type) instead of Type() for undefined. Restored infix:<cmp> sorting for different types. Failed matches return Nil.

Solomon Foster: Philosophical Issues with Rakudo’s .parse

Konrad Borowski: Perl 6 changes - 2013W09

Perl 6 specification> Inside of try is considered eager context, unless try is called in sink context, when it’s in sink context.

Konrad Borowski: I made a pointless webpage

rakudo.org: Rakudo Star 2013.02 released

Konrad Borowski: Perl 6 changes - 2013W08

Perl 6 specification> Failed matches result Nil, not (Match). id='new_features'>New features>

STD.pm6> Added a warning for smartmatch of (Match) or (Nil).

Niecza Perl> Type placeholders (like Int, which is null, except Int typed) now have their .gist method show their name within parens to suggest being undefined. For example, (Int), (Mu).

Carl Masak: t1: Tell knights from knaves

Konrad Borowski: friendly interactive shell

perl6.announce: Parrot 5.1.0 "Zombie Parrot" Released! by Bruce Gray

Jonathan Worthington (6guts): NQP on JVM gets Grammars, Multiple Dispatch

Carl Masak: Perl 6 is now half as old as Perl

Bottom-up vs top-down

Second system syndrome

Rubber, meet road

So, how're we doing?

What remains?

Conclusion

STD.pm6>

You cannot use `set` or `bag` to mean empty set or bag.

If you really meant that, you are expected to use `set()` or `bag()` instead.

id='new_features'>New features>

Niecza>

`Set`, `KeySet`, `Bag`, and `KeyBag` have `default` method.

Just like in Rakudo, except with Niecza.

Niecza Perl>

`Pair.perl` can now deal with the cases when `Pair` is the key.

Doing `((a => 'b') => 'c').perl` used to output `"a" => "b" => "c"`, what actually meant `(a => (b => 'c'))`. `.perl` now outputs correct code in that case.

Implemented Hash.ACCEPTS.

Perl 6 specification>

`DateTime` doesn’t accept `Callable` timezones anymore. 5692ced333

Rakudo Perl>

Renamed `IO` to `IO::Handle`. `IO` itself is a role. 2dca6ed950 37c9956502

`DateTime` doesn’t accept `Callable` timezones anymore. 71ea142406

id='new_features'>New features>

Rakudo Perl>

Implemented `Cool.lines`. 0d1d1194a4

Perl 6 specification>

Removed `IO::Dir` and `IO::File` classes. fb25fa475854

id='new_features'>New features>

Perl 6 specification>

`Parcel`s are allowed to have 0 elements.

Niecza Perl>

Added special cases for `big_pow` when either of values is `-1`, `0`, or `1`.

Perl 6 specification>

Renamed `weekday-of-month` to `day-of-week-in-month`.

Rakudo Perl>

Fixed signature of `Pod::Block.gist`, so it works properly with `multi`s.

`for` loops are eager now.

Added `Buf.perl`.

Fixed `Buf.gist`, so it works properly with `multi`s.

Perl 6 specification>

Added many `IO::Path` methods: `.contents`, `.is-absolute`, `.is-relative`, `.absolute`, and `.relative`.

`.path` is now NOP that returns itself. For its previous behavior, use `.Str`.

Changed `Numeric.conjugate` to `Numeric.conj`.

Rakudo Perl>

`+(23..2300000000)` is optimized.

`--help` reports `-I` and `-M` options.

Perl 6 specification>

Added `X::Range::InvalidArg`.

Rakudo Perl>

Removed lots of `pir::` usage.

Multi dispatch is now implemented in NQP, not C.

Released Rakudo 2013.03.

Niecza Perl>

`Empty.gist` returns nothing.

Perl 6 specification>

`Rat.perl` returns decimal number when denominator is normalizable to a power of 10.

Added FALLBACK methods.

Included Failure in smartmatch passthroughs.

Loosened relationship between `Nil` and `()`.

Added `List.permutations`.

`rat64` type has unsigned denominator.

STD.pm6>

`Nil` isn’t suggested for empty list in error messages anymore.

Niecza Perl>

`Rat.perl` returns decimal number when denominator is normalizable to a power of 10.

`CommonEnum.gist` returns `(Type)` instead of `Type()` for undefined.

Restored `infix:<cmp>` sorting for different types.

Failed matches return `Nil`.

Perl 6 specification>

Inside of `try` is considered eager context, unless `try` is called in sink context, when it’s in sink context.

Perl 6 specification>

Failed matches result `Nil`, not `(Match)`.

id='new_features'>New features>

STD.pm6>

Added a warning for smartmatch of `(Match)` or `(Nil)`.

Niecza Perl>

Type placeholders (like `Int`, which is null, except Int typed) now have their `.gist` method show their name within parens to suggest being undefined. For example, `(Int)`, `(Mu)`.

Rakudo Perl>

`$_` is writable in `-n`/`-p` code (one-liner).

`eqv` is faster for `Buf` objects.

Configure script fails on unknown command line options.

Rakudo Perl>

Constant folded `^..^` (infix) and `^` (prefix) operators.

Perl 6 specification>

Added `DateTime.delta` and `Date.delta` to the specification.

Rakudo Perl (in Rakudo 2013.01)>

`** 0` in regexpes works correctly matching nothing.

Rakudo Perl (post Rakudo 2013.01)>

Compiler hints invalid variable names. For example, typing `$variabel` when you already have `$variable` declared hints that variable.

Niecza Perl>

More meaningful error message is thrown when `printf` tries to consume more arguments than available.

Perl 6 specification>

`-->` (return type) acts more like a normal subroutine parameter.

Added `squish` method, working like `uniq` utility in UNIX - it only removes elements if it is duplicate of previous element.

Adjusted precedence of set operators.

STD.pm6>

Adjusted precedence of set operators.

Niecza Perl>

Adjusted precedence of set operators.

Rakudo Perl>

“SAFE” setting is now removed. You’re supposed to use “RESTRICTED” setting now.

id='new_features'>New features>

Niecza Perl>

`infix:<->(Instant, Instant)` now supported.

Bitwise operations support big integers.

Perl 6 specification>

`uniq` method/subroutine, existing for long time in implementations is now mentioned in specification (with `===` semantics).

Rakudo Perl>

Use of suffix regex modifiers (P5-ism) in `s///`, like `s////g` is now detected.

Rakudo Perl (in Rakudo 2012.12)>

When disambiguating hash/block, and `$_` variable is used, Rakudo assumes block. This could break your code if you were expecting a hash.