Test::Assertion by Elizabeth Mattijsen.a == b | c by Liminal Warmth.
.
Raku RSS Feeds
Roman Baumer (Freenode: rba #raku or ##raku-infra) / 2020-11-28T13:19:17Alexander Kiryuhin again did all the hard work to create a new Rakudo Compiler release: the Rakudo Compiler 2020.11 Release! With a bunch of new features, such as new coercion semantics, support for the [|| ] and {|| } postcircumfix operators, a new is test-assertion trait for better error reporting during testing. Plus some efficiency improvements and quite a number of bug fixes and improvements.
Sadly, shortly after the release it became clear that some typical workloads seem to be affected by a noticeable performance regression. This appears to be caused by the new coercion semantics inadvertently disabling some runtime optimizations. Fixing this has now become a blocker for the next release. It just goes to show that in Raku, it’s important to first make it work, and then make it work fast. And that a lot of users are already relying on those runtime optimizations.
Thibault Duponchelle has written an extensive history of the Raku Programming Language for a French Linux Magazine: Raku en 2020 (en Français, bien sûr). It is a trip down memory lane, but also gives an excellent technical overview of current Raku. Kudos to Thibault for the extensive research!
JJ Atria has published another nice blog post, this time about their experience porting a helper program to Raku: Porting from Go. It even comes with a new module: Timer::Stopwatch!
Vadim Belman has written an extensive blog post about how the new coercion semantics came to be, and what they bring to the average (module) developer. And how “I can probably do it in a week or two!” turned into many month’s of work (/r/rakulang comments).
It’s almost that time of the year again: Raku Advent Blog Post Time! Be sure to submit your ideas, or even better a PR with a completed Advent post to make sure there’s a slot available for you. Less than 10 left now!
Makoto Nozaki has published the TPF Meeting Minutes of the Board/Community meeting of last October.
Weekly Challenge #88 is available for your perusal. Andrew Shitov did a full review of the Raku solutions of Challenge #84.
NativeCall.push and unshift, made cas() about 15% faster and added some Windows-specific build instructions.IO::Socket::Net.new only accept typed family/type/protocol values.%*SUB-MAIN-OPTS<coerce-allomorphs-to> to have allomorph values automatically converted to non-allomorph values prior to dispatch to MAIN. They also added a deterministic method to Iterators indicating whether an iterator will always produce the same results given a set of input parameters. And fixed a few long-standing issues with REPL usage (and possibly replaced them by other issues).Pod::To::Text.This week’s new Pull Requests:
Please check them out and leave any comments that you may have!
<( )> token within comb not behave as expected? by chenyf.<!before> patterns by littlebenlittle.Test::Assertion by Elizabeth Mattijsen.a == b | c by Liminal Warmth..It has come to the attention of yours truly that many people have a collection of useful Raku code examples stored as gists. Not quite enough to turn into a distribution for the ecosystem, but handy to know about anyway. If new gists are available, they will be listed in this section! So here we go:
Being able to announce a new release is always cool. A nice set of blog posts this week, with an opportunity to brush up on your French 
And an impressive number of core improvements. Pretty sure there will be more of them in next weeks Rakudo Weekly News about the Raku Programming Language! And please: stay safe and stay healthy!
Recently I have read a lot of books on swimming — which, if you knew me, would seem unexpected. Having a fear of water after a near-drowning accident as a child, I never became a swimmer. Not even a so-so swimmer: I managed to learn what we in Norway call “Grandma swimming”, a sort of laborious and slow breast swimming with the head as high above water as humanly possible and the feet similarly low beneath.
But many years later, as an adult and a father, this slowly changed when my oldest son started attending swim practice. Even before taking up swimming as a sport, he had surpassed my abilities by a decent margin. After he became serious about training he almost instantly dwarfed me and my abilities.
As parents of swimmers know, being a swim parent involves lots of driving to-and-from and perhaps even more waiting. Sometimes I killed time waiting for him outside the pool area, looking in through the large glass windows that separated spectators —aka annoying parents — from swimmers. From a distance I was amazed by the progress he made month by month.
One summer day a year into his training I stood on a lake’s edge watching him swim happily towards the opposite side. When he passed the middle a couple of hundred feet out, I was struck by an uncomfortable thought: If anything happened to him now, I wouldn’t be able to help. And had I tried, I would probably need help myself.
In that very moment I decided to something about it. I immediately signed up for a beginner’s swim course for adults. But ten weeks and ten lessons in, hanging from the pool side panting uncontrollably, I was struck by a second thought: The progress I’d seen in the children was impossible to match for us, the slightly overweigh 40+ year old newbies. It would take time and patience to become even a so-so swimmer. And, as it turned out, it would take a lot of patience: A few years have passed and only recently have I started to feel that I master freestyle slightly — if swimming 50 meters freestyle without passing out constitutes mastering, that is. My technique is still laughable, breath continues to be an issue, and I haven’t begun to tumble turn or back stroke yet.
So now I know: This takes time.
But to boost my motivation I turned to books — like I do every time I start becoming interested in a new subject. These are not instructional or teach-yourself books, but inspirational books about the topic I’m interested in. In this case about swimmers that does unimaginable feats and/or about the history and cultural impact of swimming.
As they work as inspiration for me, maybe they will for you too. That’s why I give you quick reviews of five swimming related books I’ve read the last months. They are: Swimming to Antarctica by Lynne Cox, The art of resilience by Ross Edgley, Why we swim by Bonnie Tsui, Open Water by Mikael Rosén, and Grayson, also by Lynne Cox.
Lynne Cox: Swimming to Antarctica
This is the autobiography of the accomplished open water swimmer Lynne Cox. It starts in the seventies when Lynne’s family moves from the US east coast to California, so that the children can maximise their swim training. It’s here Lynne discovers that she’s a better long distance swimmer than a sprint swimmer and gradually switches to open water swimming.
Soon she participates in her first long distance swim —a 20 mile swim from Catalina island to mainland Los Angeles— and discovers that she has the potential for record-breaking pace. It seems like she’s a natural at the discipline. Later she will learn that her body is unique in preserving body heat in cold water conditions.
Next up are more feats, such as swimming the English channel. That one earns her an invitation to Cairo to swim the Nile, etc. While all this is happening, she starts to form an idea of becoming the first to swim from the US to the Soviet Union. Lynne sees this as a way to establish bonds and reduce tension between the people of the two countries. Alas, hardly anyone shares her enthusiasm, so large parts of the book covers the quest of getting the necessary permits to cross between two island on the Alaskan and Siberian side of the Bering strait.
That process took maybe ten years and is, to me, the book’s heart and soul. Swimming aside, this tenaciousness is a testament of her ability to persevere not only in open water but also the intricacy and bureaucracy that is international politics. As such I think Swimming through the Iron Curtain would be a more fitting title than Swimming to Antarctica. But the book is written chronologically and ends with a swim to the Antarctica, so I guess that’s why they chose the book’s title.
A weakness with the book is that it’s unusually light, almost coy, is when it comes to Lynne’s relation to other humans. Her parents, which must have been an important part of her support, is peculiarly described as almost faceless entities in her vicinity. As for romantic relations, she occasionally alludes rather vaguely to how she enjoys the company of a certain individual or how she admires the muscular body of a fellow swimmer, etc. But relations are never described deeper than that, and never more than with a few sentences. That means that this autobiography is unusually auto: Her book is a story about herself and her inner journey powered by external journeys — swims that most people can only dream of.
But no matter what the story is called or what weaknesses it may have, it’s a great read about an extraordinary human. That’s why I recommend this book.
Ross Edgley: The art of resilience
I don’t remember how I stumbled across Ross Edgeley and his “Great British Swim”, but I guess Google’s impenetrable algorithms had something to do with it. Regardless of how — when I did discover him (2018) he had just started his Red Bull sponsored swim around Great Britain, and posted weekly videos about his progress on YouTube. He synchronised his efforts to the tides for the duration of the swim: For 157 days he swam with the currents for six hours and rested the following six hours aboard his support boat. Non stop. For the entirety of the journey he never once set foot on land.
When he started the journey the farewell was rather low key, as the turnout consisted of family and friends. When he finished he’d become a household name and was welcomed by hundreds of other open water swimmer as well as large crowds on the beach. And that was well earned, if you ask me: 157 days — initially they thought they use half of that time — and 2884 kilometers (1792 miles) later, he (and his crew) had completed a feat that I think will stay uncontested for a long time.
In short, Ross’s story is an exciting one and he writes really well about it. That part of the book is impeccable. Strangely, the weakest parts are where Ross’s background as a sports scientist comes in. He’s eager to share theories about how to train, explain how endurance vs strength works, suggest workouts, etc. Every chapter ends with these science based musings. But they’re not integrated well into the storyline — yes, these too are filled with Ross’s enthusiasm, but all they do for me is punctuate and slow down an otherwise engaging story.
What I find peculiar, however, is that if you never watched the youtube videos, in the book it seems as if he got the idea and that everything fell into place by itself. In real life a Red Bull sponsorship was what made the swim possible. They kept him fed and enabled him to keep a boat and a four person crew with him at all times during the 157 days (but he’s gratuitous towards the boat’s captain and attributes much of the success to him).
It’s also interesting that this book is the opposite of Lynne Cox’s memoirs in the sense that what Ross is mostly concerned with is the external journey itself. There are some hints of musings about how the swim influenced his personal development, but they are few and far between.
In the grand scheme of things, though, these are small flaws. If you manage to fight through the sports science this is a great read!
Lynne Cox: Grayson
This book covers one very special day in Lynne Cox’s life — a day that weren’t covered in her autobiography. One early morning around daybreak, the seventeen year old Lynne is midway through a solitary open water swim practice. Suddenly she experiences unusual disturbances in the water, only to discover that it’s caused by a baby gray whale.
What seems to be a fun encounter quickly turns into a more serious matter: Communicating with an experienced elderly man on shore, she realises that the baby whale has been separated from his mother. If she swims ashore the infant will follow her, strand and die. The story details how Lynne slowly coaches the whale out to deeper water in the hope that they’ll by chance will find his mother.
Where Swimming to Antartica was a book as much about Lynne’s inner journeys as her outer, Grayson is even more of an inner journey. The book’s style reflects this. Grayson has a far more lyrical, introspective and even pensive form than her first book.
That’s not only positive. As mentioned it covers the events of this one morning only. The only perspective is Lynne’s told in the present tense. To stretch a small story about one morning from one person’s perspective to the necessary 150 pages, a lot of the text is inner monologue. In my opinion that slows the narrative down, and not in a good way. The inner monologues become fillers that doesn’t drive story.
What’s worse is that much of the inner monologue doesn’t seem entirely believable. The amount of depth and detail that Lynne allegedly remembers events and thoughts with, is more than anyone — with the possible expection of Marilu Henner — can recall some 30 odd years later. In addition many of the thoughts and reflections the 17 year old Lynne supposedly has, are astonishingly mature and filled with knowledge she possibly couldn’t have had at the moment . Scientific facts about gray whales, for instance. These are the thoughts and retrospections of a person in their late forties. There’s nothing wrong with thoughts and retrospections from late forty-somethings — after all I’m one myself. And had they been presented as such, as present day reflections of that extraordinary morning in her teens, this would probably not feel alien to the story at all. But the the choice to attribute the thoughts of a soon-to-be 50 year old to a 17-year old ends — to me —up as a significant stylistic crash.
With that in mind, I can’t help but think that if her editor had cut most of this, they’d end up with a tight and great story driven book for adolescents/young adults. As it is now, it’s not. But if you’re a less critical reader than me you’ll get a reasonably engaging book about the inner and outer journey of an almost superhuman swimmer. Should you only want to read only one book by Lynne Cox, however, Swimming to Antartica is the better choice.
Mikael Rosén: Open Water
Swedish author Mikael Rosén’s Open Water is not only about swimming itself, but also about swimming’s history, technique, science, cultural implications, racial issues, and more. Although you’d imagine that a mashup of all that would end up… well… mashy, the book is surprisingly clear and interesting despite juggling many sprawling subject. As such the book really delivers on its subtitle The History and Technique of Swimming.
Although this book talks about specific swimmers such as the pre-WW2 olympian Johnny Weissmuller, the first man to swim across the English channel, captain Matthew Webb, or modern athletes like Michael Phelphs, this is really not a book about individuals. These people are used to illustrate topics such as improvement in sports science (Weissmuller vs the Japanese swimmers that followed) or the history of open water swimming (captain Webb). Consistently interesting throughout, the most interesting part may be the second of the total eight chapter. That section explores prejudice — how female swimmers started to appear on the scene and suddenly break records previously held by men, or how a white, racist population’s negative reaction to black swimmers at public pools contributed to the establishment and strengthening of segregation laws in the southern states of the USA.
This tour de force of interesting and surprising facts reads a little like if Bill Bryson had written a book about swimming, though less humorous. But still, it’s almost on that level. Most books are not perfect, however, and this book is no exception: Written three years before Ross Edgeley’s The Art of Resilience, it shares the latter’s insistence of closing each chapter with a little sports science, training programs, suggestions of drills, etc. They don’t bother me as much in this book — as opposed to Ross’s — as this book is not a chronological story driven narrative. Therefore the training parts fits a little better into the whole. But the book wouldn’t suffer if they’d been edited out.
All in all that’s minor critisism, so I recommend this book wholeheartedly.
Bonnie Tsui: Why we swim
It’s not a coincidence that this book comes last. The reason is that this book is best described in the context of the previously mentioned books.
Why we swim is in a way an amalgamation of the science/history aspects of Mikael Rosén’s Open Water and the introspection of Lynne Cox’s books. But where the latter describes her personal growth in retrospect, Bonnie Tsui documents her quest for personal growth through swimming more or less as it happens — as a part of the process of writing the book itself it seems.
Bonnie Tsui kicks her book off with the story of Guðlaugur Friðþórsson, a fisherman that was the sole survivor after a fishing vessel sank in the frigid winter waters off Iceland. Together with two mates he started to swim towards land, but not long after he was the only one still swimming. Against all odds Friðþórsson survived a six hour swim in six degree celcius water.
For Tsui this becomes the entry point to the history of swimming. Her book is structured around five main topics, going from Survival, Well-Being, Community, Competition and ultimately to the more metaphysical and meditative subject of Flow. She takes on a tour of swimming history, starting in the stone age and the first documentation we have of humans swimming, ending with personal musings about not why we swim, but why she swims.
And this inclusion of a very visible I throughout the book — the chapter about Friðþórsson is not only about Friðþórsson but also about her meeting him and her participation in a swim honoring him—means that you can’t separate her personal journey from her exploration of the history, culture and science of swimming. Granted, Open Water is more hard core when it comes to facts, but the unique interspersion of the author personal story and the overarching topics of the book, makes this the most beautifully written of the five books I’ve mentioned here. Read it!
So… do you become a better swimmer by reading? Of course not. Only practice can improve swimming (although you may pick up some valuable hints to how you can improve through pure instructional books, such as Terry Laughlin’s Total Immersion).
But this being November 2020, the year of Covid-19, all swimming pools are closed and I’m unable to practice and improve for a while. This may be the case for you too. But while you wait for the pools to open again or the summer to heat up the sea to a more welcoming temperature, spending some time on one or more of these books wouldn’t be the worst thing to do.
Who knows? Maybe you’ll come back to the water more inspired than before.
Months of work by Vadim Belman, implementing the new type coercion protocol, was finally merged in time for the 2020.11 Rakudo compiler release. This makes coercion types such as Str() first class citizens. And allows custom classes to define their behaviour when used in coercions. For example, a user writes
sub foo(Frobnicator() $f) {...}
in their code (which specifies that the first positional parameter should be coerced to the Frobnicator class). The developer of the Frobnicator class can now actually provide code to handle that case:
class Frobnicator {
has $.value;
method COERCE($value) { # <-- new special method name
my $coerced = # something, depending on $value
Frobnicator.new(value => $coerced);
}
}
Previously, this kind of coercion was only possible for the core classes, in something like:
sub foo(Int() $f) {...}
So this is a big step forward in the malleability of, and the use of typing in, the Raku Programming Language!
It’s almost that time of the year again: Raku Advent Blog Post Time! Be sure to submit your ideas for an Advent post and make sure there’s a slot available for you.
Patrick Spek has joined the Raku Steering Council.
Looks like many people are hedging their blogging tuits for the Raku Advent Calendar. This week saw only two blog posts, one by Wenzel P.P. Peppmeyer (Parallel errors) and one by Alexey Melezhik (Ubiquitous Asynchronous Task Runner with Nice Frontend).
Weekly Challenge #87 is available for your perusal.
nqp build instructions a bit.42 does role a runtime error, as it would “taint” the value 42 for any use of the value 42. And they also made it possible to easily typecheck on typed arrays and hashes, as well as on object hashes and shaped arrays.This week’s new Pull Requests:
CStr REPRadd_native_assign_op to keywordswhen cases&make is called w/o a valid match in $/Attribute with Pod::To::Textspurtpush / unshiftPlease check them out and leave any comments that you may have!
zef by s-ro_mojosa.
𝒚.Again a week with many module updates, although the numbers have been skewed a bit by some module maintenance work by yours truly. Please check in again next week with more news about the Raku Programming Language!
The Merge of PR 3891 completes my work on new implementation of coercion in Rakudo. When I was starting with it my only intention was about to find a task to distract me from some problems I was having at the time (nothing serious yet bothersome and stressful). There was no concrete plans on my hands, so I picked one of the oldest tickets in Rakudo issue tracker. It was issue #1285 which looked quite promising as I already had some knowledge about parameterized roles internals. No doubts, I noticed Jonathan’s comment stating:
there’s no reason this can’t be made to work at some point in the future. It’s just not entirely trivial to get there.
Ok, I said to myself, he is a busy man for whom time is even pricier luxury than it is to me. I can probably do it in a week or two!
It’s always easy to guess what comes next: I was a way too naïve one; but this is for the better, as it turns out.
Anyway, back to the business. As the information about what’s been done by the merge is spread across commits, related problem-solving ticket, and the PR itself, I’m intended to gather and summarize here all information about the changes for further use by those who plan to formalize it in the documentation or use in their code.
The old coercion type object is a kind of ephemeral thing. You get one when declare something like Foo(Bar), but there
is no much use in it and it is barely available for user disposal. Basically, all support of coercions was focused on
handling parameters up to the point where Parameter class had special attributes for this; and parameter binding code
was actually implementing the functionality. A coercion type object existed and was backed by Metamodel::CoercionHOW,
but it was actually and mostly re-delegating to Mu.
The new coercion type objects are in a way resemble definites (Foo:D notation backed by
Metamodel::DefiniteHOW). First of
all, aside of being coercives they’re nominalizables (NB both terms are part of archetypes concept of
MOP). Roughly saying, for those who is
not familiar with this concept, a nominalizable type is one which wraps around another type and implements additional
properties. Consider, for example, subset Foo of Int or Int:D. For both of them .^nominalize method will return
Int. Moreover, the general plan is to have all nominalizables fully transparent when nested. In other words,
Foo:D.^nominalize will result in Int too even though it is a definite atop of a subset.
Same applies to the new coercions. Whichever one you use, would it be Int(Rat) or Int:D(Str) – both nominalize into
Int. In the second case it is done indirectly, via nominalization of the target type Int:D.
A coercion could also be of a generic archetype too. This happens when it is defined via a type capture:
role R[::T] {
method foo(T(Str) $v) {
say "V: ", $v.WHICH;
}
}
R[Int].new.foo("42"); # V: Int|42
my \how := R.^candidates[0].^lookup('foo').signature.params[1].type.HOW;
say how.^name; # Perl6::Metamodel::CoercionHOW
say how.archetypes.generic; # 1
Contrary to the old implementation, new coercions delegate to their nominal type:
class Foo {
method foo {
say "here we go!";
}
}
Foo().foo; # here we go!
Previously, the .foo line would result in No such method ‘foo’ for invocant of type ‘Foo(Any)’.
Now, lets try the following:
Foo().new; # You cannot create an instance of this type (Foo(Any))
The reason for this to fail would become clear if we add say self.^name to the method foo and try the previous
example again. We would see than that the method invocant is not Foo, as some may expect it, but it’s the coercion
Foo(Any) itself. So, what actually happens when we invoke new is that it tries to instantiate Foo(Any) and fails
the same way as it would do with Int:D.new or UInt.new.
At some point I was even considering making it possible to unwrap nominalizables and make it possible to create instances of their nominalization indirectly. But this feature has a great potential of confusing a Raku programmer, especially inexperienced one, and be helpful in concealing potential problems in code. So, the idea was dropped.
I’m thinking of adding a new smiley alongside with :U and :D – :C which would stand for coercion types. This
would allow declarations like method new(::?CLASS:C:) { ... }. But it’s not certain if there is really any value in
such addition aside of allowing a user to instantiate coercions manually. Besides and most likely this would also
require changes to multi-dispatch implementation.
Another outcome of coercions being a first-class type objects is that they currently support sensible smartmatching:
Int ~~ Int(Str) # True
Str ~~ Int(Str) # True
Num ~~ Int(Str) # False
Int(Str) ~~ Int # True
Int(Str) ~~ Str # False
There could be a confusion though when one tries Int ~~ Int:D() which results in True. To explain why it’s a correct
outcome, lets try to be more specific with Int ~~ Int:D(Str) example. This one results in False as expected. Now, if
we get back to the first one, we’d need to mentally unwrap Int:D() into it’s verbose form and note that it actually
stands for Int:D(Any). Apparently, Int ~~ Any and thus it can be accepted by the coercion.
I could’ve stopped there. But since I was on the track already, it was hard to get around two other problem-solving tickets: #22 and #137. To give a quick introspection into the tickets, I’d say that both are discussing different aspects of the following problem:
class Foo {...}
sub bar(Foo(Int) $v) {...}
The old coercions require a method named after the target type to be implemented by the constraint type. Unfortunately,
this makes the above coercion impossible because there is no way we can provide a method for each and every 3rd party
class wishing to be a coercion target. Think of all possible variations of Object class defined in public and private
modules!
Skipping all the discussions and intermediate variants, here is what I came up with eventually.
First of all, the coercion protocol is implemented by Metamodel::CoercionHOW::coerce() method. It means that the
protocol is now publicly available. say Str(Rat).^coerce(3.14).raku will now give you a string.
The protocol itself consist of the following steps:
First we look up a method named after the target type on the constraint type. This is the behavior we always had. The interesting part, by the way, is that this method supports compound class names with both the old and the new implementation:
class A::B { }
class Foo { method A::B { A::B.new; } }
sub foo(A::B(Foo) $v) { say $v.raku }
foo(Foo.new); # A::B.new
If there is no method found at the first step then the protocol looks for COERCE method on the target type which
signature would accept the value we’re trying to coerce.
class IntContainer {
has Int:D $.value is required;
proto method COERCE($) {*}
multi method COERCE(Int:D $i) { self.new: :value($i) }
multi method COERCE(Str:D $s) { self.new: :value($s.Int) }
}
sub foo(IntContainer(Any) $v) { say $v }
foo(42); # IntContainer.new(value => 42)
foo("13"); # IntContainer.new(value => 13)
foo(pi); # Impossible coercion from 'Num' into 'IntContainer': no acceptable coercion method found
Note that we don’t have a candidate for Num, thus the last error.
If no acceptable COERCE found then the protocol falls back to the target type nominalization new method and tries
to find a candidate in the same way as it was trying for COERCE.
class IntContainer {
has Int:D $.value is required;
multi method new(Numeric:D \v) { self.new: :value(v.Int) }
proto method COERCE($) {*}
multi method COERCE(Int:D $i) { self.new: :value($i) }
multi method COERCE(Str:D $s) { self.new: :value($s.Int) }
}
sub foo(IntContainer(Any) $v) { say $v }
foo(42); # IntContainer.new(value => 42)
foo("13"); # IntContainer.new(value => 13)
foo(pi); # IntContainer.new(value => 3)
Let me elaborate on some interesting and important details of the protocol.
The above code snippets provide the correct ways of implementing COERCE method. The key point I’m referring to in here
is the use of self to instantiate the resulting object. Let me show you why this is important:
class Foo { method COERCE($) { Foo.new } }
class Bar is Foo { }
sub bar(Bar() $v) { say $v.raku }
bar("oops"); # Impossible coercion from 'Str' into 'Bar': method COERCE returned an instance of Foo
Hopefully, the error message makes the situation clear: the coercion expected a Bar (or, for that matter, a subclass
of Bar) instance but got Foo instead. Apparently, Foo !~~ Bar and this is clearly an error. It is rather easy to
overlook such situation while developing a class because most of the time what we test would be something like:
class Foo { method COERCE($) { Foo.new } }
sub bar(Foo() $v) { say $v.raku }
bar("oops"); # Foo.new
It is also important to remember that no matter which way we’ve got our coerced value, it is always a subject for
typechecking. Therefore, something like method Bool { 1 } is an error too.
Coercion errors are checked by ^coerce method. If anything goes wrong it throws X::Coerce::Impossible exception. If in the
above throwing example we replace bar("oops") with Bar().^coerce("oops"), the outcome would be the same.
In a rare situation, when the class X::Coerce::Impossible is not available for the metaclass code, instead of throwing
the exception object it would just die with nqp::die opcode. But this case is unlikely to be of any interest for
anyone but core developers.
Coercion protocol doesn’t imply a limitation on use of either methods or submethods for its implementation. But depending on which one is used its behavior may have different outcomes. Consider this example:
class C1 {
method COERCE(\v) {
say "C1::COERCE";
self.new
}
}
class C2 is C1 {
submethod COERCE(\v) {
say "C2::COERCE";
C2.new;
}
}
class C3 is C2 { }
sub c1(C1()) { }
sub c2(C2()) { }
sub c3(C3()) { }
c1(1); # C1::COERCE
c2(2); # C2::COERCE
c3(3); # C1::COERCE
The difference is apparent for anybody with clear understanding of submethods. For anyone alse my advise would be to
stick with [multi-]methods. Yet, note the use of C2.new in the submethod. It is safe to be done that way for the exact
reason of the submethod not ever being called for any other target but C2. Anyway, I would insist on using self even
within a submethod because, if at some point you decide to convert it into a method, this simple approach may spare you
minutes or even who knows how many hours of locating the error.
Briefly:
my Int(Str) $v;
$v = "42";
say $v.WHICH; # Int|42
Same applies to attributes:
class Foo {
has Int(Str) $.value;
}
This is done via Scalar functionality and consequently available anywhere a containerization is used:
my Str() %h;
%h<foo> = pi;
say %h<foo>.WHICH; # Str|3.141592653589793
One I found while writing this post:
sub factory(::T) {
my sub (T $v) { say $v.WHICH }
}
factory(Int(Str))("42"); # Str|42
The other version of this example works as expected:
sub factory(::T) {
my sub (T(Str) $v) { say $v.WHICH }
}
factory(Int)("42"); # Int|42
Another problem is caused by the order of things in parameter binding code. I expected this bug to be there, but did not feel ready to mangle with it. So, consider this intentional but temporary situation:
class Foo {
method COERCE(Any:U \v) { self.new }
}
sub foo(Foo:D() $v) { say $v.WHICH }
foo(Int); # Parameter '$v' of routine 'foo' must be an object instance of type 'Foo:D(Any)', not a type object of type 'Int'. Did you forget a '.new'?
Perhaps, to fix it would be sufficient to move the coercion block higher in src/Perl6/bootstrap.c/BOOTSTRAP.nqp,
bind_one_param subroutine? Not sure. Maybe some of you, reading this, can have a look and find the right solution. I
mean, one way or another, but the definedness check must be done after coercion, not before. Unfortunately, I feel
like requiring some rest of this subject and taking care of some other tasks I was postponing all these weeks.
This works now:
enum E <a b c>;
sub foo(E:D(Int) $v) {
say "Got: ", $v, " of ", $v.^name;
}
foo(1);
This was a side effect of getting Int() == 0 to work as it is specced (a long boring story, not worth telling). Now
for code like Int == 0 instead of
Invocant of method 'Bridge' must be an object instance of type 'Int',
not a type object of type 'Int'. Did you forget a '.new'?
one would see a more user-friendly variant:
Use of uninitialized value of type Int in numeric context
28 commits and 31 changed file. Hours and hours of compilations and spectesting - I praise my recently bought HP Z840 workstation with 28 cores/56 hyperthreads, it spared me a lot of time. It was the distraction I needed. Perhaps even too much of it. 🙂 It’s now time to move on to something different. More articles in Advanced Raku For Beginners, perhaps? Will see…
I wanted to know how long I have to wait for ffmpeg to turn * into AV1. To do so I have to capture STDERR because that’s where ffmpeg outputs the status information while encoding.
my $ffmpeg-out = Channel.new;
my $ffmpeg-err = Channel.new;
react {
whenever $ffmpeg-err -> [ $stream-num, $_ ] {
.say if $verbose;
# boring parsing code goes here
}
}
px{'nice', '/usr/local/bin/ffmpeg', '-i', $input, '-c:v', 'libsvtav1', '-q', '60', '-preset', '5', '-y', $output} |» $ffmpeg-out :stderr($ffmpeg-err);The solution is quite simple. I use a Channel to transport lines of text from STDERR to a react block that does the parsing. A nice solution until ffmpeg decides to use STDERR to actually communicate error messages. Those will be gobbled up by the parsing code. As a result the exception that is thrown out of Shell::Piping has no error message to display, beside that a non-zero error code has been returned.
There are quite a few programs that output status to STDERR while operating, so a general approach would be nice. Where messages land is controlled by the adverb :stderr. To capture text to be displayed in the exception is done with :stderr(Capture). So my first impulse was to use a mixin like :stderr($ffmpeg-err but Capture). This would allow me to solve this problem. More complex solutions would be desirable. A user of my module might want to parse STDERR and write everything to a file. Both are currently possible but not at the same time. Providing a list of stream targets would imply order. I’m not sure if I can guarantee that. In the case of multiple channels it would not make sense for sure. In Raku we do have a parallel data type called Junction. In its current form it is not possible to pull a Junction apart. A pull request by lizmat to change that did not find a large fellowship. Luckily she also wrote a module called eigenstates.
use eigenstates;
sub prefix:<\foo> (Str $_, Junction :$err = all Whatever) {
say $_;
say eigenstates $err;
}
\foo "ello" :err(1|2);
# OUTPUT: ello
(1 2)This allows me to do :stderr($ffmpeg-err & Capture) and many more complex operations. For now I don’t see a use in the type of Junction beside all. Since there are still 95 years left in Rakus life cycle, I’m quite sure something will come along.
In almost a weekly event, Jonathan Worthington has just announced the 0.8.4 release of Cro, the Raku Programming Language’s set of libraries for building reactive distributed systems. What yours truly actually finds amazing, is that apart from the people of Edument, this release has contributions by 9 Raku community members, indicating a growing user base.
Arne Sommer is celebrating their 100th Raku blog post with an extensive multi-part blog post covering 100 different sequences: from Arithmetic, Geometric, Primes and Fibonacci, Palindrome and every Tom, Dick and Harry (/r/rakulang comments). A recommended read!
The Grants Committee is again accepting Raku grant proposals! So if you have an idea that could benefit from financial support, be sure to submit your proposal!
It’s almost that time of the year again: Raku Advent Blog Post Time! Be sure to submit your ideas for an Advent post and make sure there’s a slot available for you.
This week still saw some nice blog posts again:
if statement, and assignment by Mark Carter (/r/rakulang comments).Weekly Challenge #86 is available for your perusal. And a full review of Raku solutions of Weekly Challenge #83 was made by Andrew Shitov.
$*TZ and DateTime objects of long-running processes, and fixed an issue in matching Junctions.This week’s new Pull Requests:
MVM_box_{int,num,str,uint} and MVM_unbox_str duplicate functionality in reprconv.cfastcreate style logic to MVM_repr_box_int%*SUB-MAIN-OPTS<coerce-allomorphs-to>Hash::Object/Hash::Typed rolesPlease check them out and leave any comments that you may have!
Junction makes Match immutable by grobber.CArray[ of-struct] from Raku to C? by rir.Inline::Perl5 RE as constraint by Paul Procacci.In a week that appeared dominated by counting, it is good to see at least 18 community members being able to either create a new Raku module, or improve existing Raku modules. Be sure to stay healthy and stay safe! There will be more news about the Raku Programming Language next week!
Jonathan Worthington has just announced the 2020.10 release of Comma, the IDE of choice for the Raku Programming Language. And this release comes with an impressive amount of new features and even direct support for Red, the Raku ORM!
In related news, Alexandr Zahatski has announced a new version of the dedicated Pod6 desktop editor called Podlite. This release supports import from markdown and export to HTML. It’s great to see Pod6 becoming easier and easier to work with for desktop publishing. But also for documenting Raku code as well, of course!
This week saw some nice blog posts again:
Please keep those blog posts coming! They’re good exercise for your Raku Advent Calendar post now less than a month away!
The Raku Steering Council announces the ejection of Aleks-Daniel Jakimenko-Aleksejev from the Raku core team as well as from the Raku Steering Council. In a final word, Aleks-Daniel said:
Having someone in my immediate family pass away recently, I’ve been thinking lately how some things are considered inherently bad or negative, and how we always wish it was possible to turn things around. However, a death can relieve pain and stop suffering, a breakup can create space for new amazing relationships, and a person leaving a team allows both parties to move into differently great directions. I already have two very interesting projects coming up, so in some sense I’m grateful for being able to dedicate myself to them. I hope you are happy for my ejection too, as weird as it sounds
Weekly Challenge #85 is available for your perusal.
NativeCall in MoarVM and made some changes that allow better C compiler optimizations.IO::Path.symlink that allows creation of relative symlinks.@a[;] slicing capabilities in 6.e.PREVIEW. They also introduced a new Routine is test-assertion trait that allows for better error reporting from custom subroutines that perform tests.This week’s new Pull Requests:
Array.DELETE-POS return Nil on non-existent elementsRAKU_TEST_DIAG_LINE_ON_FAILdocs/running.podPlease check them out and leave any comments that you may have!
A busy week again. Glad to see so many new modules and cool core improvements such as Unicode 13.1 support and the new is test-assertion trait that will help module and core developers alike! And a new release of Comma to boot! In the midst of everything else that’s going on in the world. For which yours truly would like you to know: stay healthy and stay safe! Check in next week for more Raku news!
Welcome back to another week of the Weekly Challenge, and today I’ll briefly describe my solutions to the Week 85.
The solutions of this week actually make me think that Raku changes my definition of what is a straightforward solution. All those tiny Raku bits such as any or X or ^$N are awesome even in a not fully-optimised program.
You are given an array of real numbers greater than zero. Write a script to find if there exists a triplet (a, b, c) such that 1 < a + b + c < 2. Print 1 if you succeed otherwise 0.
So, the task is to find such three numbers from the given list so that they add up to a number between 1 and 2. Sounds as a good task for using the combinations routine.
my @r = 1.2, 0.4, 0.1, 2.5; say +so 1 < any(@r.combinations(3)>>.sum) < 2;
As you see, the solution is extremely short. It creates the possible 3-element combinations from the input data and computes the sums of each. Then, we make a test 1 < any(...) < 2 using the any-junction, which allows us to avoid any explicit loops in the program. Finally, the +so construct converts the result to first to a Boolean value (so), and then to either 1 or 0 as the task requires.
You are given a positive integer $N. Write a script to find if it can be expressed as a ^ b where a > 0 and b > 1. Print 1 if you succeed otherwise 0.
Well, some mathematics can be applied before going to the code, but let’s skip this useful step 
my $N = @*ARGS[0] // 8;
for 1..^$N X ^$N -> ($a, $b) {
say "$N = $a^$b" if $N == $a ** $b;
}
The program is quite wordy and self-explanatory, but I must mention that parentheses around loop variables are important here. Without them, you will get two pairs such as (7 2) (7 3) on each iteration instead of just two integers.
The program prints all the possible solutions, and prints nothing if it could not find it.
* * *
→ GitHub repository
→ Navigation to the Raku challenges post series
Twelve years¹ ago Larry planned the obsolescence of one of my modules. His cunning plan was executed by lizmat a fortnight ago. If you are building Rakudo from source you take another shortcut now.
use v6.e.PREVIEW;
my %detectives;
my @a = <UK London Bakerstreet>;
%detectives{||@a} = "Holms";
say %detectives{||<UK London Bakerstreet>};
dd %detectives;
# OUTPUT: Holms
Hash %detectives = {:UK(${:London(${:Bakerstreet("Holms")})})}Slippy semi lists can be quite useful while working with parsed JSON. There are plenty of hashes of hashes of hashes.
Now I got a peculiar problem. One of my modules becomes redundant if — and only if — a Raku version is requested that is younger then v6.d. The module itself can demand v6.d even when used by a program or module that itself demands v6.e. At least it is easy to test for that condition.
use v6.e.PREVIEW;
say $*RAKU.version ~~ v6.e+;
# OUTPUT: TrueWe can use that to trigger a warning in a module as soon as v6.e becomes the default.
use v6;
CHECK if $*RAKU.version ~~ v6.e+ { warn "Unattended items will be destroy without warning! (This is a warning.)" };This warning will be displayed once per precompilation so it is easy to miss. Also it will show even thought the program running the use statement might demand an older version of Raku. If only there where a subroutine that is called every time we import a module in the context of the caller!
use v6.d;
sub EXPORT {
if $*RAKU.version ~~ v6.e+ { warn "Unattended items will be destroy without warning! (This is a warning.)" };
%()
}Now use can detect time travel by means of looking up the dynamic variable $*RAKU.
use v6.e.PREVIEW;
use warner;
# OUTPUT: Unattended items will be destroy without warning! (This is a warning.)
in sub EXPORT at /home/dex/projects/raku/obsolescence/lib/warner.rakumod (warner) line 6The ability of Raku to fix past mistakes was planned right from the beginning.² I feel ready for the next 100 years.
¹) That’s a lie. The truth was hidden by the move from SVN to git. Please forgive my urge to tell a good story.
²) That’s not a lie.
Quite a number of releases this week(end): Alexander Kiryuhin released the Rakudo 2020.10 compiler release, Claudio Ramirez immediately provided many Linux package versions of that release, and JJ Merelo updated the standard Raku Alpine Docker image, as well as the special Raku Alpine Docker Image for testing modules by module developers.
And Patrick Spek also immediately sprang into action by creating a Rakudo Star 2020.10 release. Sadly without a confirmed Windows build: volunteers welcome. And kudos to all who made these releases possible!
Weekly Challenge #84 is available for your perusal. Andrew Shitov did a full review of Challenge #81 as well as a full review of Challenge #82, both with video treatments!
alloc / free calls are properly wrapped in MVM_* macros.MVM_recalloc function only attempts to zero out memory when it should.nqp:: ops that weren’t properly documented before..gitattributes file to help core developer with git diff.IO::Path.copy will not clobber the file (as copying from a directory is not supported).
and 𝐑𝐚𝐤𝐮 welcome message at the request of the TPF lawyer in aid of proper trademark registration. They also made creating a Uni and several methods on native arrays much faster. And introduced support for the prefix || operator inside postcircumfixes (allowing the %h{||@indices} syntax on multi-dimensional hashes and arrays in v6.e).rakuw.exe “window” application.Compress::Bzip2 module from being installed.multi submethods being visible for subclasses.This week’s new Pull Requests:
bind_attribute can write beyond malloc‘d buffer." in C code (NQP)nqp::indexingoptimized as Str.no-strandsMu ACCEPTS candidate for Mu:DCachedIterator roleSeq.slice(indices)" in C code (Rakudo)Please check them out and leave any comments that you may have!
if statement to a variable by anquegi.Supply method act vs tap by jakar.FatRat by Ralph Mellor.A week without any real blog posts, but with some new Raku modules! And new releases and some interesting new Raku Programming Language features. As always: please don’t forget to stay healthy and to stay safe. And make sure to check in next week for more Raku news!
While playing with .WHY I stepped onto an ENODOC. Method refers to Signature. Both fail to mention the default signature of a method. The documentation of automatic signatures fails to mention that %_ is always added to methods. Asking greppable6 for mentioning of it revealed plenty of redundant uses of *%_ in methods.
my method m1(:$named) {}
my method m2(:$named, *%_) {}
say &m1.signature;
say &m2.signature;
# OUTPUT: (Mu: :$named, *%_)
(Mu: :$named, *%_)The reason why I spotted this is .WHY. In Raku we can add comments that are attached to the language objects that follow them.
class Commented {
#| this is an attribute
has $.a-documented-attribute;
#| this is a method
method works {
say ‚working as intended‘;
}
}
say $c.^can('works')[0].WHY;
say $c.^attributes[0].WHY;
# OUTPUT: this is a method
this is an attributeThe docs fail to state how to access WHY-nodes for attributes and methods at runtime. We have to go through the MOP here because methods and attributes can be added to a type object at runtime. At least for methods I found a nicer way to handle this.
method works {
return &?ROUTINE.WHY if %_;
say ‚working as intended‘;
}
put $c.works(:why);Oddly, this leads to the awkward situation that I add an undocumented argument to a method that returns the documentation for that method. To be explicit we got the trait is implementation-detail. Being forced to use a multi makes things more cumbersome though.
#| No comment! We are sneaky!
multi method sneaky { say ‚working as intended‘ }
multi method sneaky('why') is implementation-detail {
return self.^lookup('sneaky').candidates[0].WHY;
}
say $c.sneaky('why');That leaves the question why I need .WHY on attributes. I do agree with the assessment that JSON is lacking in general. My biggest objection is the lack of comments. For config files being able to tell a co-worker to keep his dirty fingers of a certain line can save lives. Debian is making good use of comments in such files too. It can save a lot of time if you don’t have to read a man page just to change one value. With inline comments on attributes I could express both the structure of data stored in a config file and instructions how to use the config file without another HEREDOC that can get out if sync with the actual code. I really like the structure of NestedText and am playing with adding type information. In NestedText leafs default to Str so that can be omitted. In Raku we get string representations for types for free, we can just add them as text.
I can define the structure of my config file as follows.
use Data::TypedNestedText;
class Phone {
has $.mobile;
has $.home;
has $.office;
}
class Contact {
has $.name;
#| this can be multiline
has $.address;
has $.phone;
has @.additional-roles;
}For the output I wrote a naive deparser that can produce a NestedText with type annotations.
president[Contact]:
# this can be multiline
address:
> 138 Almond Street
> Topika, Kansas 20697
name: Katheryn McDaniel
phone[Phone]:
home: 1-210-555-8470
mobile: 1-210-555-5297
treasurer[Contact]:
# this can be multiline
address:
> 3636 Buffalo Ave
> Topika, Kansas 20692
name: Fumiko Purvis
phone[Phone]:
office: 1-268-555-0280
additional-roles:
- hug task force
- Christmas party teamThis is much easier to read and edit then JSON. Writing the parse might not. I will report back shortly if it turns out to be easier then expected.
I’d like to thank everyone who voted for me in the recent Raku Steering Council elections. By this point, I’ve been working on the language for well over a decade, first to help turn a language design I found fascinating into a working implementation, and since the Christmas release to make that implementation more robust and performant. Overall, it’s been as fun as it has been challenging – in a large part because I’ve found myself sharing the journey with a lot of really great people. I’ve also tried to do my bit to keep the community around the language kind and considerate. Receiving a vote from around 90% of those who participated in the Steering Council elections was humbling.
Alas, I’ve today submitted my resignation to the Steering Council, on personal health grounds. For the same reason, I’ll be taking a step back from Raku core development (Raku, MoarVM, language design, etc.) Please don’t worry too much; I’ll almost certainly be fine. It may be I’m ready to continue working on Raku things in a month or two. It may also be longer. Either way, I think Raku will be better off with a fully sized Steering Council in place, and I’ll be better off without the anxiety that I’m holding a role that I’m not in a place to fulfill.
In the machine world, we have Int and Num. A High Level Language such as Raku is a abstraction to make it easier for humans. Humans expect decimal math 0.1 + 0.2 = 0.3 to work.
Neither Int nor Num can do this! Huh? How can that be?
Well Int is base-2 and decimals are base-10. And Num (IEEE 754 Floating Point) is inherently imprecise.
Luckily we have Rat. Raku automatically represent decimals and other rational numbers as a ratio of two integers. Taking into account all the various data shuffling any program creates, the performance is practically as fast as Int. Precision, accuracy and performance are maintained for the entire realm of practical algebraic calculations.
And when we get to the limit of Rat (at around 3.614007241618348e-20), Raku automatically converts to a Num. After this, precision cannot be recovered – but accuracy and dynamic range are maintained. Modern Floating Point Units (FPUs) make Num nearly as fast as Int so performance is kept fast.
Here, we have smoothly transitioned to the world of physical measurements and irrational numbers such as square root and logarithms. It’s a one way trip. (Hint: If you want a Num in the first place, just use ‘e’ in the literal.)
Good call, Raku!
Footnote: FatRat and BigInt are not important for most use cases. They incur a performance penalty for the average program and rightly belong in library extensions, not the core language.
Today I released META6::bin with the following command.
meta6 --releaseSince I forgot to commit the changes to README.md I had to release again with.
meta6 --release --version=+As a result there are now some friendly tarballs at https://github.com/gfldex/raku-meta6-bin/releases/. To get there I had to force travis to install Test::META from HEAD. If you wish to release on github with ease, you may have to run the following commands.
zef install --/test https://github.com/jonathanstowe/Test-META.git
zef install META6::binIt should work with already existing META6.json files out of the box but likely need the setup instructions at the bottom of the README.
As soon as RabidGravy found the time to make a proper release of Test::META (shouldn’t take more then 10 seconds) I will set the proper dependencies.
Some oddities in the response of the github API (<tarball_url> doesn’t point at the tarball) gave me a good reason to have another look at Codeberg. A lot was added in the last two years. The project is run by a non-profit from Berlin and as such not based in a country that likes to punish people because they don’t like a government. If we want to provide a programming language for a global community we can’t rely on Trumpland anymore. The whole software stack is FOSS and the API looks really nice. I didn’t miss any features that are on github but I didn’t really look for projects that are handled by an organisation.
I aim to add support for Codeberg to META6::bin and shall report here before Christmas.
In this issue, we’ll take a look at an interesting pattern of passing multiple arguments to a function and use it to implement a recursive sum algorithm.
Consider the following example:
sub f($a, *@b) {
say "Head: $a";
say "Tail: @b[]";
}
f(42, 43, 44);
The signature of the function has two elements: a scalar $a and a slurpy array @b. When you call f with three parameters, the first one goes to $a, and the rest land in @b.
The program prints:
$ raku sub.raku Head: 42 Tail: 43 44
Let us apply the pattern from the example to a program that recursively computes the sum of the elements of an array of numbers:
multi sub rsum($head) {
$head
}
multi sub rsum($head, *@rest) {
$head + rsum(|@rest)
}
say rsum(10, 20, 30, 40, 50); # 150
There are no assignments here, so this program is a perfect example of programming in a functional style in Raku.
We have two versions of the rsum functions here. One needs a single value, and the second one accepts additional (but required) array. While there are more than one element in the data list, the second version of the function is run. On the last iteration, a single elements remains, and thus the simpler version of the function is executed. The sum of a single element is the element itself. In the output, you get the desired sum.
* * *
Find the code of this issue on GitHub and browse for more Raku Pearls.
While watching RaycatWhoDat liking Raku, I realised that a List of Matches is not a MatchList.
say 'a1'.match(/ \d /).replace-with('#');
say 'a1b2'.match(/ \d /, :g).replace-with('#');
# OUTPUT: a#
# No such method 'replace-with' for invocant of type 'List'
in block <unit> at /home/dex/tmp/tmp-2.raku line 8The 2nd call of replace-with will fail because .match with :g returns a List of Match. Of course we can play the easy game and just use subst which does the right thing when called with :g. This wouldn’t make a good blog post though.
To make replace-with work with a List we can use a where-clause. Any Match will have a copy of the original Str but not to the original Regex so we actually have to build a list of Str of everything that was not matched. This can be done by using the indices stored in .from and .to.
multi sub replace-with(\l where (.all ~~ Match), \r --> Str) {
my $orig := l.head.orig;
my @unmatched;
@unmatched.push: $orig.substr(0, l.head.from);
for ^(l.elems - 1) -> $idx {
@unmatched.push: $orig.substr(l[$idx].to, l[$idx+1].from - l[$idx].to);
}
@unmatched.push: $orig.substr(l.tail.to);
(@unmatched Z (|(r xx l.elems), |'')).flat.join;
}
say 'a1vvvv2dd3e'.match(/ \d /, :g).&replace-with('#');
# OUTPUT: a#vvvv#dd#eIf the original string does not end with a match, the list of matches will be one short to be just zipped in. That’s why I have to extend the list of replacements by an empty string before feeding it to Z.
So if subst is doing it right why bother with .replace-with? Because sometimes we have to use $/.
if 'a1bb2ccc3e' ~~ m:g/ \d / {
say $/.&replace-with('#');
}Often we could change the code but when a routine from a module returns Match or a list thereof, we are out of luck. For completeness we need a few more multies.
multi sub replace-with(Match \m, \r --> Str) {
m.replace-with(r);
}
multi sub replace-with(Match \m, &r --> Str) {
m.replace-with(r(m));
}
multi sub replace-with(\l where (.all ~~ Match), &r) {
my $orig := l.head.orig;
my @unmatched;
@unmatched.push: $orig.substr(0, l.head.from);
for ^(l.elems - 1) -> $idx {
@unmatched.push: $orig.substr(l[$idx].to, l[$idx+1].from - l[$idx].to);
}
@unmatched.push: $orig.substr(l.tail.to);
(@unmatched Z (|l.map(&r), |'')).flat.join;
}Even if the problem is solvable it still bugs me. We have :g in many places in Raku to provide quite a lot of DWIM. In some places that concept breaks and almost all of them have to do with lists. Often ».comes to the rescue. When we actually have to work on the list and not the individual elements, even that doesn’t work. The methods on strings just work because in this case we deliberately avoid breaking the list of characters apart.
If you follow this blog you know that I’m leaning strongly towards operators. Sadly ., .? and ». are not real infixes which we can overload. Nor can we declare infixes that start with a . or we could introduce an operator that turns it’s LHS to a list and then does a dispatch to multis that can handle lists of a certain type.
Without that we need to change .match to return a subclass of List that got the method .replace-with. We could stick it into Cool but that is a crowded place already.
We don’t really have a nice way to augment the return values of builtin methods. So this will have to be fixed in CORE.
I have been unsatisfied with the installation process for MoarPerf for a little while now. You have to either compile the javascript (and css) yourself using npm (or equivalent) which takes a little while, or you have to rely on me releasing an "everything built for you" version to my github repo.
The last few days I've repeatedly bonked my metaphorical pickaxe against the stone wall that is unpleasantly long build times and an endless stream of small mistakes in order to bring you MoarPerf in an AppImage.
AppImage is a format for linux programs that allows programs to be distributed as a single executable file without a separate install process.
The AppImage for MoarPerf includes a full Rakudo along with the dependencies of MoarPerf, the built javascript, and the Raku code. This way you don't even have to have a working Rakudo installation on the machine you want to use to analyze the profiler results. Yours truly tends to put changes in MoarVM or nqp or Rakudo that sometimes prevent things from working fine, and resetting the three repos back to a clean state and rebuilding can be a bit of an annoyance.
With the MoarPerf AppImage I don't have to worry about this at all any more! That's pretty nice.
With an AppImage created for MoarPerf it was not too much work to make an AppImage for Rakudo without a built-in application.
The next step is, of course, to pack everything up nicely to create a one-or-two-click solution to build AppImages for any script that you may be interested in running.
There has also already been a module that creates a windows installer for a Raku program by installing a custom MoarVM/nqp/Rakudo into a pre-determined path (a limitation from back when Rakudo wasn't relocatable yet), and maybe I should offer an installer like this for windows users, too? The AppImage works much like this, too, except it already makes use of the work that made Rakudo relocatable, so it doesn't need to run in a pre-defined path.
If you want to give building AppImages a try as well, feel free to steal everything from the rakudo-appimage repository, and have a look at the .travis.yml and the appimage folder in the moarperf repo!
In any case, I would love to hear from people, whether the AppImages for Rakudo and MoarPerf work on their machines, and what modules/apps they would like to have in an AppImage. Feel free to message me on twitter, write to the Raku users mailing list, or find me on freenode as timotimo.
Thanks for reading, stay safe, and see y'all later!
In this issue we’ll briefly talk about practical cases where junctions are really handy — both in syntax and in making the program simpler.
Here is an interesting example of how you can use the any-junction in a single operation where you would traditionally should have a loop.
my $a = 40;
my @b = 1..5;
say 'Found the answer' if $a + any(@b) == 42;
Examine the addition $a + any(@b). We compare its result with a single number 42 as if any(@b) were a single integer.
anyA classical example of using any-junctions is something like this:
say 'There are negative ints' if 1 | -2 | 3 < 0;
An alternative syntax — use a function:
say 'There are negative ints' if any(1, -2, 3) < 0;
Another option — a method:
say 'There are negative ints' if (1, -2, 3).any < 0;
allThe following examples are similar to the examples from the previous section but employ the all-junction instead:
say 'Only positive ints' if 1 & 2 & 3 > 0; say 'Only positive ints' if all(1, 2, 3) > 0; say 'Only positive ints' if (1, 2, 3).all > 0;
MAIN Imagine a program that takes integer numbers from command line. You need to distinguish between the cases when the integers are either all positive or all negative.
One of the possible ways to check the input data is to use multi MAIN functions and dispatch their calls using the where clauses.
multi sub MAIN(*@a where @a.elems > 0 && .all ~~ Int && .all > 0) {
say 'All positive integers'
}
multi sub MAIN(*@a where @a.elems > 0 && all(@a) < 0 && all(@a) ~~ Int) {
say 'All negative integers'
}
I am using two different forms of creating the all-junction: .all and all(@a). You are free to build a WhateverCode block too.
There where clauses demand that all of the input data (i.e., the elements in @a) are integers (.all ~~ Int) and either positive (.all > 0) or negative (all(@a) < 0).
Run the program with different input data:
$ raku all-1.raku 10 20 30 All positive integers $ raku all-1.raku -10 -20 -30 All negative integers $ raku all-1.raku -10 -20 30 Usage: all-1.raku [<a> ...] all-1.raku [<a> ...]
The previous example can be re-written in a different style that introduces new data types to the program:
subset PositiveInt of Int where * > 0;
subset PositiveIntArray of Array where .all ~~ PositiveInt && .elems > 0;
subset NegativeInt of Int where * < 0;
subset NegativeIntArray of Array where .all ~~ NegativeInt && .elems > 0;
multi sub MAIN(*@a where * ~~ PositiveIntArray) {
say 'All positive integers'
}
multi sub MAIN(*@a where * ~~ NegativeIntArray) {
say 'All negative integers'
}
The subsets define the rules, and you can use them directly in the where clauses in other functions.
* * *
Find the code of this issue on GitHub and browse for more Raku Pearls.
In this issue, we’ll talk about the built-in wrap routine and its possible applications.
wrapThe wrap routine — as you may guess from its name — wraps another function. Let us go straight to the example.
sub greet($name) {
say "Hello, $name!";
}
&greet.wrap({
callwith($^arg.uc)
});
greet('Venusians'); # Hello, VENUSIANS!
First, we’ve got a regular function that takes a string and prints the message with it. In the end, the function is called as usual.
In-between, we see that the function is wrapped into another function, which is actually called when you call greet instead of it. In this example, the wrapping function calls the original function with the modified argument: callwith($^arg.uc). Here, $^arg is a placeholder variable that the wrapper gets when the original function is called in the last line of the program.
unwrapLet us extend the previous program and remove the wrapper after some time. To do that, you need to save the wrapper object in a variable, which you can later pass to the unwrap routine.
sub greet($name) {
say "Hello, $name!";
}
my $x = &greet.wrap({
callwith($^arg.uc)
});
greet('Venusians'); # Hello, VENUSIANS!
&greet.unwrap($x);
greet('Venusians'); # Hello, Venusians!
One of the useful examples of wrappers can be measuring time of execution of a slow function. Here is how you can do it:
sub slow {
sleep(1);
}
sub timer {
my $t0 = now;
callsame;
say now - $t0;
}
&slow.wrap(&timer);
slow();
This time, the wrapper uses callsame to call the original function with the same arguments. In this example, I am using a bit different syntax of calling the wrap method. This option is more suitable for bigger wrapping functions.
Another example — a logger that you can place around a function that visits the sites, for example.
use HTTP::UserAgent;
sub browse($url) {
my $ua = HTTP::UserAgent.new;
my $response = $ua.get($url);
say $response.status-line;
}
&browse.wrap(-> $url {
note "{DateTime.now} $url";
callsame;
});
browse('https://google.com/');
browse('https://raku.org/');
Again, just for the sake of diversity, I used a pointy block here. You do not need to change the browser function to start logging. Just create a wrapper which does the job:
$ raku wrap-log.raku > /dev/null 2020-09-17T23:04:32.383453+02:00 https://google.com/ 2020-09-17T23:04:32.807365+02:00 https://raku.org/
And that’s it for now!
* * *
Find the code of this issue on GitHub and browse for more Raku Pearls.
In this issue, we’ll discuss three elements in the Raku programming language that have the -rw suffix.
substr-rwWith a regular substr routine (used as either a method or a function), you get a copy of the substring, which is an immutable value. If you want to modify the original string, you can use substr-rw.
Consider the following two examples of using it:
my $string = 'Hello, World!'; $string.substr-rw(7, 5) = 'Mundo'; my $greeting := $string.substr-rw(0, 5); $greeting = '¡Hola'; say $string; # ¡Hola, Mundo!
In the first case, we assign the new value to the result of the method call. In the second case, we bind it to a scalar. The original string is modified after both assignments. Notice that the length of the replacement string may differ from the length of the replaced part.
return-rwYou can use return-rw in places where you traditionally use return. But with the longer variant, you actually return the object rather than the value.
sub my-value {
state $value = 3.14;
return-rw $value;
}
my-value++;
say my-value; # 4.14
The my-value++ construct modifies the state variable inside the function, so you increment it and with the next function call you get the same container with a modified value.
take-rwA similar pair exists for take. Use it together with gather, in which case you return the container and you have full access to it, not only to its value.
Examine the following program, where the gather block takes $x and $y.
my $x = 42;
my $y = 2020;
my $data := gather {
take $x;
take-rw $y;
};
say $data; # (42 2020), is a Seq
# $data[0]++; # Error
$data[1]++;
say "$x, $y"; # 42, 2021
(Notice that the result is bound to a scalar.)
You cannot modify the value gathered by take $x, but you can do so with what was returned by take-rw $y. For the second variable, the program prints its updates value.
* * *
Find the code of this issue on GitHub and browse for more Raku Pearls.
After month and a half full of many events, I finally got time to complete one more article for the Advanced Raku For Beginners series.
Frankly, I’m not happy about it. It feels to me that my not the perfect English is gotten even worse; not all topics and quirks are covered. But at least I’m getting back in these waters. So, just let it be. I’m warming up. And hope to advance into another subject soon.
Just a reminder to anybody passing by this blog that the election to Raku Steering Council is going on now and will be taking place until Sep 20. More details can be found in the voting form and the original announcement.
I have cast my ballot.
A little preface with an off-topic first. In the process of writing this post I was struck by the worst sysadmin’s nightmare: loss of servers followed by a bad backup. Until the very last moment I have had well-grounded fears of not finishing the post whatsoever. Luckily, I made a truce with life to get temporary respite. A conclusion? Don’t use bareos with ESXi. Or, probably, just don’t use bareos…
While picking up a RFC for my previous advent post I was totally focused on language-objects section. It took me a few passes to find the right one to cover. But in the meantime I realized that a very important topic is actually missing from the list. “Impossible!” – I said to myself and went onto another hunt later. Yet, neither search for “abstract class”, nor for “role” didn’t come up with any result. I was about to give up and make the conclusion that the idea came to life later, when the synopses were written or around so.
But, wait, what interface is mentioned as a topic of a OO-related RFC? Oh, that interface! As the request body states it:
Add a mechanism for declaring class interfaces with a further method for declaring that a class implements said interface.
At this point I realized once again that it is now a full 20 years behind us. That the text is from the times when many considered Java as the only right OO implementation! And indeed, by reading further we find the following statement, likely to be affected by some popular views of the time:
It’s now a compile time error if an interface file tries to do anything other than pre declare methods.
Reminds of something, isn’t it? And then, at the end of the RFC, we find another one:
Java is one language that springs to mind that uses interface polymorphism. Don’t let this put you off – if we must steal something from Java let’s steal something good.
Good? Good?!! Oh, my… Java’s attempt to solve problems of C++ multiple inheritance approach by simply denying it altogether is what drove me away from the language from the very beginning. I was fed up with Pascal controlling my writing style as far back as in early 90s!
Luckily, those involved in early Perl6 design must have shared my view to the problem (besides, Java itself has changed a lot since). So, we have roles now. What they have in common with abstract classes and the modern interfaces is that a role can define an interface to communicate with a class, and provide implementation of some role-specific behavior too. It can also do a little more than only that!
What makes roles different is the way a role is used in Raku OO model. A class doesn’t implement a role; nor it inherits from it as it would with abstract classes. Instead it does the role; or the other word I love to use for this: it consumes a role. Technically it means that roles are mixed into classes. The process can be figuratively described as if the compiler takes all methods and attributes contained by role’s type object and re-plants then onto the class. Something like:
role Foo {
has $.foo = 42;
method bar {
say "hello!"
}
}
class Bar does Foo { }
my $obj = Bar.new;
say $obj.foo; # 42
$obj.bar; # hello!How is it different from inheritance? Let’s change the class Bar a little:
class Baz {
method bar {
say "hello from Baz!"
}
}
class Bar does Foo is Baz {
method bar {
say "hello from Bar!";
nextsame
}
}
Bar.new.bar; # hello from Bar!
# hello from Baz!nextsame in this case re-dispatches a method call to the next method of the same name in the inheritance hierarchy. Simply put, it passes control over to the method Baz::bar, as one can see from the output we’ve received. And Foo::bar? It’s not there. When the compiler mixes the role into Bar it finds that the class does have a method named bar already. Thus the one from Foo is ignored. Since nextsame only considers classes in the inheritance hierarchy, Foo::bar is not invoked.
With another trick the difference from interface consumption can also be made clear:
class Bar {
method bar {
say "hello from Bar!"
}
}
my $obj = Bar.new;
$obj.bar; # hello from Bar!
$obj does Foo;
$obj.bar; # hello!In this example the role is mixed into an existing object, thanks to the dynamic nature of Raku which makes this possible. When a role is applied this way its content is enforced over the class content, similarly to a virus injecting its genetic material into a cell effectively overriding internal processes. This is why the second call to bar is dispatched to the Foo::bar method and Bar::bar is nowhere to be found on $obj this time.
To have this subject fully covered, let me show you some funny code example. The operator but used in it behaves like does except it doesn’t modify its LHS object; instead but creates and returns a new one:
my $s1 = "not empty means true";
my $s2 = $s1 but role { method Bool { False } };
say $s1 ?? "true" !! "false";
say $s2 ?? "true" !! "false";This snippet I’m leaving for you to try on your own because it’s time for my post to move onto another topic: role parameterization.
Consider the example:
role R[Str:D $desc] {
has Str:D $.description = $desc;
}
class Foo does R["some info"] { }
say Foo.new.description; # some infoOr more practical one:
role R[::T] {
has T $.val is rw;
}
class ContInt does R[Int] { }
ContInt.new.val = "oops!"; # "Type check failed..." exception is thrownThe latter example utilizes so called type capture where T is a generic type, the concept many of you are likely to know from other languages, which turns into a concrete type only when the role gets consumed and supplied with a parameter, as in class ContInt declaration.
The final iteration for parametrics I’m going to present today would be this more extensive example:
role Vect[::TX] {
has TX $.x;
method distance(Vect $v) { ($v.x - $.x).abs }
}
role Vect[::TX, ::TY] {
has TX $.x;
has TY $.y;
method distance(Vect $v) {
(($v.x - $.x)² + ($v.y - $.y)²).sqrt
}
}
class Foo1 does Vect[Rat] { }
class Foo2 does Vect[Int, Int] { }
my $foo1 = Foo1.new(:x(10.0));
my $foo2 = Foo2.new(:x(10), :y(5));
say $foo1; # Foo1.new(x => 10.0)
say $foo2; # Foo2.new(x => 10, y => 5)
say $foo2.distance(Foo2.new(:x(11), :y(4))); # 1.4142135623730951Hopefully, the code explains itself. Most certainly it nicely visualizes the long way made by the language designers since the initial RFC was made.
At the end I’d like to share a few interesting facts about Raku roles and their implementation by Rakudo.
use v6.e.PREVIEW; # 6.e is not released yet
role R { submethod TWEAK { say "R" } }
class Foo { submethod TWEAK { say "Foo" } }
class Bar is Foo does R { submethod TWEAK { say "Bar" } }
Bar.new; # Foo
# R
# Barrole R { say "Role" }
class Foo { say "Foo" }
# FooThen modify class Foo so that it consumes R:
class Foo does R { say "Foo" }
# Role
# FooThe difference in the output is explained by the fact that role body gets invoked when the role itself is mixed into a class. Try adding one more class consuming R alongside with Foo and see how the output changes. To make the distinction between class and role bodies even more clear, make your new class inherit from Foo. Even though is and does look alike they act very much different. 3. Square brackets in role declaration enclose a signature. As a matter of fact, it is the signature of role body subroutine! This makes a few very useful tricks possible:
# Limit role parameters to concrete numeric objects.
role R[Numeric:D ::T $default] {
has T $.value = $default;
}
class Foo[42.13] { };
say Foo.new.x; # 42.13Or even:
# Same as above but only allow specific values.
role R[Numeric:D ::T $default where * > 10] {
has T $.value = $default;
} Moreover, in case when few different parametric candidates are declared for a role, choosing the right one is a task of the same kind as choosing the right routine of a few multi candidates and based on matching signatures to the parameters passed. 4. Rakudo implements a role using four different role types! Let me demonstrate one aspect of this with the following snippet based on the example for the previous fact:
for Foo.^roles -> \consumed {
say R === consumed
}=== is a strict object identity operator. In our case we can consider it as a strict type equivalence operator which tells us if two types are actually exactly the same one.
And as I hope to have this subject covered later in a more extensive article, at this point I would make it a classical abrupt open ending by providing just the output of the above snippet as a hint:
False
Originally Submitted by Simon Cozens, RFC 28 on August 4, 2020, this RFC asked the community to make sure that whatever updates were made, that Perl 6 was still definitely recognizable as Perl. After 20 years of design, proofs-of-concept, implementations, two released language versions, we’ve ended up with something that is definitely Perlish, even if we’re no longer a Perl.
At the time the RFCs were submitted, the thought was that this language would be the next Perl in line, Perl 6. As time went on before an official language release, Perl 5 development picked up again, and that team & community wanted to continue on its own path. A few months ago, Perl 6 officially changed its name to Raku – not to get away from our Perl legacy, but to free the Perl 5 community to continue on their path as well. It was a difficult path to get to Raku, but we are happy with the language we’re shipping, even if we do miss having the Perl name on the tin.
Let’s dig into some of the specifics Simon mentions in his RFC.
We’ve got a golden opportunity here to turn Perl into whatever on earth we like. Let’s not take it.
This was a fine line that we ended up crossing, even before the rename. Specific design decisions were changed, we started with a fresh implementation (more than once if you count Pugs & Parrot & Niecza …). We are Perlish, inspired by Perl, but Raku is definitely different.
Nobody wins if we bend the Perl language out of all recognition, because it won’t be Perl any more.
I argue that eventually, everyone won – we got a new and improved Perl 5 (and soon, a 7), and we got a brand new language in Raku. The path wasn’t clear 20 years ago, but we ended up in a good place.
Some things just don’t need heavy object orientation.
Raku’s OO is everywhere: but it isn’t required. While you can treat everything as an object:
| 3.sqrt.say; |
You can still use the familiar Perlish forms for most features. say sqrt 3;
Even native scalars (which don’t have the overhead of objects) let you treat them as OO if you want.
| my uint32 $x = 32; | |
| say $x; | |
| $x.^name.say; |
Even though $x here doesn’t start out as an object, by calling a meta-method on it, the compiler cheats on our behalf and outputs Int here, the closest class to our native int.
But we avoid going the extent of Java; for example, we don’t have to define a class with a main method in order to execute a program.
Strong typing does not equal legitimacy.
Similar to the OO approach, we don’t require typing, but allow you to gradually add it. You can start with an untyped scalar variable, but as you further develop your code, you can add a type to that declared variable, and to parameters to subs & methods. The types can be single classes, subsets, Junctions, where clauses with complicated logic: you can use as much or as little typing as you want. Raku’s multi routines (subs or methods with the same name but different arguments) give you a way to split up your code based on types that is then optimized by the compiler. But you can use as little or as much of it as you want.
Just because Perl has a map operator, this doesn’t make it a functional programming language.
I think Raku stayed true to this point – while there are functional elements, the polyglot approach (supporting multiple different paradigms) means that any one of them, including functional, doesn’t take over the language. But you can declare routines pure, allowing the compiler to constant fold calls to that routine when the args are known at compile time.
Perl is really hard for a machine to parse. … It’s meant to be easy for humans to understand.
Development of Raku definitely embraced this thought – “torture the implementators on behalf of the users”. This is one of the reasons it took us a while to get to here. But on that journey, we designed and developed new language parsing tools that we not only use to build and run Raku, but we expose to our users as well, allowing them to implement their own languages and “Slangs” on top of our compiler.
Finally, now that the Perl team is proposing a version jump to 7, I suspect the Perl community will raise similar concerns to those raised by Simon. Raku and Perl 7 have taken two different paths, but both will be recognizable to the Perl 5 RFC contributors from 20 years ago.
Yet another nice goodie from Damian, truly what you might expect from the interlocutor and explicator!
The fat comma operator, =>, was originally used to separate values – with a twist. It behave just like , operator did, but modified parsing to stringify left operand.
It saved you some quoting for strings and so this code for hash initialization:
| my %h = ( | |
| 'a', 1, | |
| 'b', 2, | |
| ); |
could be written as:
| my %h = ( | |
| a => 1, | |
| b => 2, | |
| ); |
Here, bare a and b are parsed correctly, without a need to quote them into strings. However, the usual hash assignment semantics is still the same: pairs of values are processed one by one, and given that => is just a “left-side stringifying” comma operator, interestingly enough the code above is equivalent to this piece:
| my %h = ( a => 1 => b => 2 => ); |
The proposal suggested changing the meaning of this “special” operator to become a constructor of a new data type, Pair.
A Pair is constructed from a key and a value:
| my @pairs = a => 42, 1 => 2; | |
| say @pairs[0]; # a => 42 | |
| say @pairs[1]; # 1 => 2; | |
| say @pairs[1].key.^name; # Int, not a Str |
The @pairs list contains just 2 values here, not 4, one is conveniently stringified for us and the second just uses bare Int literal as a key.
It turns out, introducing Pair is not only a convenient data type to operate on, but this change offers new opportunities for… subroutines.
Raku has first class support of signatures, both for the sake of the “first travel class” pun here and for the matter of it, yes, actually having Signature, Parameter and Capture as first-class objects, which allows for surprising solutions. It is not a surprise it supports named parameters with plenty of syntax for it. And Pair class has blended in quite naturally.
If a Pair is passed to a subroutine with a named parameter where keys match, it works just so, otherwise you have a “full” Pair, and if you want to insist, a bit of syntax can help you here:
| sub foo($pos, :$named) { | |
| say "$pos.gist(), $named.gist()"; | |
| } | |
| foo(42); # 42, (Any) | |
| try foo(named => 42); # Oops, no positionals were passed! | |
| foo((named => 42)); # named => 42, (Any) | |
| foo((named => 42), named => 42); # named => 42, 42 |
As we can see, designing a language is interesting: a change made in one part can have consequences in some other part, which might seem quite unrelated, and you better hope your choices will work out well when connected together. Thanks to Damian and all the people who worked on Raku design, for putting in an amazing amount of efforts into it!
And last, but not the least: what happened with the => train we saw? Well, now it does what you mean if you mean what it does:
| my %a = a => 1 => b => 2; | |
| say %a.raku; # {:a(1 => :b(2))} |
And yes, this is a key a pointing to a value of Pair of 1 pointing to a value of Pair of b pointing to value of 2, so at least the direction is nice this time. Good luck and keep your directions!
Proposed on 7 September 2000, frozen on 20 September 2000, depends on RFC 159: True Polymorphic Objects proposed on 25 August 2000, frozen on 16 September 2000, also by Nathan Wiger and already blogged about earlier.
tie anyway?RFC 200 was about extending the tie functionality as offered by Perl.
This functionality in Perl allows one to inject program logic into the system’s handling of scalars, arrays and hashes, among other things. This is done by assigning the name of a package to a data-structure such as an array (aka tying). That package is then expected to provide a number of subroutines (e.g. FETCH and STORE) that will be called by the system to achieve certain effects on the given data-structure.
As such, it is used by some of Perl’s core modules, such as threads, and many modules on CPAN, such as Tie::File. The tie functionality of Perl still suffers from the problems mentioned in the RFC.
In Raku, everything is an object, or can be considered to be an object. Everything the system needs to do with an object, is done through its methods. In that sense, you could say that everything in Raku is a tied object. Fortunately, Rakudo (the most advanced implementation of the Raku Programming Language) can recognize when certain methods on an object are in fact the ones supplied by the system, and actually create short-cuts at compile time (e.g. when assigning to a variable that has a standard container: it won’t actually call a STORE method, but uses an internal subroutine to achieve the desired effect).
But apart from that, Rakudo has the capability of identifying hot code paths during execution of a program, and optimize these in real time.
Jonathan Worthington gave two very nice presentations about this process: How does deoptimization help us go faster from 2017, and a Performance Update from 2019.
Because everything in Raku is an object and access occurs through the methods of the classes of these objects, this allows the compiler and the runtime to have a much better grasp of what is actually going on in a program. Which in turn gives better optimization capabilities, even optimizing down to machine language level at some point.
And because everything is “tied” in Raku (looking at it using Perl-filtered glasses), injecting program logic into the system’s handling of arrays and hashes can be as simple as subclassing the system’s class and providing a special version of one of the standard methods as used by the system. Suppose you want to see in your program when an element is fetched from an array, one need only add a custom AT-POS method:
class VerboseFetcher is Array { # subclass core's Array class
method AT-POS($pos) { # method for fetching an element
say "fetching #$pos"; # tell the world
nextsame # provide standard functionality
}
}
my @a is VerboseFetcher = 1,2,3; # mark as special and initialize
say @a[1]; # fetching #12
The Raku documentation contains an overview of which methods need to be supplied to emulate an Array and to emulate a Hash. By the way, the whole lemma about accessing data structure elements by index or key is recommended reading for someone wanting to grok those aspects of the internals of Raku.
In a blog post about RFC 168 about making things less special, it was already mentioned that really nothing is special in Raku. And that (almost) all aspects of the language can by altered inside a lexical scope. So what the above example did to the Array class, can be done to any of Raku’s core classes, or any other classes that have been installed from the ecosystem, or that you have written yourself.
But it can be overwhelming to have to supply all of the logic needed to fully emulate an array or a hash. Especially when you first try to do this. Therefore the ecosystem actually has two modules with roles that help you with that:
Both modules only require you to implement 5 methods in a class that does these roles to get the full functionality of an array or a hash, completely customized to your liking.
In fact, the flexibility of the approach of Raku towards customizability of the language, actually allowed the implementation of Perl’s tie built-in function in Raku. So if you’re porting code from Perl to Raku, and the code in question uses tie, you can use this module as a quick intermediate solution.
Let’s look at the problems that were mentioned with tie in RFC 200:
Raku is completely extensible and pluggable in (almost) all aspects of its implementation. There is no limitation to which classes one can and one cannot extend.
All interfaces use methods in Raku, since everything is an object or can be considered as one. Use of classes and methods should be clear to any programmer using Raku.
In Raku, it is all the same speed during execution. And every customization profits from the same optimization features like every other piece of code in Raku. And will be, in the end, optimized down to machine code when possible.
In Raku, operators are multi-dispatch subroutines that allow additional candidates for custom classes to be added.
Typeglobs don’t exist in Raku. All interfacing in Raku is done by supplying additional methods (or subroutines in case of operators). No duplication of effort is needed, so no such problem.
One may argue that the kludgey syntax of Perl has been replaced by another kludgey syntax in Raku. That is probably in the eye of the beholder. Fact is that the syntax in Raku for injecting program logic, is not different from any other subclassing or role mixins one would otherwise do in Raku.
Nothing from RFC 159 actually was implemented in the way it was originally suggested. However, solutions to the problems mentioned have all been implemented in Raku.
Proposed on 25 August 2000, frozen on 16 September 2000
RFC159 introduces the concept of true polymorphic object.
Objects that can morph into numbers, strings, booleans and much more on-demand. As such, objects can be freely passed around and manipulated without having to care what they contain (or even that they’re objects).
When one looks at how 42, "foo", now work in Raku nowadays, one can only see that that vision has pretty much been implemented. Because most of the time, one doesn’t really care about the fact that 42 is really an Int object, "foo" is really a Str object and that now represents a new Instant object every time it is called. The only thing one cares about, is that they can be used in expressions:
say "foo" ~ "bar"; # foobar
say 42 + 666; # 708
say now - INIT now; # 0.0005243RFC159 lists a number of method names to be used to indicate how an object should behave under certain circumstances, with a fallback provided by the system if the class of the object does not provide that method. In most cases these methods did not make it into Raku, but some of them did with a different name:
| Name in RFC | Name in Raku | When |
|---|---|---|
| STRING | Str | Called in a string context |
| NUMBER | Numeric | Called in a numeric context |
| BOOLEAN | Bool | Called in a boolean context |
And some of them even retained their name:
| Name in RFC | When |
|---|---|
| BUILD | Called in object blessing |
| STORE | Called in an lvalue = context |
| FETCH | Called in an rvalue = context |
| DESTROY | Called in object destruction |
but with sometimes subtly different semantics from the RFC.
In the end, only a limited set of special methods was decided on for Raku. All of the other methods in RFC159 have been implemented by polymorphic operators that coerce when needed. For instance the proposed PLUS method has been implemented as an infix + operator that has a “default” candidate that coerces its operands to a number.
So, effectively, if you have an object of class Foo and you want that to act as a number, one only needs to add a Numeric method to that class. An expression such as:
my $foo = Foo.new;
say $foo + 42;is effectively executing:
say infix:<+>( $foo, 42 );and the infix:<+> candidate that takes Any objects, does:
return infix:<+>( $foo.Numeric, 42.Numeric );And if such a class Foo does not provide a Numeric method, then it will throw an exception.
In Raku, object destruction is non-deterministic. If an object is no longer in use, it will probably get garbage collected. The probable part is because Raku does not know a global destruction phase, unlike Perl. So when a program is done, it just does an exit (although that logic does honour any END blocks).
An object is marked “ready for removal” when it can no longer be “reached”. It then has its DESTROY method called when the garbage collection logic kicks in. Which can be any amount of time after it became unreachable.
If you need deterministic calling of the DESTROY method, you can use a LEAVE phaser. Or if that doesn’t allow you to scratch your itch, you can possibly use the FINALIZER module.
Conceptually, you can think of a container in Raku as an object with STORE and FETCH methods. Whenever you set a value in a container, it conceptually calls the STORE method. And whenever the value inside the container is needed, it conceptually calls the FETCH method. In pseudo-code:
my $foo = 42; # Scalar.new(:name<$foo>).STORE(42)But what if you want to control access to a scalar value, similar to Perl’s tie? Well, in Raku you can, with a special type of container class called Proxy. An example of its usage:
sub proxier($value? is copy) {
return-rw Proxy.new(
FETCH => method { $value },
STORE => method ($new) {
say "storing";
$value = $new
}
)
}
my $a := proxier(42);
say $a; # 42
$a = 666; # storing
say $a; # 666Subroutines return their result values de-containerized by default. There are basically two ways of making sure the actual container is returned: using return-rw (like in this example), or by marking the subroutine with the is rw trait.
Since FETCH only makes sense on scalar values, there is no support for FETCH on compound values, such as hashes and arrays, in Raku. I guess one could consider calling FETCH in such a case to be the Zen slice, but it was decided that that would just return the compound value itself.
The STORE method on compound values however, allows for some interesting functionality. The STORE method is called whenever there is an initialization of the entire compound value. For instance:
@a = 1,2,3;basically executes:
@a := @a.STORE( (1,2,3) );But what if you don’t have an initialized @a yet? Then the STORE method is supposed to actually create a new object and initialize this with the given values. And the STORE method can tell, because then it also receives a INITIALIZE named argument with a True value. So when you write this:
my @b = 1,2,3;what basically gets executed is:
@b := Array.new.STORE( (1,2,3), :INITIALIZE );Now, if you realize that:
my @b;is actually short for:
my @b is Array;it’s only a small step to realize that you can create your own class with customized array logic, that can replace the standard Array logic with your own. Observe:
class Foo {
has @!array;
method STORE(@!array) {
say "STORED @!array[]";
self
}
}
my @b is Foo = 1,2,3; # STORED 1 2 3However, when you actually start using such an array, you are confronted with some weird results:
say @b[0]; # Foo.new
say @b[1]; # Index out of range. Is: 1, should be in 0..0Without getting into the reasons for these results, it should be clear that to completely mimic an Array, a lot more is needed. Fortunately, there are ecosystem modules available to help you with that: Array::Agnostic for arrays, and Hash::Agnostic for hashes.
The BUILD method also subtly changed its semantics. In Raku, method BUILD will be called as an object method and receive all of the parameters given to .new, after which it is fully responsible for initializing object attributes. This becomes more visible when you use the internal helper module BUILDPLAN. This module shows the actions that will be performed on an object of a class when built with the default .new method:
class Bar {
has $.score = 42;
}
use BUILDPLAN Bar;
# class Bar BUILDPLAN:
# 0: nqp::getattr(obj,Foo,'$!score') = :$score if possible
# 1: nqp::getattr(obj,Foo,'$!score') = 42 if not setThis is internals speak for: – assign the value of the optional named argument score to the $!score attribute – assign the value 42 to the $!score attribute if it was not set already
Now, if we add a BUILD method to the class, the buildplan changes:
class Bar {
has $.score = 42;
method BUILD() { }
}
use BUILDPLAN Bar;
# class Bar BUILDPLAN:
# 0: call obj.BUILD
# 1: nqp::getattr(obj,Foo,'$!score') = 42 if not setNote that there is no automatic attempt to take the value of the named argument score anymore. Which means that you need to do a lot of work in your custom BUILD method if you have many named arguments, and only one of them needs special handling. That’s why the TWEAK method was added:
class Bar {
has $.score = 42;
method TWEAK() { }
}
use BUILDPLAN Bar;
# class Bar BUILDPLAN:
# 0: nqp::getattr(obj,Foo,'$!score') = :$score if possible
# 1: nqp::getattr(obj,Foo,'$!score') = 42 if not set
# 2: call obj.TWEAKNote that the TWEAK method is called after all of the normal checks and initializations. This is in most cases much more useful.
Although the idea of true polymorphic objects has been implemented in Raku, it turned out quite different from originally envisioned. In hindsight, one can see why it was decided to be unpractical to try to support an ever increasing list of special methods for all objects. Instead, a choice was made to only implement a few key methods from the proposal, and for the others the approach of automatic coercions was taken.
I’m not the blogging kind of person and usually don’t post without a good reason. For a long while even a good reason wasn’t enough for me to write something. But things are changing, and this subject I should have mentioned earlier.
We’re currently in the process of forming The Raku Steering Council which is considered as a potentially effective governance model for the language and the community. It’s aimed at taking off load from the shoulders of Jonathan Worthington who currently bears the biggest responsibility for the vast majority of problems the community and the language development encounter.
The biggest advantages of the Council as I see them are:
Disclaimer: everything stated above is my personal view of the situation which is to be sumed up as: the damn good thing is happening!
To the point! The Council is not an elite closed club. Anybody can nominate himself! Just read the election announce.
BTW, the announce currently states the Aug 2 is the last date to nominate. This is about to change to Sep 6. Still, don’t procrastinate too much, let the community know about your nomination and yourself!
I’m publishing the next article from ARFB series. This time rather short one, much like a warm up prior to the main workout.
But I’d like to devote this post to another subject. It’s too small for an article yet still worth special note. It was again inspired by one more post from Wenzel P.P. Peppmeyer. Actually, I knew there going to be a new post from him when found an error report in Rakudo repository. And this is the subject of the report which made me write the post.
In the report Wenzel claims that the following code results in incorrect Rakudo behaviour:
class C { };
my \px = C.new;
sub postcircumfix:«< >»(C $c is raw) {
dd $c;
}
px<ls -l>;
And that either the operator redefinition must work or the error message he gets is less than awesome:
===SORRY!=== Error while compiling /home/dex/projects/raku/lib/raku-shell-piping/px.raku Missing required term after infix at /home/dex/projects/raku/lib/raku-shell-piping/px.raku:9 ------> px<ls -l>⏏; expecting any of: prefix term
Before I tell why things are happening as intended here, let me notice two
problems with the code itself which I copied over as-is since it doesn’t work
anyway. First, the postcircumfix sub must be a multi and in Wenzel’s post it
is done correctly. Second, it must receive two arguments: first is the object it
is applied to, second is what is enclosed into the angle brakets.
So, why won’t it work as one might expect? In Raku there is a class of syntax
constructs which look like operators but in fact they’re syntax sugars. There
may be different reasons why is it done so. For example, the assignment operator
= is done this way to achieve better performance. < > makes what is inclosed
inside of it a string or a list of strings. Because of this it belongs to the
same category, as quotes "", for example. Therefore, it can only be
implemented properly as a syntax construct. When we try to redefine it we break
the compiler’s parsing and instead of a postcircumfix it finds a pair of less
than and greater than operators. Because the latter doesn’t have rhs
statement hence the error we see.
And you know, it was really useful to make this post as I realized that closing of the tickat was preliminary and that such compiler behavior is still incorrect because the attempt to redefine the op should prbably not result in bad parsing.
Good, that's the click-baity title out of the way. Sorry for taking such a long time to write again! There really has been everything going on.
To get back into blogging, I've decided to quickly write about a change I made some time ago already.
This change was for the "instrumented profiler", i.e. the one that will at run-time change all the code of the user's program, in order to measure execution times and count up calls and allocations.
In order to get everything right, the instrumented profiler keeps an entire call graph in memory. If you haven't seen something like it yet, imagine taking stack traces at every point in your program's life, and all these stack traces put together make all the paths in the tree that point at the root.
This means, among other things, that the same function can come up multiple times. With recursion, the same function can in fact come up a few hundred times "in a row". In general, if your call tree can become both deep and wide, you can end up with a whole truckload of nodes in your tree.
Is it a bad thing to have many nodes? Of course, it uses up memory. Only a single path on the tree is ever interesting at any one moment, though. Memory that's not read from or written to is not quite as "expensive". It never has to go into the CPU cache, and is even free to be swapped out to disk and/or compressed. But hold on, is this really actually the case?
It turns out that when you're compiling the Core Setting, which is a code file almost 2½ megabytes big with about 71½ thousand lines, and you're profiling during the parsing process, the tree gets enormous. At the same time, the parsing process slows to a crawl. What on earth is wrong here?
Well, looking at what MoarVM spends most of its time doing while the profiler runs gives you a good hint: It's spending almost all of its time going through the entirety of the tree for garbage collection purposes. Why would it do that, you ask? Well, in order to count allocated objects at every node, you have to match the count with the type you're allocating, and that means you need to hold on to a pointer to the type, and that in turn has to be kept up to date if anything moves (which the GC does to recently-born things) and to make sure types aren't considered unused and thrown out.
That's bad, right? Isn't there anything we can do? Well, we have to know at every node which counter belongs to which type, and we need to give all the types we have to the garbage collector to manage. But nothing forces us to have the types right next to the counter. And that's already the solution to the problem:
Holding on to all types is now the job of a little array kept just once per tree, and next to every counter there's just a little number that tells you where in the array to look.
This increases the cost of recording an allocation, as you'll now have to go to a separate memory location to match types to counters. On the other hand, the "little number" can be much smaller than before, and that saves memory in every node of the tree.
More importantly, the time cost of going through the profiler data is now independent of how big the tree is, since the individual nodes don't have to be looked at at all.
With a task as big as parsing the core setting, which is where almost every type, exception, operator, or sub lives, the difference is a factor of at least a thousand. Well, to be honest I didn't actually calculate the difference, but I'm sure it's somewhere between 100x faster and 10000x faster, and going from "ten microseconds per tree node" to "ten microseconds per tree" isn't a matter of a single factor increase, it's a complexity improvement from O(n) to O(1). As long as you can find a bigger tree, you can come up with a higher improvement factor. Very useful for writing that blog post you've always wanted to put at the center of a heated discussion about dishonest article titles!
Anyway, on testing my patch, esteemed colleague MasterDuke had this to say on IRC:
timotimo: hot damn, what did you do?!?! stage parse only took almost twice as long (i.e., 60s instead of the normal 37s) instead of the 930s last time i did the profile
(psst, don't check what 930 divided by 60 is, or else you'll expose my blog post title for the fraud that it is!)
Well, that's already all I had for this post. Thanks for your attention, stay safe, wear a mask (if by the time you're reading this the covid19 pandemic is still A Thing, or maybe something new has come up), and stay safe!
It was an emotional moment to see the keynote talk at TPRCiC from Sawyer X announcing that perl 7.00 === 5.32. Elation because of the ability of the hardcore perl community to finally break free of the frustrating perl6 roadblock. Pleasure in seeing how the risky decision to rename perl6 to raku has paid off and hopefully is beginning to defuse the tensions between the two rival communities. And Fear that improvements to perl7 will undermine the reasons for many to try out raku and may cannibalise raku usage. (Kudos to Sawyer to recognising that usage is an important design goal).
Then the left side of my brain kicked in. Raku took 15 years of total commitment of genius linguists to ingest 361 RFCs and then synthesise a new kind of programming language. If perl7 seeks the same level of completeness and perfection as raku, surely that will take the same amount of effort. And I do not see the perl community going for the same level of breaking changes that raku did. (OK maybe they could steal some stuff from raku to speed things up…)
And that brought me to Sadness. To reflect that perl Osborned sometime around 2005. That broke the community in two – let’s say the visionaries and the practical-cats. And it drove a mass emigration to Python. Ancient history.
So now we have two sister languages, and each will find a niche in the programming ecosystem via a process of Darwinism. They both inherit the traits (https://en.wikipedia.org/wiki/Perl#Design) that made perl great in the first place….
The design of Perl can be understood as a response to three broad trends in the computer industry: falling hardware costs, rising labor costs, and improvements in compiler technology. Many earlier computer languages, such as Fortran and C, aimed to make efficient use of expensive computer hardware. In contrast, Perl was designed so that computer programmers could write programs more quickly and easily.
Perl has many features that ease the task of the programmer at the expense of greater CPU and memory requirements. These include automatic memory management; dynamic typing; strings, lists, and hashes; regular expressions; introspection; and an eval() function. Perl follows the theory of “no built-in limits,” an idea similar to the Zero One Infinity rule.
Wall was trained as a linguist, and the design of Perl is very much informed by linguistic principles. Examples include Huffman coding(common constructions should be short), good end-weighting (the important information should come first), and a large collection of language primitives. Perl favours language constructs that are concise and natural for humans to write.
Perl’s syntax reflects the idea that “things that are different should look different.” For example, scalars, arrays, and hashes have different leading sigils. Array indices and hash keys use different kinds of braces. Strings and regular expressions have different standard delimiters. This approach can be contrasted with a language such as Lisp, where the same basic syntax, composed of simple and universal symbolic expressions, is used for all purposes.
Perl does not enforce any particular programming paradigm (procedural, object-oriented, functional, or others) or even require the programmer to choose among them.
But perl7 and raku serve distinct interests & needs:
| Thing… | perl7 | raku |
| compilation | static parser | one pass compiler |
| compile speed | super fast | relies on pre-c0mp |
| execution | interpreted | virtual machine |
| execution speed | super fast | relies on jit |
| module library | CPAN native | CPAN import |
| closures | yes | yes |
| OO philosophy | Cor not module | pervasive |
| OO inheritance | Roles + Is | Roles + Is + multiple |
| method invocation | -> | . |
| type checking | no | gradual |
| sigils | idiosyncratic | consistent |
| references | manual | automatic |
| unicode | feature guard | core |
| signatures | feature guard | core |
| lazy execution | nope | core |
| Junctions | nope | core |
| Rat math | nope | core |
| Sets & Mixes | nope | core |
| Complex math | nope | core |
| Grammars | nope | core |
| mutability | nope | core |
| concurrency | nope | core |
| variable scope | “notched” | cleaner |
| operators | C-like | cleaner (e.g. for ->) |
| switch | no | gather/when |
| regexen | classic | cleaner |
| eval | yes | shell |
| AST macros | huh? | … |
A long list and perhaps a little harsh on perl since many things may be got from CPAN – but when you use raku in anger, you do see the benefit if having a large core language. Only when I made this table, did I truly realise just what a comprehensive language raku is, and that perl will genuinely be the lean and mean option.
And, lest we forget our strengths:
When I first saw Python code, I thought that using indents to define the scope seemed like a good idea. However, there’s a huge downside. Deep nesting is permitted, but lines can get so wide that they wrap lines in the text editor. Long functions and long conditional actions may make it hard to match the start to the end. And I pity anyone who miscounts spaces and accidentally puts in three spaces instead of four somewhere — this can take hours to debug and track down. [Source: here]
Coming from Python, the Raku object model is recognizable, but brings a tad more structure:
What works for me, as a convenience seeker, is:
These are the things you want if you are writing in a more procedural or functional style and using class as a means to define a record type.
Here’s the rub…
When we describe OO, terms like “encapsulation” and “data hiding” often come up. The key idea here is that the state model inside the object – that is, the way it chooses to represent the data it needs in order to implement its behaviours (the methods) – is free to evolve, for example to handle new requirements. The more complex the object, the more liberating this becomes.
However, getters and setters are methods that have an implicit connection with the state. While we might claim we’re achieving data hiding because we’re calling a method, not accessing state directly, my experience is that we quickly end up at a place where outside code is making sequences of setter calls to achieve an operation – which is a form of the feature envy anti-pattern. And if we’re doing that, it’s pretty certain we’ll end up with logic outside of the object that does a mix of getter and setter operations to achieve an operation. Really, these operations should have been exposed as methods with a names that describes what is being achieved. This becomes even more important if we’re in a concurrent setting; a well-designed object is often fairly easy to protect at the method boundary.
(source jnthn https://stackoverflow.com/questions/59671027/mixing-private-and-public-attributes-and-accessors-in-raku)
Let’s fix that:
Now, I had to do a bit more lifting, but here’s what I got:
And, in contrast to Chapter 1:
I also discovered Larry’s First Law of Language Redesign: Everyone wants the colon
Apocalypse 1: The Ugly, the Bad, and the Good https://www.perl.com/pub/2001/04/02/wall.html/
I conclude that Larry’s decision was to confer the colon on the method syntax, subtly tilting the balance towards the strict model: by preferring $p.y: 3 over $p.y = 2.
Having hit rock bottom with an ‘I can’t understand my own code sufficiently enough to extend/maintain it’, I have been on a journey to review the perl5 Physics::Unit design and to use this to cut through my self made mess of raku Physics::Unit version 0.0.2.
Now I bring the perspective of a couple of years of regular raku coding to bear, so I am hoping that the bastard child of mature perl5 and raku version one will surpass both in the spirit of David Bowie’s “Pretty Things”.
One of the reasons I chose Physics::Units as a project was that, on the face of it, it seemed to have an aspect that could be approached by raku Grammars – helping me learn them. Here’s a sample of the perl5 version:
Yes – a recursive descent parser written from scratch in perl5 – pay dirt! There are 215 source code lines dedicated to the parse function. 5 more screens like this…
So I took out my newly sharpened raku tools and here’s my entire port:
Instead of ranging over 215 lines, raku has refined this down to a total of 58 lines (incl. the 11 blank ones I kept in for readability) – that’s a space saving of over 70%. Partly removal of parser boilerplate code, partly the raku Grammar constructs and partly an increased focus on the program logic as opposed to the mechanism.
For my coding style, this represents a greater than a two-thirds improvement – by getting the whole parser onto a single screen, I find that I can get the whole problem into my brain’s working memory and avoid burning cycles scrolling up and down to pin down butterflies bugs.
Attentive students will have noted that the Grammar / code integration provides a very natural paradigm for loading real-world data into an OO system, the UnitAction class starts with a stub object and populates ‘has’ attributes as it goes.
Oh and the raku code does a whole lot more such as support for unicode superscripts (up to +/-4), type assignment and type checking, offsets (such as 0 K = 273.15 °C), wider tolerance of user input and so on. Most importantly Real values are kept as Rats as much as possible which helps greatly for example, when round tripping 38.5 °C to °F and back it is still equals 38.5 °C!
One final remark – use Grammar::Tracer is a fantastic debugging tool for finding and fixing the subtle bugs that can come in and contributing to quickly getting to the optimum solution.
For anyone wondering where my occasional blog on raku has been for a couple of months – sorry. I have been busy wrestling with, and losing to, the first released version of my Physics::Measure module.
Of course, this is all a bit overshadowed by the name change from perl6 to raku. I was skeptical on this, but did not have a strong opinion either way. So kudos to the folks who thrashed this out and I am looking forward to a naissance. For now, I have decided to keep my nickname ‘p6steve’ – I enjoy the resonance between P6 and P–sics and that is my niche. No offence intended to either camp.
My stated aim (blogs passim) is to create a set of physical units that makes sense for high school education. To me, inspired by the perl5 Physics::Unit module, that means not just core SI units for science class, but also old style units like sea miles, furlongs/fortnight and so on for geography and even history. As I started to roll out v0.0.3 of raku Physics::Unit, I thought it would be worthwhile to track a real-world high school education resource, namely OpenStax CNX. As I came upon this passage, I had to take the firkin challenge on:
While there are numerous types of units that we are all familiar with, there are others that are much more obscure. For example, a firkin is a unit of volume that was once used to measure beer. One firkin equals about 34 liters. To learn more about nonstandard units, use a dictionary or encyclopedia to research different “weights and measures.” Take note of any unusual units, such as a barleycorn, that are not listed in the text. Think about how the unit is defined and state its relationship to SI units.
Disaster – I went back to the code for Physics::Unit and, blow me, could I figure out how to drop in a new Unit: the firkin??…. nope!! Why not? Well Physics::Unit v:0.0.3 was impenetrable even to me, the author. Statistically it has 638 lines of code alongside 380 lines of heredoc data. Practically, while it passes all the tests 100%, it is not a practical, maintainable code base.
How did we get here? Well I plead guilty to being an average perl5 coder who really loves the expressivity that Larry offers … but a newbie to raku. I wanted to take on Physics::Measure to learn raku. Finally, I have started to get raku – but it has taken me a couple of years to get to this point!
My best step now – bin the code. I have dumped my original effort, gone back to the original perl5 Physics::Unit module source and transposed it to raku. The result: 296 lines of tight code alongside the same 380 lines of heredoc – a reduction of 53%! And a new found respect for the design skill of my perl5 forbears.
I am aiming to release as v0.0.7 in April 2020.
By the way, you could replace … * with Inf or the unicode infinity symbol ∞ to make it more readable, i.e.
my @a = 1, 1, * + * … ∞;
— — or — —
my @a = 1, 1, * + * … Inf;
As I understand this, * + * … * means the following:
First— * + * sums the two previous elements in the list. … * tells this to do this an infinite number of times; i.e.
1, 1, (1 + 1)
1, 1, 2, (1 + 2)
1, 1, 2, 3, (2 + 3)
1, 1, 2, 3, 5, (3 + 5)
1, 1, 2, 3, 5, 8, (5 + 8), etc.
… three dots means that it does it lazy, i.e. that it does not generate an element before you call it. This can be good for large lists that are computationally heavy.
Hello everyone! In the last report I said that just a little bit of work on the heap snapshot portion of the UI should result in a useful tool.
Photo by Sticker Mule / Unsplash
Here's my report for the first useful pieces of the Heap Snapshot UI!
Last time you already saw the graphs showing how the number of instances of a given type or frame grow and shrink over the course of multiple snapshots, and how new snapshots can be requested from the UI.
The latter now looks a little bit different:
Each snapshot now has a little button for itself, they are in one line instead of each snapshot having its own line, and the progress bar has been replaced with a percentage and a little "spinner".
There are multiple ways to get started navigating the heap snapshot. Everything is reachable from the "Root" object (this is the norm for reachability-based garbage collection schemes). You can just click through from there and see what you can find.
Another way is to look at the Type & Frame Lists, which show every type or frame along with the number of instances that exist in the heap snapshot, and the total size taken up by those objects.
Clicking on a type, or the name or filename of a frame leads you to a list of all objects of that type, all frames with the given name, or all frames from the given file. They are grouped by size, and each object shows up as a little button with the ID:
Clicking any of these buttons leads you to the Explorer.
Here's a screenshot of the explorer to give you an idea of how the parts go together that I explain next:
The explorer is split into two identical panels, which allows you to compare two objects, or to explore in multiple directions from one given object.
There's an "Arrow to the Right" button on the left pane and an "Arrow to the Left" button on the right pane. These buttons make the other pane show the same object that the one pane currently shows.
On the left of each pane there's a "Path" display. Clicking the "Path" button in the explorer will calculate the shortest path to reach the object from the root. This is useful when you've got an object that you would expect to have already been deleted by the garbage collector, but for some reason is still around. The path can give the critical hint to figure out why it's still around. Maybe one phase of the program has ended, but something is still holding on to a cache that was put in as an optimization, and that still has your object in it? That cache in question would be on the path for your object.
The other half of each panel shows information about the object: Displayed at the very top is whether it is an object, a type object, an STable, or a frame.
Below that there is an input field where you can enter any ID belonging to a Collectable (the general term encompassing types, type objects, stables, and frames) to have a look.
The "Kind" field needs to have the number values replaced with human-readable text, but it's not the most interesting thing anyway.
The "Size" of the Collectable is split into two parts. One is the fixed size that every instance of the given type has. The other is any extra data an instance of this type may have attached to it, that's not a Collectable itself. This would be the case for arrays and hashes, as well as buffers and many "internal" objects.
Finally, the "References" field shows how many Collectables are referred to by the Collectable in question (outgoing references) and how many Collectables reference this object in question.
Below that there are two buttons, Path and Network. The former was explained further above, and the latter will get its own little section in this blog post.
Finally, the bottom of the panel is dedicated to a list of all references - outgoing or incoming - grouped by what the reference means, and what type it references.
In this example you see that the frame of the function display from elementary2d.p6 on line 87 references a couple of variables ($_, $tv, &inv), the frame that called this frame (step), an outer frame (MAIN), and a code object. The right pane shows the incoming references. For incoming references, the name of the reference isn't available (yet), but you can see that 7 different objects are holding a reference to this frame.
The newest part of the heap snapshot UI is the Network View. It allows the user to get a "bird's eye" view of many objects and their relations to each other.
Here's a screenshot of the network view in action:
The network view is split into two panes. The pane on the left lists all types present in the network currently. It allows you to give every type a different symbol, a different color, or optionally make it invisible. In addition, it shows how many of each type are currently in the network display.
The right pane shows the objects, sorted by how far they are from the root (object 0, the one in Layer 0, with the frog icon).
Each object has one three-piece button. On the left of the button is the icon representing the type, in the middle is the object ID for this particular object, and on the right is an icon for the "relation" this object has to the "selected" object:
This view was generated for object 46011 (in layer 4, with a hamburger as the icon). This object gets the little "map marker pin" icon to show that it's the "center" of the network. In layers for distances 3, 2, and 1 there is one object each with a little icon showing two map marker pins connected with a squiggly line. This means that the object is part of the shortest path to the root. The third kind of icon is an arrow pointing from the left into a square that's on the right. Those are objects that refer to the selected object.
There is also an icon that's the same but the arrow goes outwards from the square instead of inwards. Those are objects that are referenced by the selected object. However, there is currently no button to have every object referenced by the selected object put into the network view. This is one of the next steps I'll be working on.
Customizing the colors and visibility of different types can give you a view like this:
And here's a view with more objects in it:
Interesting observations from this image:
You have no doubt noticed that the buttons for collectables are very different between the network view and the type/frame lists and the explorer. The reason for that is that I only just started with the network view and wanted to display more info for each collectable (namely the icons to the left and right) and wanted them to look nicer. In the explorer there are sometimes thousands of objects in the reference list, and having big buttons like in the network view could be difficult to work with. There'll probably have to be a solution for that, or maybe it'll just work out fine in real-world use cases.
On the other hand, I want the colors and icons for types to be available everywhere, so that it's easier to spot common patterns across different views and to mark things you're interested in so they stand out in lists of many objects. I was also thinking of a "bookmark this object" feature for similar purposes.
Before most of that, the network viewer will have to become "navigable", i.e. clicking on an object should put it in the center, grab the path to the root, grab incoming references, etc.
There also need to be ways to handle references you're not (or no longer) interested in, especially when you come across an object that has thousands of them.
But until then, all of this should already be very useful!
Here's the section about the heap snapshot profiler from the original grant proposal:
Looking at the list, it seems like the majority of intended features are already available or will be very soon!
Until now the user had to download nodejs and npm along with a whole load of javascript libraries in order to compile and bundle the javascript code that powers the frontend of moarperf.
Fortunately, it was relatively easy to get travis-ci to do the work automatically and upload a package with the finished javascript code and the backend code to github.
You can now visit the releases page on github to grab a tarball with all the files you need! Just install all backend dependencies with zef install --deps-only . and run service.p6!
And with that I'm already done for this report!
It looks like the heap snapshot portion of the grant is quite a bit smaller than the profiler part, although a lot of work happened in moarvm rather than the UI. I'm glad to see rapid progress on this.
I hope you enjoyed this quick look at the newest pieces of moarperf!
- Timo
It has been a while since the last progress report, hasn't it?
Over the last few months I've been focusing on the MoarVM Heap Snapshot Profiler. The new format that I explained in the last post, "Intermediate Progress Report: Heap Snapshots", is available in the master branch of MoarVM, and it has learned a few new tricks, too.
The first thing I usually did when opening a Heap Snapshot in the heapanalyzer (the older command-line based one) was to select a Snapshot, ask for the summary, and then for the top objects by size, top objects by count, top frames by size, and/or top frames by count to see if anything immediately catches my eye. In order to make more sense of the results, I would repeat those commands for one or more other Snapshots.
Snapshot Heap Size Objects Type Objects STables Frames References
======== ================= ======= ============ ======= ====== ==========
0 46,229,818 bytes 331,212 686 687 1,285 1,146,426
25 63,471,658 bytes 475,587 995 996 2,832 1,889,612
50 82,407,275 bytes 625,958 1,320 1,321 6,176 2,741,066
75 97,860,712 bytes 754,075 1,415 1,416 6,967 3,436,141
100 113,398,840 bytes 883,405 1,507 1,508 7,837 4,187,184
Snapshot Heap Size Objects Type Objects STables Frames References
======== ================= ========= ============ ======= ====== ==========
125 130,799,241 bytes 1,028,928 1,631 1,632 9,254 5,036,284
150 145,781,617 bytes 1,155,887 1,684 1,685 9,774 5,809,084
175 162,018,588 bytes 1,293,439 1,791 1,792 10,887 6,602,449
Realizing that the most common use case should be simple to achieve, I first implemented a command summary all and later a command summary every 10 to get the heapanalyzer to give the summaries of multiple Snapshots at once, and to be able to get summaries (relatively) quickly even if there's multiple hundreds of snapshots in one file.
Sadly, this still requires the parser to go through the entire file to do the counting and adding up. That's obviously not optimal, even though this is an Embarrassingly Parallel task, and it can use every CPU core in the machine you have, it's still a whole lot of work just for the summary.
For this reason I decided to shift the responsibility for this task to MoarVM itself, to be done while the snapshot is taken. In order to record everything that goes into the Snapshot, MoarVM already differentiates between Object, Type Object, STable, and Frame, and it stores all references anyway. I figured it shouldn't have a performance impact to just add up the numbers and make them available in the file.
The result is that the summary table as shown further above is available only milliseconds after loading the heap snapshot file, rather than after an explicit request and sometimes a lengthy wait period.
The next step was to see if top objects by size and friends could be made faster in a similar way.
I decided that adding an optional "statistics collection" feature inside of MoarVM's heap snapshot profiler would be worthwhile. If it turns out that the performance impact of summing up sizes and counts on a per-type and per-frame basis makes capturing a snapshot too slow, it could be turned off.
> snapshot 50
Loading that snapshot. Carry on...
> top frames by size
Wait a moment, while I finish loading the snapshot...
Name Total Bytes
==================================== =============
finish_code_object (World.nqp:2532) 201,960 bytes
moarop_mapper (QAST.nqp:1764) 136,512 bytes
!protoregex (QRegex.nqp:1625) 71,760 bytes
new_type (Metamodel.nqp:1345) 40,704 bytes
statement (Perl6-Grammar.nqp:951) 35,640 bytes
termish (Perl6-Grammar.nqp:3641) 34,720 bytes
<anon> (Perl6-BOOTSTRAP.c.nqp:1382) 29,960 bytes
EXPR (Perl6-Grammar.nqp:3677) 27,200 bytes
<mainline> (Perl6-BOOTSTRAP.c.nqp:1) 26,496 bytes
<mainline> (NQPCORE.setting:1) 25,896 bytes
EXPR (NQPHLL.nqp:1186) 25,760 bytes
<anon> (<null>:1) 25,272 bytes
declarator (Perl6-Grammar.nqp:2189) 23,520 bytes
<anon> (<null>:1) 22,464 bytes
<anon> (<null>:1) 22,464 bytes
Showing the top objects or frame for a single snapshot is fairly straight-forward in the commandline based UI, but how would you display how a type or frame develops its value across many snapshots?
Instead of figuring out the best way to display this data in the commandline, I switched focus to the Moarperf Web Frontend. The most obvious way to display data like this is a Line Graph, I believe. So that's what we have now!
And of course you also get to see the data from each snapshot's Summary in graph format:
And now for the reason behind this blog post's Title.
Using Jonathan's module Concurrent::Progress (with a slight modification) I sprinkled the code to parse a snapshot with matching calls of .increment-target and .increment. The resulting progress reports (throttled to at most one per second) are then forwarded to the browser via the WebSocket connection that already delivers many other bits of data.
The result can be seen in this tiny screencast:
The recording is rather choppy because the heapanalyzer code was using every last drop of performance out of my CPU while it was trying to capture my screen.
There's obviously a lot still missing from the heap snapshot analyzer frontend GUI, but I feel like this is a good start, and even provides useful features already. The graphs for the summary data are nicer to read than the table in the commandline UI, and it's only in this UI that you can get a graphical representation of the "highscore" lists.
I think a lot of the remaining features will already be useful after just the initial pieces are in place, so a little work should go a long way.
I didn't spend the whole time between the last progress report and now to work directly on the features shown here. Apart from Life Intervening™, I worked on fixing many frustrating bugs related to both of the profilers in MoarVM. I added a small subsystem I call VMEvents that allows user code to be notified when GC runs happen and other interesting bits from inside MoarVM itself. And of course I've been assisting other developers by answering questions and looking over their contributions. And of course there's the occasional video-game-development related experiment, for example with the GTK Live Coding Tool.
Finally, here's a nice little screencap of that same tool displaying a hilbert curve:
That's already everything I have for this time. A lot has (had to) happen behind the scenes to get to this point, but now there was finally something to look at (and touch, if you grab the source code and go through the needlessly complicated build process yourself).
Thank you for reading and I hope to see you in the next one!
- Timo
You’re right. I’ll let the article stand as it is and reflect my ignorance as it was when I wrote it :-)
This week’s Perl Weekly Challenge (#19) has two tasks. The first is to find all months with five weekends in the years from 1900 through 2019. The second is to program an implementation of word wrap using the greedy algorithm.
Both are pretty straight-forward tasks, and the solutions to them can (and should) be as well. This time, however, I’m also going to do the opposite and incrementally turn the easy solution into an unnecessarily complex one. Because in this particular case we can learn more by doing things the unnecessarily hard way. So this post will take a look at Dates and date manipulation in Perl 6, using PWC #19 task 1 as an example:
Write a script to display months from the year 1900 to 2019 where you find 5 weekends i.e. 5 Friday, 5 Saturday and 5 Sunday.
Let’s start by finding five-weekend months the easy way:
#!/usr/bin/env perl6
say join "\n", grep *.day-of-week == 5, map { Date.new: |$_, 1 }, do 1900..2019 X 1,3,5,7,8,10,12;The algorithm for figuring this out is simple. Given the prerequisite that there must be five occurrences of not only Saturday and Sunday but also Friday, you A) *must* have 31 days to cram five weekends into. And when you know that you’ll also see that B) the last day of the month MUST be a Sunday and C) the first day of the month MUST be a Friday (you don’t have to check for both; if A is true and B is true, C is automatically true too).
The code above implements B and employs a few tricks. You read it from right to left (unless you write it from left to right, like this… say do 1900..2019 X 1,3,5,7,8,10,12 ==> map { Date.new: |$_, 1 } ==> grep *.day-of-week == 5 ==> join “\n”; )
Using the X operator I create a cross product of all the years in the range 1900–2019 and the months 1, 3, 5, 7, 8, 10, 12 (31-day months). In return I get a sequence containing all year-month pairs of the period.
The map function iterates through the Seq. There it instantiates a Date object. A little song and dance is necessary: As Date.new takes three unnamed integer parameters, year, month and day, I have to do something to what I have — a Pair with year and month. I therefore use the | operator to “explode” the pair into two integer parameters for year and month.
You can always use this for calling a sub routine with fixed parameters, using an array with parameter values rather than having separate variables for each parameter. The code below exemplifies usage:
my @list = 1, 2, 3;
sub explode-parameters($one, $two, $three) {
…do something…
}#traditional call
explode-parameters(@list[0], @list[1], @list[2]);
# …or using |
explode-parameters(|@list);
Back to the business at hand — the .grep filters out the months where the 1st is a Friday, and those are our 5 weekend months. So the output of the one-liner above looks something like this:
...
1997-08-01
1998-05-01
1999-01-01
...
This is a solution as good as any, and if a solution was all we wanted, we could have stopped here. But using this task as an example I want to explore ways to utilise the Date class. Example: The one-liner above does the job, but strictly speaking it doesn’t output the months but the first day of those months. Correcting this is easy, because the Date class supports something called formatters and use the sprintf syntax. To do this you utilise the named parameter “formatter” when instantiating the object.
say join "\n", grep *.day-of-week == 5, map { Date.new: |$_, 1, formatter => { sprintf "%04d/%02d", .year, .month } }, do 1900..2019 X 1,3,5,7,8,10,12;Every time a routine pulls a stringified version of the date, the formatter object is invoked. In our case the output has been changed to…
...
1997/08
1998/05
1999/01
...
Formatters are powerful. Look into them.
Now to the overly complex solution. This is the unthinking programmer’s solution, as we don’t suppose anything. The program isn’t told that 5 weekend months only can occur on 31 day months. It doesn’t know that the 1st of such months must be a Friday. All it knows is that if the last day of the month is not Sunday, it figures out the date of the last Sunday (this is not very relevant when counting three-day weekends, but could be if you want to find Saturday+Sunday weekends, or only Sundays).
#!/usr/bin/env perl6
my $format-it = sub ($self) {
sprintf "%04d month %02d", .year, .month given $self;
}sub MAIN(Int :$from-year = 1900, Int :$to-year where * > $from-year = 2019, Int :$weekend-length where * ~~ 1..3 = 3) {
my $date-loop = Date.new($from-year, 1, 1, formatter => $format-it);
while ($date-loop.year <= $to-year) {
my $date = $date-loop.later(day => $date-loop.days-in-month);
$date = $date.truncated-to('week').pred if $date.day-of-week != 7;
my @weekend = do for 0..^$weekend-length -> $w {
$date.earlier(day => $w).weekday-of-month;
};
say $date-loop if ([+] @weekend) / @weekend == 5;
$date-loop = $date-loop.later(:1month);
}
}This code can solve the task both for three day weekends, but also for weekends consisting of Saturday + Sunday, as well as only Sundays. You control that with the command line parameter weekend-length=[1..3].
This code finds the last Sunday of each month and counts whether it has occured five times that month. It does the same for Saturday (if weekend-length=2) and Friday (if weekend-length=3). Like this:
my @weekend = do for 0..^$weekend-length -> $w {
$date.earlier(day => $w).weekday-of-month;
};The code then calculcates the average weekday-of-month for these three days like this:
say $date-loop if ([+] @weekend) / @weekend == 5;
This line uses the reduction operator [+] on the @weekend list to find the sum of all elements. That sum is divided by the number of elements. If the result is 5, then you have a five day weekend.
As for fun stuff to do with the Date object:
.later(day|month|year => Int) — adds the given number of time units to the current date. There’s also an earlier method for subtracting.
.days-in-months — tells you how many days there are in the current month of the Date object. The value may be 31, 30, 29 (february, leap year) or 28 (february).
.truncated-to(week|month|day|year) — rolls the date back to the first day of the week, month, day or year.
.weekday-of-month — figures out what day of week the current date is and calculates how many of that day there has been so far in that month.
Apart from this you’ll see that I added the formatter in a different way this time. This is probably cleaner looking and easier to maintain.
In the end this post maybe isn’t about dates and date manipulation at all, but rather is a call for all of us to use the documentation even more. It’s often I think that Perl 6 should have a function for x, y or z — .weekday-of-month is one such example — and the documentation tells me that it actually does!
It’s very easy to pick up Perl 6 and program it as you would have programmed Perl 5 or other languages you know well. But the documentation has lots of info of things you didn’t have before and that will make programming easier and more fun when you’ve learnt about them.
I guess you don’t need and excuse to delve into the docs, but if you do the Perl Weekly Challenge is an excellent excuse for spending time in the docs!
This is slightly tangential to the Rakudo Perl 6 Performance Analysis Tooling grant from The Perl Foundation, but it does interact closely with it, so this is more or less a progress report.
The other day, Jonathan Worthington of Edument approached me with a little paid Job: Profiling very big codebases can be tedious, especially if you're interested in only one specific fraction of the program. That's why I got the assignment to make the profiler "configurable". On top of that, the Comma IDE will get native access to this functionality.
The actual request was just to allow specifying whether individual routines should be included in the profile via a configuration file. That would have been possible with just a simple text file that contains one filename/line number/routine name per line. However, I have been wanting something in MoarVM that allows much more involved configuration for many different parts of the VM, not just the profiler.
Obligatory cat photo.
That's why I quickly developed a small and simple "domain-specific language" for this and similar purposes.
The language had a few requirements:
There's also some things that aren't necessary:
While thinking about what exactly I should build – before I eventually settled on building a "programming language" for this task – I bounced back and forth between the simplest thing that could possibly work (for example, a text file with a list of file/line/name entries) and the most powerful thing that I can implement in a sensible timeframe (for example, allowing a full NQP script). A very important realization was that as long as I require the first line to identify what "version" of configuration program it is, I could completely throw away the current design and put something else instead, if the need ever arises. That allowed me to actually commit to a design that looked at least somewhat promising. And so I got started on what I call confprog.
Here's an example program. It doesn't do very much, but shows what it's about in general:
version = 1
entry profiler_static:
log = sf.name;
profile = sf.name eq "calculate-strawberries"
profile |= sf.cu.filename eq "CustomCode.pm6"
The entry decides which stage of profiling this program is applied to. In this case, the profiler_static means we're seeing a routine for the first time, before it is actually entered. That's why only the information every individual invocation of the frame in question shares is available via the variable sf, which stands for Static Frame. The Static Frame also allows access to the Compilation Unit (cu) that it was compiled into, which lets us find the filename.
The first line that actually does something assigns a value to the special variable log. This will output the name of the routine the program was invoked for.
The next line will turn on profiling only if the name of the routine is "calculate-strawberries". The line after that will also turn on profiling if the filename the routine is from is "CustomCode.pm6".
Apart from profiler_static, there are a couple more entry points available.
profiler_static, which runs the first time any routine is encountered, and stores the decision until the profile run is finished.profiler_dynamic will be evaluated every time a routine gets entered that would potentially be profiled otherwise (for example because a profiler_static program turned it on, or because there is no profiler_static program, which means that every routine is eligible for profiling).heapsnapshot will be run every time a GC run happens. It lets the user decide whether a GC run should result in a heap snapshot being taken or not, based on whether the run has been a minor or major collection, and arbitrary other factors, including time since start of the program, or time since the last heap snapshot was taken.spesh and jit I'm not entirely sure about yet. I want to allow turning the specializer or the jit off completely for certain routines, but I also want to offer control over individual optimizations in spesh (turning off inlining for one particular routine, preventing scalar replacement but only for Rat types, ...) and behaviour of the jit (use the template jit for everything except one specific piece of one particular routine, ...).The syntax is still subject to change, especially before the whole thing is actually in a released version of MoarVM.
There is a whole lot of other things I could imagine being of interest in the near or far future. One place I'm taking inspiration from is where "extended Berkeley Packet Filter" (eBPF for short) programs are being used in the linux kernel and related pieces of software:
Oversimplifying a bit, BPF was originally meant for tcpdump so that the kernel doesn't have to copy all data over to the userspace process so that the decision what is interesting or not can be made. Instead, the kernel receives a little piece of code in the special BPF language (or bytecode) and can calculate the decision before having to copy anything.
eBPF programs can now also be used as a complex ruleset for sandboxing processes (with "seccomp"), to decide how network packets are to be routed between sockets (that's probably for Software Defined Networks?), what operations a process may perform on a particular device, whether a trace point in the kernel or a user-space program should fire, and so on.
So what's the status of confprog? I've written a parser and compiler that feeds confprog "bytecode" (which is mostly the same as regular moarvm bytecode) to MoarVM. There's also a preliminary validator that ensures the program won't do anything weird, or crash, when run. It is much too lenient at the moment, though. Then there's an interpreter that actually runs the code. It can already take an initial value for the "decision output value" (great name, isn't it) and it will return whatever value the confprog has set when it runs. The heap snapshot profiler is currently the only part of MoarVM that will actually try to run a confprog, and it uses the value to decide whether to take a snapshot or not.
Next up on the list of things to work on:
Apart from improvements to the confprog programming language, the integration with MoarVM lacks almost everything, most importantly installing a confprog for the profiler to decide whether a frame should be profiled (which was the original purpose of this assignment).
After that, and after building a bit of GUI for Comma, the regular grant work can resume: Flame graphs are still not visible on the call graph explorer page, and heap snapshots can't be loaded into moarperf yet, either.
Thanks for sticking with me through this perhaps a little dry and technical post. I hope the next one will have a little more excitement! And if there's interest (which you can signal by sending me a message on irc, or posting on reddit, or reaching me via twitter @loltimo, on the Perl 6 discord server etc) I can also write a post on how exactly the compiler was made, and how you can build your own compiler with Perl 6 code. Until then, you can find Andrew Shitov's presentations about making tiny languages in Perl 6 on youtube.
I hope you have a wonderful day; see you in the next one!
- Timo
PS: I would like to give a special shout-out to Nadim Khemir for the wonderful Data::Dump::Tree module which made it much easier to see what my parser was doing. Here's some example output from another simple confprog program:
[6] @0
├ 0 = .Label .Node @1
│ ├ $.name = heapsnapshot.Str
│ ├ $.type = entrypoint.Str
│ ├ $.position is rw = Nil
│ └ @.children = [0] @2
├ 1 = .Op .Node @3
│ ├ $.op = =.Str
│ ├ $.type is rw = Nil
│ └ @.children = [2] @4
│ ├ 0 = .Var .Node @5
│ │ ├ $.name = log.Str
│ │ └ $.type = CPType String :stringy @6
│ └ 1 = String Value ("we're in") @7
├ 2 = .Op .Node @8
│ ├ $.op = =.Str
│ ├ $.type is rw = Nil
│ └ @.children = [2] @9
│ ├ 0 = .Var .Node @10
│ │ ├ $.name = log.Str
│ │ └ $.type = CPType String :stringy §6
│ └ 1 = .Op .Node @12
│ ├ $.op = getattr.Str
│ ├ $.type is rw = CPType String :stringy §6
│ └ @.children = [2] @14
│ ├ 0 = .Var .Node @15
│ │ ├ $.name = sf.Str
│ │ └ $.type = CPType MVMStaticFrame @16
│ └ 1 = name.Str
├ 3 = .Op .Node @17
│ ├ $.op = =.Str
│ ├ $.type is rw = Nil
│ └ @.children = [2] @18
│ ├ 0 = .Var .Node @19
│ │ ├ $.name = log.Str
│ │ └ $.type = CPType String :stringy §6
│ └ 1 = String Value ("i am the confprog and i say:") @21
├ 4 = .Op .Node @22
│ ├ $.op = =.Str
│ ├ $.type is rw = Nil
│ └ @.children = [2] @23
│ ├ 0 = .Var .Node @24
│ │ ├ $.name = log.Str
│ │ └ $.type = CPType String :stringy §6
│ └ 1 = String Value (" no heap snapshots for you my friend!") @26
└ 5 = .Op .Node @27
├ $.op = =.Str
├ $.type is rw = Nil
└ @.children = [2] @28
├ 0 = .Var .Node @29
│ ├ $.name = snapshot.Str
│ └ $.type = CPType Int :numeric :stringy @30
└ 1 = Int Value (0) @31
Notice how it shows the type of most things, like name.Str, as well as cross-references for things that appear multiple times, like the CPType String. Particularly useful is giving your own classes methods that specify exactly how they should be displayed by DDT. Love It!
 |
| Both ADD and SUB refer to the same LOAD node |
 |
| The DO node is inserted for the LET operator. It ensures that the value of the LOAD node is computed before the reference in either branch |
This evening, I enjoyed the New Year’s fireworks display over the beautiful Prague skyline. Well, the bit of it that was between me and the fireworks, anyway. Rather than having its fireworks display at midnight, Prague puts it at 6pm on New Year’s Day. That makes it easy for families to go to, which is rather thoughtful. It’s also, for those of us with plans to dig back into work the following day, a nice end to the festive break.
So, tomorrow I’ll be digging back into work, which of late has involved a lot of Perl 6. Having spent so many years working on Perl 6 compiler and runtime design and implementation, it’s been fun to spend a good amount of 2018 using Perl 6 for commercial projects. I’m hopeful that will continue into 2019. Of course, I’ll be continuing to work on plenty of Perl 6 things that are for the good of all Perl 6 users too. In this post, I’d like to share some of the things I’m hoping to work on or achieve during 2019.
The MoarVM specializer learned plenty of new tricks this year, delivering some nice speedups for many Perl 6 programs. Many of my performance improvement hopes for 2019 center around escape analysis and optimizations stemming from it.
The idea is to analyze object allocations, and find pieces of the program where we can fully understand all of the references that exist to the object. The points at which we can cease to do that is where an object escapes. In the best cases, an object never escapes; in other cases, there are a number of reads and writes performed to its attributes up until its escape.
Armed with this, we can perform scalar replacement, which involves placing the attributes of the object into local registers up until the escape point, if any. As well as reducing memory operations, this means we can often prove significantly more program properties, allowing further optimization (such as getting rid of dynamic type checks). In some cases, we might never need to allocate the object at all; this should be a big win for Perl 6, which by its design creates lots of short-lived objects.
There will be various code-generation and static optimizer improvements to be done in Rakudo in support of this work also, which should result in its own set of speedups.
Expect to hear plenty about this in my posts here in the year ahead.
The current Rakudo startup time is quite high. I’d really like to see it fall to around half of what it currently is during 2019. I’ve got some concrete ideas on how that can be achieved, including changing the way we store and deserialize NFAs used by the parser, and perhaps also dealing with the way we currently handle method caches to have less startup impact.
Both of these should also decrease the base memory use, which is also a good bit higher than I wish.
Some folks – myself included – are developing increasingly large applications in Perl 6. For the current major project I’m working on, runtime performance is not an issue by now, but I certainly feel myself waiting a bit on compiles. Part of it is parse performance, and I’d like to look at that; in doing so, I’d also be able to speed up handling of all Perl 6 grammars.
I suspect there are some good wins to be had elsewhere in the compilation pipeline too, and the startup time improvements described above should also help, especially when we pre-compile deep dependency trees. I’d also like to look into if we can do some speculative parallel compilation.
In Perl 6.d, we got non-blocking await and react support, which greatly improved the scalability of Perl 6 concurrent and parallel programs. Now many thousands of outstanding tasks can be juggled across just a handful of threads (the exact number chosen according to demand and CPU count).
For Perl 6.e, which is still a good way off, I’d like to having something to offer in terms of making Perl 6 concurrent and parallel programming safer. While we have a number of higher-level constructs that eliminate various ways to make mistakes, it’s still possible to get into trouble and have races when using them.
So, I plan to spend some time this year quietly exploring and prototyping in this space. Obviously, I want something that fits in with the Perl 6 language design, and that catches real and interesting bugs – probably by making things that are liable to occasionally explode in weird ways instead reliably do so in helpful ways, such that they show up reliably in tests.
In the 16 months since I revealed it, Cro has become a popular choice for implementing HTTP APIs and web applications in Perl 6. It has also attracted code contributions from a couple of dozen contributors. This year, I aim to see Cro through to its 1.0 release. That will include (to save you following the roadmap link):
I founded Comma IDE in order to bring Perl 6 a powerful Integrated Development Environment. We’ve come a long way since the Minimum Viable Product we shipped back in June to the first subscribers to the Comma Supporter Program. In recent months, I’ve used Comma almost daily on my various Perl 6 projects, and by this point honestly wouldn’t want to be without it. Like Cro, I built Comma because it’s a product I wanted to use, which I think is a good place to be in when building any product.
In a few months time, we expect to start offering Comma Community and Comma Complete. The former will be free of charge, and the latter a commercial offering under a subscribe-for-updates model (just like how the supporter program has worked so far). My own Comma wishlist is lengthy enough to keep us busy for a lot more than the next year, and that’s before considering things Comma users are asking for. Expect plenty of exciting new features, as well as ongoing tweaks to make each release feel that little bit nicer to use.
This year will see me giving my first keynote at a European Perl Conference. I’m looking forward to being in Riga again; it’s a lovely city to wander around, and I remember having some pretty nice food there too. The keynote will focus on the concurrent and parallel aspects of Perl 6; thankfully, I’ve still a good six months to figure out exactly what angle I wish to take on that, having spoken on the topic many times before!
I also plan to submit a talk or two for the German Perl Workshop, and will probably find the Swiss Perl Workshop hard to resist attending once more. And, more locally, I’d like to try and track down other Perl folks here in Prague, and see if I can help some kind of Praha.pm to happen again.
I need to keep my travel down to sensible levels, but might be able to fit in the odd other bit of speaking during the year, if it’s not too far away.
While I want to spend most of my time building stuff rather than talking about it, I’m up for the occasional bit of teaching. I’m considering pitching a 1-day Perl 6 concurrency workshop to the Riga organizers. Then we’ll see if there’s enough folks interested in taking it. It’ll cost something, but probably much less than any other way of getting a day of teaching from me. :-)
Well, a good night’s sleep first. :-) But tomorrow, another year of fun begins. I’m looking forward to it, and to working alongside many wonderful folks in the Perl community.
I want to revive Carl Mäsak's Coding Contest as a crowd-sourced contest.
The contest will be in four phases:
For the first phase, development of tasks, I am looking for volunteers who come up with coding tasks collaboratively. Sadly, these volunteers, including myself, will be excluded from participating in the second phase.
I am looking for tasks that ...
This is non-trivial, so I'd like to have others to discuss things with, and to come up with some more tasks.
If you want to help with task creation, please send an email to moritz.lenz@gmail.com, stating your intentions to help, and your freenode IRC handle (optional).
There are other ways to help too:
In these cases you can use the same email address to contact me,
or use IRC (moritz on freenode) or twitter.