What are history-animations?
History-animations build on the following feature (one that was already existing): whenever we select an s-expression in our “tree” (the structural view on the right hand side of the window) we show the history of that particular s-expression in the panel on the left. That is, whenever we change the cursor in the tree, we switch what is shown in the history.
The animation under discussion: any part of history that shows up both before and after this switch will “float” from its pre-switch position to its post-switch position in a number of steps. The idea is to make it visually more clear that there is a relationship between the histories at different levels of the tree.
More details and examples can be found in a separate article. In the present article we’ll zoom in on the implementation.
The current version of the editor supports 2 versions of rendering histories: one in which the histories are rendered as s-expressions themselves, as presented in the paper the paper “Clef Design”; one in which the effects of each note are shown in the context of the structure on which it is played (i.e.: more like a traditional rendering of a diff). Animations of transitions are implemented for both of these; where the implementations diverge this will be pointed out in the below.
Identity of notes and textures
The key idea in the animations is to float textures from some pre-switch to a post-switch location. This hinges on the assumption that we have a shared identity for the textures pre- and post-switch. E.g. to float some open-bracket from one location to the next, we need to know which open-bracket we’re talking about (there are many, and they look very similar).
Note that the particular animation under consideration is the following: when swichting which part of our structural view (the “tree”) is selected, update the historical view.
Thus, the assumption of shared identity, in this case, is: there is overlap between the histories of different parts of our tree. For each of the elements (notes) of the history we can establish an identity, and when viewing a different history, we can establish whether any two notes across these two histories are the same one, i.e. share this identity.
The fact that parts of histories are shared across different parts of our structure is detailed in the paper “Clef Design”
In terms of the implementation, the solution is to have some addressing scheme for the textures that is global in the sense that it is shared between the pre-and post-switch environments. Using this addressing scheme we can identify textures: same address means same texture.
Such an addressing scheme for textures is obtained in a number of steps.
The formalization of the note-address is implemented in the class
The intuition here is: when the whole history is written out as an expression, the address of a particular note is a path trough that expression. An example could be: of the global score, take the 6th item; of that item take the only child, of that item again take the only child. The 2 main possible parts of such paths are: the nth item of a Score, and the only child. The doctests provide further details.
Push global NoteAddress to the tree
In the second step, we construct a tree by playing this global history of notes, annotated with their global address (here and here). We use the regular mechanism of playing a score to get a tree (This one – in fact, it’s not 100% identical for implementation reasons, as documented in the code, but in terms of behavior it is). The only difference is: because the input Notes have now been annotated with a global address, the scores as constructed at each sub-expression in the resulting tree are now consisting of notes which have a global address. This means that when we fetch the “local score” (the score to be rendered) we have information about the global address of each note.
Finally, we make sure to keep the annotations around in each step of the conversion to textures, as well as add conversion-specific information when needed. The implementations of this final step are unique for each of the two different styles of rendering.
ELS’18 style rendering
In the case rendering of in the style of the ELS’18 paper, the tree of notes is first converted to an s-expr, and these s-expressions are then converted to the actual textures with locations.
We need step-specific address information for each of these steps. When converting to
an s-expression, we annotate the elements that are specific to the fact that the
note is being rendered as an s-expression (i.e. the fact that the Note’s fields
and its type, when converted to an s-expression, turn into particular
further s-expressions). Let’s consider the case of
become-atom as an
when the note
(become-atom foo) is represented as an s-expression the whole
s-expression is annotated as representing the whole note (by not providing any
further annotation), the atom
become-atom is annotated as being the name of
the note, and the atom
foo is annotated as being the field
atom of that
A particular property of this style of rendering histories, is that the recursive nature of the histories is preserved in the rendering. That is: a note may contain further notes; when the note is rendered, the notes it contains are also rendered.
With regards to the assignment of addresses to textures, the implication is straightforward: each rendered note is assigned with the address of that particular note.
An example is drawn below: if the chord below is the item at position 1 in some other history, the children of that chord are at some subpath.
(chord ((insert 0 (become-list)) (extend 0 (insert 0 (become-list))))) ^ ^ ^ | | | (@1) (@1, @0) (@1, @1)
The effect of this approach on the animation is precisely as intended: when switching from a larger context to a smaller one, the “surrounding” notes that are not applicable in the smaller context float out of view; but those that are applicable in both views (the inner ones), float from their old position on the screen to the new one. (The reverse applies when switching from a smaller context to one surrounding it)
Another way of rendering notes is by rendering them “in their structural context”. That is: by showing their effect on the existing structure on which they are being played. This is how diffs are traditionally displayed.
In this view, the recursive nature of notes is not made explicit. For each note in some list of notes (for example: those that make up a single score), the effect of each indivual note on a structure are grouped together. The fact each such note may itself be composed of any number of other notes is left implict.
Thus, when switching from a larger historical context to a smaller one, it is not the case that some surrounding notes disappear, while notes contained by them remain in view.
There simply is no direct rendering of notes in this view: everything that is rendered is a structure and some effects on that structure. This means that any addressing must also apply to such structures. And that any floating of related elements is always floating of some structural element.
It is at this structural level that a similar effect as in the above, of surrounding context disappearing, can be seen: when switching to a smaller structural context, less surrounding structure is shown in the in-context rendering of history, and vise versa for switching to a larger, surrounding, context:
The implentation details are in the implementing class,
The mixing of ‘construction’ and ‘structure’ is reflected in the address of the
rendered elements; each rendered element is denoted first by the note which it
represents (in terms of a
NoteAddress), and second by an address (
for stability over time) in the tree. (further steps in the rendering chain add
further details, i.e.
One final caveat: the NoteAddress
NoteAddress part of this
always the address of the deepest (leaf-most) possible note. For example, when
rendering the note
(extend 0 (insert 0 (become-list))), the address of
(become-list) is used in the
ICHAddress. This ensures we have a singular
identity across context-switches. (It is only this deepest NoteAddress that can
be relied on to always be availalble).
The actual animation is rather straightforward: do a linear interpolation for (source, target) for the attributes (x, y, alpha).
We set a clock at an interval (I’ve set 1/60, but I’m not actually getting this
at all on my local machine). Kivy will tell you how much time has actually
passed since the last tick. We then calculate the fraction
dt / remaining_time.
This approach is automatically robust for missed frames (i.e. the missed frame
will not be rendered, but the total animation time and the position of the
texture at the next frame are unaffected)