* Working on KeyframeAnimation ctors and related bits (still work in progress)
* Architecture design work
Shane, Doug, Steve:
* tests in the polyfil
* time sources
2. ANIMATION TIMELINES AND TRIGGERS
As I understand the two proposals we have:
a) A simple time source is strictly a root-level thing. It's basically like a group but without the ownership semantics (i.e. keeps its children alive forever). Groups, implement the same interface and add ownership semantics and scheduling for their children (par/seq/mediacontroller-style-sync).
b) Animations can have BOTH a parent group and a time source and there is a defined method to manage the two so that the time source gets applied inside the group's time window. In the common case, an animation's time source is its parent group.
Let's consider use cases for (b).
Use case 1: A primarily UI-driven "animation" with timed parts
Description: The animation is driven by scrolling but when you reach certain key points animations are triggered. For example, a comic where as you scroll the characters move, but when the hero's fist connects with the villain's cheek, a caption saying "Blammo!" springs out with a starburst in the background. This part animates over 0.5s regardless of progress of the scrollbar. However, if you're scrolling really quickly and scroll past a certain threshold the animation should jump to the end or be cancelled (presumably defined by the fill mode).
Analysis: This fits really nicely with (b). You have time windows controlled by the scrolling that set out when the timed animations can play. When you scroll fast you hit the end of the window and that could be used to fast-forward the timed animations to their fill state.
Note that this could currently be achieved with script by simply triggering independent animations whose time source is the document time source when we hit the "begin" event for the parent group. Likewise the "end" event could be used to fast-forward. However, a declarative solution is likely preferable for this use case since it does appear to be something one might expect an epub publisher to produce. That said, a simple script framework could interpret some suitable markup and realise this use case using the Web Animations API and that might even be a preferable approach given that this use (mixing UI-driven time sources and clock-driven time sources) is probably rare.
Shane raises issue of jank involved in script-based triggering.
Agreed declarative solution for this is better in the long-term.
Use case 2: A primarily time-driven animation with UI driven parts
Description: There is an interactive cartoon which plays by itself but requires the user to perform certain actions in order for it to continue. e.g. drag the door open to enter.
Analysis: Whilst again, (b) seems to meet this case well, my impression of this sort of use case is that it will get complex really quickly (e.g. you'll have choose-your-own-adventure style interactions) and you're not going to be able to do it without some script. So long as you permit the possibility of using script we can already do all this.
Shane: I agree - this isn't a great fit for either (a) or (b) unless the interaction is very simple. State machines will help us a lot here, though - and once they do, (b) is important.
Use case 3: Momentum the easy way
It's probably possible to achieve momentum using a purely UI-driven approach by simply controlling the values you return from ScrollTimeSource.currentTime. That might be a bit tricky and might mean you can't re-use the time source for multiple animations?
Shane: Script is the killer here, especially on mobile platforms. Or do you mean defining specific behaviours in the spec? I think we're going to have to do that regardless of which approach we take.
Brian: Yes, I mean you could define a TimeSource in the spec that returns momentum-weighted values for, e.g. ScrollTimeSource.currentTime. Not from script.
Brian: Regarding re-using time sources, I think it's reasonable to require independent time sources for animations that behave independently. That is, you don't simply have one time source representing vertical scroll, but provide the possibility of creating several such time sources, each offering their own momentum. You'd define the notches on the time source not the animations.
Shane: Agreed. I don't think momentum the easy way is the right use case for (b) - if anything, (b) give you "momentum the flexible but complicated way" (i.e. supporting post-user-interaction animations that don't neatly fit into whatever declarative time sources we provide).
On the other hand, time source and time source transformation makes it easier to define the momentum time sources, because they're adapters over the horizontal or vertical scrolling time source.
Going forward: For me (Brian), the question is what is the minimum we can include in v1 without compromising our ability to support these kind of use cases in the future? Can we start with (a) and incorporate something like (b) in the future?
My suggestion is we do pretty much (a), or, in fact, even less:
* Define a TimeSource interface with the single member:
attribute double currentTime;
* Replace TimedItem.parentGroup with TimedItem.parentTime
* Make AnimationGroup implement TimeSource
* Make MediaController implement TimeSource?
-- this, amazingly, might just work since MediaController includes the following attribute:
attribute double currentTime;
(it's settable, it's in seconds, and it matches the naming in our API)
(We could extend media controllers to point to a parent time source or we could just say that for now they always use the document time source.)
Shane: this is more or less what I wanted to do for media element synchronization. So: yes!
* Introduce a global instance of time source that has its zero time as the moment the document begins (this definition needs work since I want to revisit the autoplay behaviour)--and when you try to set it it throws a NoModificationAllowedError exception
* It's tempting to make Animation implement TimeSource but does that make sense to slave an Animation to another Animation?
* That's it. No custom time source or anything like that. Yet.
In the future, we can define additional types of time sources and they should plug into the model just fine. It could even be a separate spec (a module), not a revision of Web Animations.
Shane: this sounds like a great idea.
I wonder if you could achieve (b) by wrapping up the intersection of the time window and time space by adding an intermediate time source?
e.g. for use case 1
Par (start: 0, end: 3)
Par (start: 3, end: 5, id)
TimeSource (start: 0) <-- time-based
Anim (prop: height, dur:1s) <-- time-based
The point is, I think we can leave this out entirely and plug it in later. The main issue is probably naming. If we expect an animation in future to possibly have both a parent group and a time source then we might not want to use 'timeSource' to represent what later be just the 'parentGroup'.
But I think it's ok. We can use 'timeSource' for now and later either use the approach above of adding intermediate time sources or just add 'primaryTimeSource'? 'playTimeSource'?
Shane: In case I haven't already made it abundantly clear, I'm very happy to leave stuff out as long as we've thought of the long term implications of where we are going and how to get there. If we can capture the spirit of (b) by nesting TimeSources then sure, let's do it. I'd like to make sure we can before committing to this path though. It would require the time window from the parent group being passed through the time source, which seems OK as time sources need to provide time windows to their children anyway.
> Summary of discussion so far:
- Discussion centres around whether we should separate the concepts of group ownership and time source at the API.
-- one view is to make these separate concepts from the beginning and children of groups just happen to have derivations of the same parent by default in terms of time
-- the other view is to conflate the two concepts for the sake of simplicity--for children of groups your parent group is your parent time source because you can't specify otherwise
* I'm leaning towards Timeline since I do often hear people say things like, "I want to tie this animation to the video's timeline"
-- so perhaps it's not so weird to think of a parent timeline?
* Also, HTML5 has: "Each MediaController also has its own defined timeline."
* Then you'd have TimedItem.parentTimeline which makes a little more sense.
isn't bad either
Timeline also has the advantage that if there is a CSS property for this it's probably going to be animation-timeline and not animation-timesource.
3. INTEGRATION WITH MEDIA
* Subclassing groups from MediaController seems attractive but that doesn't fix the outstanding issues regarding pausing/reversing/seeking children of a repeating par group or a seq group.
> We are all currently leaning towards keeping the Media subsystem of HTML at arms length rather than trying to merge the two.
* Summarising MediaController:
-- similar to the idea of a "loose" par container we have previously raised
-- children can be paused independently
-- unpausing a child makes it catch up to the group time
-- seeking a child throws an InvalidStateException
-- setting playbackRate on child (closest thing to reversing) is ignored and parent playbackRate is used instead
-- the group itself can't be repeated or reversed
-- its playback rate can be set and governs all children
* Comparison to <svg>
-- <svg> children should be able to be seeked independently
-- <svg> children should be able to be paused independently without catchup behaviour
-- <svg> children should be able to have their playback rate adjusted independently
i.e. it doesn't map so well to <svg> despite what is indicated below
* The investigation into MediaController set out with the hope that it might provide a path to fixing the current complications involved in pausing/reversing/seeking descendent items of a repeating par group or a seq group but that no longer appears to be the case. We have to solve those issue some other way.
* It seems attractive to simply remote such operations on descendent elements to their parents (and so on up the chain) however this appears difficult for the case of seeking due to the fact that timing functions that can currently be applied to such groups are not always invertible
> On further consideration this may be more complex, less intuitive, and less powerful.
* Pausing can probably be remoted fairly easily but whether we do this or not depends on how see address seeking since we might want to be consistent there
* Current options for addressing seeking include:
- Restrict the timing functions allowed on such sync groups (i.e. basically don't allow timingFunction) so we can remote seek operations up the chain
- Allow a kind of local seek that gets reset when the current iteration of the parent changes
- Simply disallow seeking children of a sync group (by either throwing an exception or just ignoring the change)
- This is consistent with HTMLMediaElement
> Discussed the notion of having a fixed time window for each timed item. The window is established based on the 'default' timing parameters for the item. 'Live' changes to the item such as pausing and reversing do NOT affect the calculation of the time window. This provides fairly easy-to-reason-about behaviour for the cases when we have children in repeating groups.
> The document time sources establishes an infinite time window for its children.
> 'Live' state does NOT reset when the parent group enters another iteration. If you pause animation A during iteration 1 of the parent group, it will still be paused when you enter iteration 2.
Suggested outcome: needs further investigation
The long version:
Brian: I think we need to work on this ASAP since it will influence the architecture.
Shane: To be perfectly honest I think proper grouping is more important than integration with media (because proper grouping strongly influences what effects are possible within the animations specification, and is likely to be used more than integration with media). Maybe we can have both though!
Thinking out aloud:
Can we replace groups with media controllers and just add a 'seq' mode to media controllers? Which effectively calls play() at the appropriate times?
Shane: I don't like the idea of things which are scheduled to play not being represented as play()ing.
Brian: I think a more feasible route is to have groups subclass media controllers.
Then you can put stuff in a group either by nesting or by setting the mediaGroup attribute
For animation groups created using <seq>/<par> markup you'll need the concept of a default instance for working out what group to put video and audio into
Shane: would this be the group itself?
Brian: yes, I'm mixing up terms a bit.
You could restrict these groups to only contain groups/animations/media (i.e. not vanilla media controllers) but even that restriction may not be necessary
In that case it might be better to rename AnimationGroup to TimingGroup (MediaGroup? TimingController? SyncGroup?)
SyncGroup might more sense if we restrict the operations possible on descendent items as proposed below
But should media controllers own their animations? Presumably yes? So should we extend media controllers to include all the same methods as AnimationGroup for adding and removing children?
Shane: no because this would massively increase the difficulty of getting stuff accepted
Brian: agreed. probably we can get away without adding this?
The ideal outcome here would also allow us to address the outstanding issue of how calling pause() or reverse() or seeking etc. should work for animations that are in seq containers or repeating par containers. For pause(), options include making it:
pause the group
throw an exception
handle timeDrift specially such that the pause component resets when currentIteration changes
Shane: are we on the same page here with what should happen? I think pause should introduce a one-off delay that persists until play is called (i.e. it doesn't impact future iterations and it doesn't appear if you play() the animation or a parent again later). I think that delay should impact the endTime of the animation and therefore should delay other items in the sequence group.
Brian: If pause pauses the group that question only arises when you're tied to a media controller. Then you have to decide what happens if you seek backwards from that point. We could probably differentiate between seeking the parent vs seeking the animation in that case. Need to look into what is natural for media controllers in that situation. It's probably already defined there.
Shane: I think reverse should reverse the current animation and set its endTime to now + however long the animation has already played. This should also delay (or advance) other items in the sequence group.
Shane: Obviously throwing an exception or silently failing don't fit this :) Pausing the group does (I think) as does resetting timeDrift on currentIteration change.
Brian: That's what's in the spec. The difficulty is that in order to maintain the current position you need to do a seek and then you have to work out when and how to reset that seek so it doesn't show up in subsequent iterations, or, alternatively apply the seek to the parent, but so far that looks impossible unless we restrict the sorts of timing functions you can apply to groups.
Pausing the group seems attractive. It's NOT how MediaControllers work but it's not entirely different either. Pausing a media controller pauses the children but the reverse is not true (however, if a child gets blocked, the controller does too). So we could say these groups are a special type of strict MediaController.
Adopting the architecture outlined in item 2 above means most animations would not be subject to these constraints. It would only be when you specifically created a group that you'd encounter this.
Likewise for par and seq, setting the current time of a child would seek the parent group?
Is this even possible? Can we do a reverse translation from child active time to parent iteration time? I don't think we can.
Shane: Why not? (I'm not saying we can, just that it isn't obvious that we can't).
Brian: It's actually the transformation from parent iteration time back to parent active time (which is where seeking takes place) that's not reversible since you can have timing functions that flatten time or (in future) are not monotonically increasing.
One option would be to restrict the sorts of timing functions that can be applied to groups (i.e. no timing functions). Then you could probably do the reverse conversion and apply the seek to the parent. But applying a timing function to a group seems really useful. We used it in the demo, for example.
Likewise, should reversing a child reverse the parent. Not sure it should. Can we do something sensible here? Or just throw an exception?
Under this arrangment of having par/seq/mediacontroller, and presuming animations can be associated with vanilla HTML5 MediaControllers, it feels a little bit like these regular MediaControllers would end up become equivalent to the "loose" par groups we floated back in March/April but with the critical difference that they can't be repeated so we don't have to worry about how pausing works in that case.
In most cases vanilla media controllers are what you want. You can't repeat them, apply easing functions, reverse them etc. but you can seek them and you can seek the children independently.
Then you have two strict sequencing containers par and seq which let you scale, seek, reverse etc. but put some restrictions on what you can do with the children in terms of independently controlling them.
Is the distinction between MediaController and ParGroup too subtle for authors?
Perhaps making ParGroup and SyncGroup a subclass of SyncGroup makes the distinction clearer?
An <svg> element would map to a vanilla media controller (since you can pause the children of an <svg> element independently but you can't reverse/repeat an <svg> element). So for the element syntax we mightn't even need a third type of element, just <svg>, <par>, <seq>.
Given the refactoring outlined in point 7 below we can easily remove reversing etc. from media controller.
Shane: this is quite hard to follow. I *think* you're proposing that <par> and <seq> hierarchies remote *all* pause/play/seek/reverse commands up to the root group, and maintain a *strict* relationship internally. If that's what you're saying .. I love it. It should dramatically simplify a lot of the logic of dealing with drifts in current time and with reversing etc.
Brian: Sorry about the hard to follow bit. It's very much stream-of-consciousness because I thought I worked something out then I realised it had problems. I don't think we can remote all pause/play/seek/reverse to the parent unless we restrict the sorts of timing functions that can be used on the parent. I think I need to investigate whether we can do something sensible for reverse and seeking. It might look like this:
if parent is a sync group (i.e. par or seq) pause the parent -- or actually have the parent become effectively paused because one of its children is
if parent is a sync group, do reverse as usual and then perform a local compensatory seek to maintain the current position -- this kind of local seek gets reset when the current iteration of the parent changes (and maybe some other situations regarding fill modes)
again, make it a local seek??
I need to write out a proper proposal for this and see if it works.
Shane: Taking the third group (<svg> / MediaController / loose par group) and Time Sources I think we could potentially even go further. Reversing and pausing are very basic examples of Time Source transformers. I need to think about this more, and I'd really like to see a much cleaner explanation.
Brian: In terms of aligning better with HTMLMediaElement some possible adjustments:
Adjust autoplay behaviour to match
I haven't looked into the specifics of this but I think this would be highly desirable
Add a read-only boolean ended attribute (marks when animation has finished). Note this is stative but we seem to be moving towards a model where the methods on TimedItem itself are largely stative whilst the Timing structure is stateless. Its definition would basically be in terms of whether the previous sample was greater than or equal to (endpoint-exclusive timing) the endTime.
In HTMLMediaElement, changes to playbackRate preserve the play position in the same way our changePlaybackRate does, i.e. it's stative.
I wonder if we can have some arrangement whereby we can match that behaviour and also provide a stateless version too.
The most obvious is to have Anim.playbackRate adjust the rate whilst preserving position and Anim.timing.playbackRate simply adjust the model in a stateless way (causing a jump). Obviously that's horrible as describe in the next section where we'll consider alternatives.
Note that HTMLMediaElement does have defaultPlaybackRate which is a little bit like a stateless version. It's different--basically it takes effect, for example, when the user stops fast-forwarding--but it feels similar in some sense.
Also, in HTMLMediaElement, playbackRate can be set to zero and it will preserve the current position.
We can't easily do this in our current model. I've yet to come up a good way of achieving it without either introducing magic or touching other values like iterationStart.
Next meeting: Mon Dec 10, 17:30 PST / Tues 11 Dec 12:30 AEDST / Tues 11 Dec 10:30 JST @ ???