rendering issues w/ custom DisplayObject

Nate

Hi guys, I'm working on the official Starling runtime for Spine, which does 2D skeletal animation:
http://esotericsoftware.com/

I have it working except for two issues:

1) It renders the spineboy animation as desired:

...but it doesn't animate. I see render is being called each frame and the vertices of the images I'm drawing change over time, but what is drawn doesn't actually change.

2) In my getBounds method, if I return a rectangle that is 0,0 0x0 it renders fine. If I return the actual bounds, the rendering is squished. Why does this happen? Why does getBounds affect rendering?

If you'd like to take a look, you'll can import these 3 projects into Flash Builder:
spine-as3
spine-starling and spine-starling-example
You can download the projects with git or grab the whole thing as a zip with this link:
https://github.com/EsotericSoftware/spine-runtimes/archive/master.zip

My custom DisplayObject is here:
https://github.com/EsotericSoftware/spine-runtimes/blob/master/spine-starling/spine-starling/src/spine/SkeletonSprite.as
Again, the vertices are changing, but the rendering isn't. I'm perplexed! Thanks for any help! 🙂

noizz

Sorry for a bit of derail - is there a reason you used a custom object? Native Image/MovieClip object seem to be a bit more "friendly" to work with.

Anyway - I am glad you are working on AS3, looking forward to some examples too 🙂

Nate

I wanted to do the most low level, efficient solution. A Spine animation is made up of many images, I don't think there is a simpler way to draw. Also, other Starling classes won't know how to compute the bounds. I need to draw and compute bounds no matter what. Please let me know if I'm going off in the wrong direction though!

Ninjadan

I was able to get it working by calling onVertexDataChanged on the SkeletonImage after you have manipulated the vertexData in the render call of SkeletonSprite.

Spine is looking fantastic and im really excited to start working with it.

Ninjadan

Something else you may want to look at doing to make it play nicer with starling is to make SkeletonAnimationSprite implement IAnimatable and instead of using an internal enter frame loop you can use the advanceTime( deltaTime:Number) function. You will then need to add the instance of SkeletonAnimationSprite to a starling juggler. You could use Starling.juggler.add() in your game class for the example.

Ninjadan

I have spent a little while looking at your getBounds problem, the distortion only occurs if you directly manipulate the height and width properties, such as you are doing in the example Game class. Im not really familiar enough with starling or spine to understand what you are attempting to do here or why it then distorts (im guessing you want to resize the object?) but if you don't set these values then the getBounds appears to work and there is no distortion. You can still manipulate the size of your sprite by using the scaleX and scaleY properties and modifying these values is reflected in the getBounds.

Anyway I hope I have been of some help. I have been playing with spine in starling all morning and the possibilities this opens up with starling games is amazing. Thanks!

Nate

Thanks Ninjadan! That does help a lot! 🙂 onVertexDataChanged fixed my rendering. I'll use IAnimatable, thanks for the tip. 🙂

I understand the getBounds issue better. I guess when I return a size, Starling uses that for the unscaled size. Then if I set the size of my skeleton animation explicitly, Starling scales it to that size from the size returned from getBounds. The thing is, my skeleton animation size varies constantly as the images that make up the skeleton move around. Is the right thing to do to return 0,0 0x0 from getBounds? Does that have any drawbacks?

Edit: I see hitTest uses getBounds, so I wrote my own hitTest. Any other drawbacks to getBounds returning 0,0 0x0?

Edit2: Example running here (click him to jump):
http://esotericsoftware.com/spine/files/starling/spineboy/

en_en_tan

what a nice example you have Nate! Glad to see spine 2D boy animated here 😃

I will be waiting for another example and of course a tutorial 😃

Daniel

Wow, Nate, it's awesome to see you're working on a Starling runtime!!

The "getBounds" method is used for several things in Starling; the default "hitTest" implementation relies on it (you found that out already), and also the "width" and "height" properties. So if you return an empty rectangle, those methods won't work any more.

But maybe we can make your life easier on the whole class. It depends on how the elements of the objects can be transformed. Are those elements just rectangular images, i.e.: do you need to move around the 4 vertices of each image without any limitations?

Because if that's not the case, and those are just images that are scaled and transformed, you could use the "QuadBatch" class for your rendering.

That would work like this: extend the "DisplayObjectContainer" class instead of "DisplayObject". Its only child will be a "QuadBatch" object. The quad batch contains all the images of your character, and is "reset" and refilled whenever the vertices change.

"getBounds" and the hit test would just work by default; however, if you want them to be more exact, you can still override the hitTest method.

Alternatively, you can still subclass "DisplayObject" and use the QuadBatch just for rendering. It contains a "renderCustom" method that you can call in the object's render method. You'd forward the hit test to that object, as well.

The advantage of this approach would be that people would not be able to access the quadbatch. (But if that's worth the effort, is another question.)

In any case, such an approach might simplify rendering for you. However, I don't know how much control you need over each vertex ... so I might also be mistaken.

Daniel

BTW, another advantage of using a quad batch would be that the next Starling version will probably expose a setting allowing people to decide if those objects should be batched. For small characters, this might make sense, and would speed up rendering.

Nate

Thanks for the info, Daniel!

The regions I'm drawing have 4 vertices but are quadrilaterals, not necessarily rectangles, so I need to specify the vertices rather than SRT. Subclassing image to expose the vertices seems to work well, though onVertexDataChanged was a gotcha (I should RTFM better :p). I'm currently using RenderSupport batchQuad, which seems to use a QuadBatch under the covers. The vertices almost always change every frame, since the animation is interpolated.

It seems ok if getBounds 0,0 0x0 makes me implement my own hitTest and that width/height don't work. The width/height don't make much sense for a skeleton, whose size is determined by bone/image positions. Implementing hitTest isn't too bad and in the future I might do hit detection with the individual images, or better with bounding boxes when those are implemented.

Given the above, do you think my implementation is ok? The file with most of the code in question is here:
https://github.com/EsotericSoftware/spine-runtimes/blob/master/spine-starling/spine-starling/src/spine/SkeletonSprite.as
I went with the raw Image vertices because I'm a sucker for premature optimization. 😉 In my defense this is library code and I want games to be able to use as many animations as they want with minimal overhead. 😃

Daniel

I just had a look at the implementation, and the way you're doing it is just fine. You're calling the "support.batchQuad" method, and that's the best thing you can do performance-wise! =)

So that looks perfect to me, using the QuadBatch class directly wouldn't have made that more efficient.

One small thing: even if the bounds are always empty, you could at least make sure the "x" and "y" coordinates of the bounds are correct. I'm doing the same in the particle system, because that has the exact same problem.

Here's the code you could use instead:

private static var sHelperPoint:Point = new Point(); // to avoid object allocations

/** Returns an empty rectangle at the particle system's position. Calculating the
 * actual bounds would be too expensive. */
public override function getBounds(targetSpace:DisplayObject,
                                   resultRect:Rectangle=null):Rectangle
{
    if (resultRect == null) resultRect = new Rectangle();

    getTransformationMatrix(targetSpace, sHelperMatrix);
    MatrixUtil.transformCoords(sHelperMatrix, 0, 0, sHelperPoint);

    resultRect.setTo(sHelperPoint.x, sHelperPoint.y, 0, 0);
    return resultRect;
}

That's all I could find in my nit-picking attempts! 😉
Good work!! =)

Nate

Awesome, thanks for taking a look! 🙂 I'll incorporate that getBounds code. 🙂

noizz

I was able to compile it with FlashBuilder 4.7 and AIR 3.6 (not sure on details), but on Intellij Idea with AIR 3.7/ASC2.0 I end up with compiler sending the following error:

A:\AS3\SpineTests\Project\src\spine\Skeleton.as
[SplineGame (module SpineTests)]: Implicit coercion of a value of type Bone
to an unrelated type SkeletonData.

It is regarding this function's return value:

public function get rootBone () : SkeletonData {
	if (_bones.length == 0)
		return null;
	return _bones[0];
}

Any help?

EDIT: Working after the fix, thanks!

Nate

Fixed in git, thanks!

Jeff

Nate,

Wanted to post this comment on this topic since it relates to some suggestions Daniel mentioned above about possibly using QuadBatches to render a SkeletonSprite's content (even if they weren't directly in the display list).

However, I was motivated by the requests on this other topic to possibly separate the need for every render() call to also calculate and create the rendered Quads/QuadBatches:
http://forum.starling-framework.org/topic/spine-performance-issue

I played with this a bit over the holiday break, and I put together a variation of this idea on a forked version of spine-runtimes:
https://github.com/jamikado/spine-runtimes

Specifically the changes only occur in one class, SkeletonSprite, and in this one commit:
https://github.com/jamikado/spine-runtimes/commit/e88637e62e05ed29213eb61450da92f50f76517d

Not sure if you would be interested in checking it out as a suggestion, but I'll outline the potential benefits here in case others are interested. Maybe there are issues with this modification that needs further study since I'm relatively new to Spine's runtime system, but it seems to work fine with my tests.

First, there is one unrelated optimization change I made in SkeletonSprite which was to remove the instance variable vertices:Vector.<Number> and make it just a reusable static helper since you don't need that duplicated on every SkeletonSprite instance since you only use it to locally grab data within a single method call like the other temp static helpers.

Basically, the changed code moves the actual creation of quads generated by SkeletonSprite out from the render() call and into a cached set of QuadBatches (more than one, since as you know there is a potential for a state change from one slot rendering to the next) that will only be updated when advanceTime is called.

While it is true that in most cases, SkeletonAnimations will be updating *and* rendering on every enterFrame, this change allows for an improvement in the following cases:

1) Starling.stop(false) when wanting to pause the entire Starling display list but still need rendering to occur, such as when temporarily pausing a game level.

2) Whenever you have any SkeletonAnimation that is temporarily (or permanently) removed from any active juggler. Rendering is still getting called with the created QuadBatches, but if no advanceTime is being done, you avoid the need to recompute and create the quads on each and every render() call, instead you just send the cached QuadBatches to render.

All the code that creates and updates this internal QuadBatch array is in direct relation to the same way RenderSupport itself builds up and reuses QuadBatch instances, so the code is well tested and stable in its technique.

There should be no extra increase in draw calls compared to the older method (I added a batchable property on SkeletonSprite just for kicks and giggles, and it defaults to true), and the QuadBatch instances are reused and trimmed in only the rarest of cases, so there should also not be any additional impact on temporary object creation.

This doesn't really improve the rendering times compared to the older technique, but it should allow for performance improvements with paused SkeletonAnimations (either all or some at a time) if this is a common practice in a given app scenario.

I could see this being useful when there might be lots of SkeletonAnimations in a scene but only a few go active in animation when interacted with, while the rest just reuse their QuadBatches when advanceTime does not get triggered.

I don't know if in the future there will ever be some API that can inform whether after calling update(delta) no actual change in the state also occurred so one could possibly even avoid updating the QuadBatches in this scenario too, but for now, if advanceTime gets called, the QuadBatches get rebuilt...

Nate

Good point that vertices can be static.

It is expected that the skeleton will change every frame, since the point of using Spine is to have things animated. Typically an idle animation is used, even for something like a tree or other background object, so things feel move alive. A pause screen would benefit from optimizations that avoid work if the skeleton isn't updated, but of course an app needs to be able to run at the desired frame rate while unpaused. There are probably some situations where a Spine skeleton isn't animated, but I have a feeling it is pretty uncommon.

Jeff

Ok, but you seem to have agreed to use Starling's starling.animation.IAnimatable interface to control the animated state of the skeletons.

Like all other IAnimatable uses, Starling users will pause Starling or take things into and out of active Jugglers to have control on their calls to advanceTime.

Forcing new calculations on every render(), independent of whether an advanceTime was even called since the last render(), seems a bit wasteful in my opinion, but I'm just throwing it out there as a suggestion, thanks for taking the time to read it over.

Nate

True it is wasteful if the skeleton is not modified. The question is if it is worth optimizing. I'm just not sure it is, but will definitely keep it in mind.

There is a sort of similar situation with applying animations. The animation is stateless and doesn't make any assumptions about the skeleton or the last key frame applied. Each frame and for each timeline in the animation it does a binary search to determine the next key frame after the current animation time. This hasn't been an issue so far but is another thing to keep an eye on.

rares

hey guys!

Sry to hijack this thread but my issue is somehow related.
I am trying to determine the width and height of my animation (max width/height) and getBounds as you already mentioned returns a lovely 0.

Do you have any clue on how i can get the with/height?
Thanks!