Use mesh as mask

EZ

Hi,

I'm building a game inventory, which should behave something like this (see gif):

The user clicks on an item on the screen
The item is tweened to the correct slot in the inventory

Currently I'm using a quad mask to mask the items in the inventory (the black squares).
The problem, as you can see in the attached animated-gif, is that items in the far left and right side of the inventory get clipped, naturally, by the quad mask.

So I've created a mesh mask (green area, red border), in this shape, thinking that I'll be able to mask the inventory using it:

As it turns out, it doesn't work.
I guess that the mask is using the bounding box...

I know there's a stencil mask, but I suppose it has a larger impact on performance, am I right?

Any ideas?

Daniel

Hm, this should actually work! So you've created a Mesh with custom vertex- and index data in the green form above, and the mask doesn't conform to that?

When you render that mesh directly (add it to the stage), it shows up correctly, doesn't it? Maybe you could post the code that's creating the mask? Then I'll check it out.

BTW, a simpler alternative would be to use the "Canvas" class to draw that shape. Either by drawing two rectangles, or by creating a "Polygon" with the object's outline, and adding that to the canvas.

Nevertheless: it should work with a mesh, too.

EZ

Yes - the green form above is what it looks when I directly add it to the stage (without the red border - that was added using Photoshop).
Here is another snapshot, with the vertex indexes (Photoshop):

Another issue I've encountered was problems with touch events, when the mask was created with negative values, for example:

var mask_quad:Quad = new Quad(-128, -128);

I guess it might be a misuse, but it might point to some other bugs...

As for the "Canvas" option:
What would be the best approach, in terms of performance hit:
Mesh?
Canvas?
Stencil mask?

The code for creating the mesh mask, using 4 quads (my stage size is 1024x768):

private function buildMask():Mesh
{
var vertex_arr:Array = [0, 0, 256, 0, 0, 384, 256, 384, 704, 0, 960, 0, 704, 384, 960, 384, 256, 448, 704, 448];
//var color_arr:Array = [];

var vertexData:VertexData = new VertexData("position:float2, color:bytes4");
var i:int = 0;
for (i = 0; i < 10; i++) {
vertexData.setPoint(i, "position", vertex_arr[2 * i], vertex_arr[2 * i + 1]);
vertexData.setColor(i, "color", 0x00FF00);
}

var indexData:IndexData = new IndexData();
indexData.addQuad(0, 1, 2, 3);
indexData.addQuad(1, 4, 3, 6);
indexData.addQuad(4, 5, 6, 7);
indexData.addQuad(3, 6, 8, 9);

return new Mesh(vertexData, indexData);
}

And for adding it to the container:

var mask_mesh:Mesh = buildMask();
//this.addChild(mask_mesh);
mask_mesh.x = 32;
mask_mesh.y = -384;
//mask_mesh.alpha = 0.5;
_inventory_sprt.mask = mask_mesh;

I'd be happy to share the entire code, if needed.

Daniel

Hm, I just used that exact code as a mask, and it worked fine for me!

Ah, but I think I've got an idea about what the problem is. This is probably an AIR application, right? Please check if you have activated the following setting in the application XML:

<depthAndStencil>true</depthAndStencil>

The reason why you need this: when you were using just a quad, Starling could use the scissor rectangle for masking, which is fastest. When you switch to Mesh or Canvas (and you have to do that, there's no other way), Starling will automatically switch to the stencil buffer.

As for the performance: the Canvas class will internally create a Mesh, too, so it will be just as fast as your custom Mesh! It might just be a little less code to create the mask. But you already did that, anyway, so you can safely stick with your implementation. 🙂

I hope that helps!

EZ

Yes - that works!
Indeed this is an AIR project and indeed - this setting was set to false!
I don't remember seeing this setting mentioned anywhere in the wiki/docs...

Thanks for the quick and accurate response 🙂

Daniel

Ah, that's great news! 🙂

I don't remember seeing this setting mentioned anywhere in the wiki/docs...

Damn, I was sure it's mentioned somewhere, but I can't find it either, right now. 😉
So I just added it right to the "mask" property: commit 292e755.

Anyway: take a look at the trace output of your game. Right at the beginning, Starling should have logged a warning that it's lacking this setting. I agree, though, that it's easy to overlook. 😉

joshtynjala

Hey Daniel, do you think you could wait until a mask is set before you display the warning? If you're not using masks at first, it's easy to ignore the warning as noise. Then, when a mask fails to work, the warning doesn't have the same impact because people get used to seeing it all the time.

Daniel

That's an excellent idea, Josh! You're right, that way it's easier to see.
I just made that change.

joshtynjala

Perfect! Thanks, Daniel.