[SOLVED] AGAL Compressed Texture GPU Sampler Error

hardcoremore

Hi,

I have following GPU error:

Error: Error #3763: Sampler 0 binds a texture that that does not match the read mode specified in AGAL. Reading compressed or single/dual channel textures must be explicitly declared.
at flash.display3D::Context3D/drawTriangles()
at starling.display.materials::StandardMaterial/drawTriangles()[C:\AS3Libraries\starling-graphics-extension\extension2.0\src\starling\display\materials\StandardMaterial.as:189]
at starling.display.graphics::GraphicRenderer/render()[C:\AS3Libraries\starling-graphics-extension\extension2.0\src\starling\display\graphics\GraphicRenderer.as:545]
at starling.display:😃isplayObjectContainer/render()[C:\AS3Libraries\starling\starling\src\starling\display\DisplayObjectContainer.as:406]

This is AGAL code that is used:

var format:String = texture.format;
var formatFlag:String;

switch (format)
{
	case Context3DTextureFormat.COMPRESSED:
		formatFlag = "dxt1"; break;
	case Context3DTextureFormat.COMPRESSED_ALPHA:
		formatFlag = "dxt5"; break;
	default:
		formatFlag = "rgba";
}

var agal:String = 
	"tex ft1, v1, fs0 <2d, repeat, linear, "+formatFlag+"> \n" +
	"mul ft2, v0, fc0 \n" +
	"mul oc, ft1, ft2";

compileAGAL(Context3DProgramType.FRAGMENT, agal);

What is interesting about this error is that it only occurs when I try to use different texture types with the AGAL above at runtime. I am passing 2 types of textures to this AGAL. One is Texture Atlas entire texture (which is always ATF ETC2) and another is POT Render Texture.

If I am using only one type of texture either Atlas texture or POT Render Texture with this AGAL everything is working fine no matter how many times or even if I use different textures ( different Texture atlases ) but as long as I use same texture type as first time used.

So If I started using that AGAL with Texture atlas texture everything is fine but as soon as I try to use POT Texture after using Atlas texture error is thrown. Or if I am using only POT texture also everything is fine but as soon as I try to use Atlas texture after using POT texture error is thrown.

This is the rendering function. It is from starling graphics extension:

public function drawTriangles(context:Context3D,
							  matrix:Matrix3D,
							  vertexBuffer:VertexBuffer3D,
							  indexBuffer:IndexBuffer3D,
							  alpha:Number = 1,
							  numTriangles:int = -1):void
{
	context.setVertexBufferAt(0, vertexBuffer, 0, Context3DVertexBufferFormat.FLOAT_3);
	context.setVertexBufferAt(1, vertexBuffer, 3, Context3DVertexBufferFormat.FLOAT_4);
	context.setVertexBufferAt(2, vertexBuffer, 7, Context3DVertexBufferFormat.FLOAT_2);
	
	if(!program)
	{				
		if(_materialTexture)
		{
			_fragmentShader.updateTextureAgal(_materialTexture);
		}

		program = Program3DCache.getProgram3D(context, _vertexShader, _fragmentShader);
	}

	context.setProgram(program);

	/**
	 * 
	 * FOR TEXTURE FILLS
	 */   
	if(_materialTexture)
	{
		RenderUtil.setSamplerStateAt(0, false, TextureSmoothing.TRILINEAR, true);
		context.setTextureAt(0, _materialTexture.base);
	}

	context.setProgramConstantsFromMatrix(Context3DProgramType.VERTEX, 0, matrix, true);
	_vertexShader.setConstants(context, 4);
	
	// Multiply display obect's alpha by material alpha.
	finalAlpha = _alpha * alpha;

	_colorVector[0] = ((_color >> 16) / 255) * finalAlpha;
	_colorVector[1] = (((_color & 0x00FF00) >> 8) / 255) * finalAlpha;
	_colorVector[2] = ((_color & 0x0000FF) / 255) * finalAlpha;
	_colorVector[3] = finalAlpha;

	context.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT, 0, _colorVector);

	_fragmentShader.setConstants(context, 1);

	context.drawTriangles(indexBuffer, 0, numTriangles);
}

This error occurs when this line is executed from render function above:

context.drawTriangles(indexBuffer, 0, numTriangles);

JohnBlackburne

Pretty sure you are looking in the right place - the read mode in the error I can only think refers to the tex AGAL instruction, and it's there that you are changing what you are reading.

But it's not something I've tried doing myself, I've mostly depended on Starling for drawing textured quads. I notice your AGAL code is the same as Starling's so should work the same way, so the root cause probably lies elsewhere.

I wonder if it's a batching issue? Again in Starling it uses the same AGAL program, same constants for a single batch. When it starts a new batch they can be changed. Changing texture in particular forces it to start a new batch, which then can use AGAL that matches it.

I am not familiar with SGE or how it renders things, but you might need to look at how it ensures that it changes AGAL when the texture changes.

hardcoremore

JohnBlackburne but you might need to look at how it ensures that it changes AGAL when the texture changes

This might be the very reason. Can you tell me more about why and when and how AGAL needs to change if the texture type change?

Because I understood that this switch makes sure that correct AGAL is used for certain texture:

switch (format)
{
	case Context3DTextureFormat.COMPRESSED:
		formatFlag = "dxt1"; break;
	case Context3DTextureFormat.COMPRESSED_ALPHA:
		formatFlag = "dxt5"; break;
	default:
		formatFlag = "rgba";
}

Why that switch is not enough in this case.

hardcoremore

I HAVE SOLVED THIS! 🙂

The problem was that this AGAL I was using was cached in static variable like this:

public static var TEXTURE_FRAGMENT_SHADER:IShader = new TextureVertexColorFragmentShader();

But now I have created 3 more types for each texture type that is used. So now my AGAL code looks like this:

public class TextureVertexColorFragmentShader extends AbstractShader
{
	private var textureTypeFormat:String;
	
	public function TextureVertexColorFragmentShader(textureType:String = null)
	{
		textureTypeFormat = textureType;
	}
	
	override public function updateTextureAgal(texture:Texture):void
	{
		var formatFlag:String;
		
		switch (textureTypeFormat)
		{
			case Context3DTextureFormat.COMPRESSED:
				formatFlag = "dxt1"; break;
			case Context3DTextureFormat.COMPRESSED_ALPHA:
				formatFlag = "dxt5"; break;
			default:
				formatFlag = "rgba";
		}

		var agal:String = 
			"tex ft1, v1, fs0 <2d, repeat, linear, "+formatFlag+"> \n" +
			"mul ft2, v0, fc0 \n" +
			"mul oc, ft1, ft2";
		
		compileAGAL(Context3DProgramType.FRAGMENT, agal);		
	}
}

So now I am injecting texture type into constructor and compiling AGAL based on that texture type. So I have now 3 cached shaders instead of one like this:

public static var TEXTURE_FRAGMENT_SHADER_COMPRESSED:IShader = new TextureVertexColorFragmentShader(Context3DTextureFormat.COMPRESSED);
public static var TEXTURE_FRAGMENT_SHADER_COMPRESSED_ALPHA:IShader = new TextureVertexColorFragmentShader(Context3DTextureFormat.COMPRESSED_ALPHA);
public static var TEXTURE_FRAGMENT_SHADER_COLOR:IShader = new TextureVertexColorFragmentShader();

This is awesome, thanks for the help John, one more edge case solved for the starling graphics extension 🙂 . You can add [SOLVED] to the title of this thread 🙂