I think it’s unlikely the GPU is the problem. I mean, the AGAL you are using is no more complex than the AGAL that is used to draw pixels normally, it might even be simpler, and so should not tax any GPU able to run AGAL 1 code.
It’s more likely to be the overhead of the FragmentFilter. Worst case it is rendering everything in your scene into a texture, then drawing that texture using the AGAL code in the filter. So everything gets drawn twice, once into a texture which may be slower.
If nothing changes frame to frame you could use
cache() to stop it redrawing for no reason. But if anything changes, in the scene or the filter (such as AGAL constants) it has to redraw everything.
The bigger it is the worse the performance will be, so if there’s anything that can be excluded, done differently, or update less often, to reduce the number of pixels changing every frame it should help.
A far more involved fix would be draw everything at a lower resolution then scale it up to produce the final result. This is quite a common approach, used even in expensive game consoles, but is not supported directly in Starling so you would have to jury-rig it yourself. And I am not sure how much it would help.