Flash RTS-like Benchmark

UnderwaterBird

Hi guys,

as I'm planning to create an RTS game, I was searching for a modern benchmark to compare on how other technologies stack up with AS3 and if JS was still slower in a simple real-life code, as some benchmarks pointed that it was twice as slow and some, more recent, that up to 5 times faster than AS3.

So I've created my simple, not deeply optimized code of units collisions and ported it to Flash/AS3, Typescript (NodeJs + Vite) and most recent .NET.
Yes, there's still a place to optimize, but it's not a real game code yet, just to compare relative performance.
I've used Vector in flash as it's the fastest in Flash.

        unitCount = 2000;

        var i:int = this.unitCount;
        while (i > 0) {
            --i;
            var unit1:Unit = this.units[i];

            var j:int = this.unitCount;
            while (j > 0) {
                --j;
                var unit2:Unit = this.units[j];

                if (unit1 === unit2) continue;

                var distance:Number = Math.abs(this.distance(unit1.x, unit1.y, unit2.x, unit2.y));

                var maxRadius:Number = Math.max(unit1.radius, unit2.radius);

                if (distance < maxRadius)
                    unit1.moveAwayFrom(unit2);
            }
        }

private function distance(x1:Number, y1:Number, x2:Number, y2:Number):Number {
        return Math.sqrt((x2 - x1) * (x2 - x1) + (y2 - y1) * (y2 - y1));
}

class Unit {
    internal var x:Number = Math.random() * 800;
    internal var y:Number = Math.random() * 600;
    internal var radius:Number = 2 + Math.random() * 3;

    public function moveAwayFrom(unit:Unit):void {
        var speedX:Number = (unit.x - this.x) * .1;
        var speedY:Number = (unit.x - this.x) * .1;

        this.x -= speedX;
        this.y -= speedY;

        unit.x += speedX;
        unit.y += speedY;
    }
}

And I was a bit shocked about the results:

So we have 2000 * 2000 = 4 000 000 of collisions (with square root) and some other calculations.

For smooth 60 FPS we need no more than 0.01667 sec. per frame

Flash with Vectors: from 0.381sec to 0.649sec. (launched in Animate and in IDEA)
Typescript with Arrays: 0.03sec. with Arrays (launched in Chrome via NodeJs and Vite)
Typescript with hash Set: approx. 0.01 sec.
.NET with Arrays: 0.022sec - 0.038sec depending on x32 (faster) x64 (slower) and standalone app build
.NET with hash set was up to 50% faster than TS with hash set, I did not save the numbers, though
Unity still not tested, I'm pretty sure with Burst Compiler it will win.

Tested on Ryzen 5 3600 CPU and Windows 11

So JS looks at least 10 times faster than AS3, and in a worst scenario up to 65 times?

What are your conclusions? Perhaps, my tests have a huge flaw I do not see, or flash is indeed not a performant option at all comparing to other technologies?

elegantdeer

Maybe there is a way to replace those Math.random() Math.abs() Math.sqrt() Math.max() functions, but I don't think that it will give enough performance boost, so, yes, ActionScript virtual machine has not been updated for ages. Thanks for sharing 👍

bwhiting

As the deer mentioned it is probably the function calls that are the as3 bottleneck. I would inline them with the compiler flag and replace Math.max and Math.abs with simple if statements where possible.

Js engines are good at automatically optimising and inlining code for you compared to the as3 compiler. So to get a fairer comparison I would inline the function calls and see how it fairs.

bwhiting

Also as a side note, you can use squared distance to compare distances without the need for the expensive square root, just remember to square the other side of the equation.

(You probably knew this but I thought would mention just in case)

pol2095

I think on iOS or Android, as3 can be faster, it use ahead-of-time compilation from ActionScript to machine code.

FlashFix

Did you use the newest Harman Air SDK ?

UnderwaterBird

FlashFix
No, I've used 50.2.4.3 in IDEA (non-debug run in IDEA, non-debuggable swf) and Animate 2024 + Flash Player 32 target (non-debug as well).

With latest beta 51.0.0.3 I have 0.585sec. - 0.623 non-debug run with non-debuggable swf while running in IDEA, 0.113 - 0.135sec. when running exported exe from IDEA. My bad not to try exe the first time, but still 4-13 times slower than in Typescript.

UnderwaterBird

Unity 2022.3.20 is 0.026sec. with Arrays (EXE 64-bit, IL2CPP, .Net Standard 2.1, Faster runtime, Master)

Unity Burst compiler is 0.001sec. with NativeArray (same executable), though I'm a bit skeptical here as the result seems too good, but I can't figure out why.

pol2095

You can try flascc ?

UnderwaterBird

pol2095
I'm afraid I have not enough experience to try it.

I've benchmarked all this to make my own decision on what popular technology is suitable for a small 2D RTS with a lot of units and my result is that both Unity (which was obvious) and TypeScript (which was surprising) are good enough for it.

I hope my benchmarks will help other developers to make a decision what tool to use in case they need performance.

As of me, I'm preferring TypeScript, as while performance is quite similar, it has many quality-of-life features that Unity lacks for 2D (like decent speed in project launching, instant live reload, way easier syntax in tools (PixiJS, Spine plugins, etc.), TS is based on ES same as AS3, etc. ).

pol2095

If you share a complete as3 code sample, I can test it.

UnderwaterBird

pol2095
It's the same as in first message, just with launch code:

package
{
    import flash.display.Sprite;
    import flash.text.TextField;

    public class Main extends Sprite
    {
        public function Main()
        {
            var txtResult:TextField = new TextField();
            txtResult.width = 400;
            txtResult.defaultTextFormat.size = 30;
            addChild(txtResult);
            new RtsBench(txtResult);
        }
    }
}

package
{
    import flash.text.TextField;

    public class RtsBench
    {
        var unitCount:Number = 2000;
        var units:Vector.<Unit> = new Vector.<Unit>();

        public function RtsBench(txtResult:TextField)
        {
            this.createUnits();

            var nowTs:Number = new Date().getTime();

            this.doStuff();

            var nowTs2:Number = new Date().getTime();

            var timeDifferenceTs:Number = nowTs2 - nowTs;

            var timeDifferenceSec:Number = timeDifferenceTs / 1000;
            var timeMaxSec:Number = 1 / 60;

            var resultMessage:String = "AS3 time passed: " + (timeDifferenceSec) + "sec.\n"
                    + "time max for 60 FPS: " + (timeMaxSec) + "sec.\n"
                    + "\ntest with " + this.unitCount + " units " + (timeDifferenceSec < timeMaxSec ? "PASSED" : "FAILED");

            if (txtResult) txtResult.text = resultMessage;
            else trace(resultMessage);
        }

        private function createUnits():void
        {
            var i:int = this.unitCount;
            while (i > 0)
            {
                var unit:Unit = new Unit();
                this.units.push(unit);
                --i;
            }
        }

        private function doStuff():void
        {
            unitCount = 2000;

            var i:int = this.unitCount;
            while (i > 0)
            {
                --i;
                var unit1:Unit = this.units[i];

                var j:int = this.unitCount;
                while (j > 0)
                {
                    --j;
                    var unit2:Unit = this.units[j];

                    if (unit1 === unit2) continue;

                    var distance:Number = Math.abs(this.distance(unit1.x, unit1.y, unit2.x, unit2.y));

                    var maxRadius:Number = Math.max(unit1.radius, unit2.radius);

                    if (distance < maxRadius)
                        unit1.moveAwayFrom(unit2);
                }
            }
        }

        private function distance(x1:Number, y1:Number, x2:Number, y2:Number):Number
        {
            return Math.sqrt((x2 - x1) * (x2 - x1) + (y2 - y1) * (y2 - y1));
        }
    }
}

class Unit
{
    internal var x:Number = Math.random() * 800;
    internal var y:Number = Math.random() * 600;
    internal var radius:Number = 2 + Math.random() * 3;

    public function moveAwayFrom(unit:Unit):void
    {
        var speedX:Number = (unit.x - this.x) * .1;
        var speedY:Number = (unit.x - this.x) * .1;

        this.x -= speedX;
        this.y -= speedY;

        unit.x += speedX;
        unit.y += speedY;
    }
}

HaimSh

This is rather pointless benchmark, what you would want to test are RTS possible bottlenecks.
First is the cost of rendering many units, buildings, trees and particles.
Second is garbage collection(tested in a real game not a tech demo).
Third is pathfinding .
Fourth is the cost of AI (another than pathfinding)

Considering the code, it would be ease to reduce the algorithmic complexity such as separating the units into grids .
If you really want to optimize AS3 code you would want to use Structure of arrays, so have the Unit class store only index and have the positions in Vector.<Number>

UnderwaterBird

HaimSh
for a generic RTS and for a first plan I would agree, but in my case it's extremely revealing, as I hope it might be for some other game genres that require mass calculations.

I've worked with a lot of graphics on mobile devices and I'm pretty sure that on almost any PC any GPU-rendering engine will easily handle few thousands of objects on screen with correct implementation of MeshBatches / Atlases. And I'm planning to fit all gameplay inside a single atlas, at worst one for all units, one for all buildings and one for all backs, so we have 4 draw calls (UI is 4-th), not an issue for any decent engine.

Garbage collection behavior is something we can predict: Unity incremental GC should do it's job without freezes and V8 GC is modern with reference counting, so I believe I have enough reasons to expect its decent behavior. Some performance optimizations in V8 sound mind-blowing, so I'm quite sure they did not mess up their GC.

Pathfinding is based on calculations, which this tiny benchmark represents well enough.

AI development is a mystery to me, but in worst case scenario I can postpone it, as my idea is more of a MP-oriented. Still, AI is affected primarily by code speed as well.

And one of the most important part personally for me, which I'm more silent about, is about fail-safe approach of picking a technology. It's smart to either use a technology you know good (which this test shows me it's not an option) or to pick a technology, learning which will benefit you in some other ways, because game might fail, so I have to be pretty sure at least my experience will benefit me in the job market, which is quite pessimistic right now, especially gamedev.

And my test shows me that NodeJs is indeed a viable option, as I had only code performance concerns left.

pol2095

I tried your code in OpenFL and your results are strange :
OpenFL html5 Chrome Win : 0.070 sec (Firefox : 0.100 sec)
as3 Air x64 app Win : 0.100 sec

UnderwaterBird

pol2095
Previously, I've corrected myself about flash exe build in IDEA, I was unable to imagine that just running non-debug build is 5 times slower than launching executable (0.113 - 0.135sec.), so only running SWF is that much slow (0.38 - 0.585sec.).

But as of Html5, my code, while is almost exactly the same, is noticeably quicker than in your result, so there should be some issue on your way of launching. Moreover, using hash sets, which are absent in raw AS3, improves html5 result even more.

Sadly, I can't edit my first post to add/correct AIR exe values and add Unity.

What's interesting, in-page Ruffle script (nightly builds, not even beta) took 5 sec. for 1000 units. Ruffle was quite good at emulating my games, but its code speed seem to suffer, at least as a page script in my little test. All my games work very good in Ruffle.

General

Interesting experiment. I tried running a simpler one, running this in AS3:

			var tm:uint = getTimer()
			var s:Number = 0;
			var count:int = 10000000;
			for (var i:int = 0; i < count; i++){
				s += i * i;
			}
			trace(getTimer()-tm)

I've got 325 ms
And then I've run this js in browser console:

	var tm = Date.now()
			var s = 0;
			var count = 10000000;
			for (var i = 0; i < count; i++){
				s += i * i;
			}
			console.log(Date.now()-tm)

And got 104 ms, 3 times faster.

However, as it was pointed earlier, for RTS many more things are needed.
And for example, I managed to get hordes of creatures in my AS3 game: https://www.youtube.com/watch?v=SgYpu1pkd-c

pol2095

@General
Are you using the debugger ?
I tried your code on flascc, the result is 0 ms (this test is not significant).

General

pol2095 Ah, looks like yes, I run the debug mode