Flash RTS-like Benchmark

UnderwaterBird

Unity 2022.3.20 is 0.026sec. with Arrays (EXE 64-bit, IL2CPP, .Net Standard 2.1, Faster runtime, Master)

Unity Burst compiler is 0.001sec. with NativeArray (same executable), though I'm a bit skeptical here as the result seems too good, but I can't figure out why.

pol2095

You can try flascc ?

UnderwaterBird

pol2095
I'm afraid I have not enough experience to try it.

I've benchmarked all this to make my own decision on what popular technology is suitable for a small 2D RTS with a lot of units and my result is that both Unity (which was obvious) and TypeScript (which was surprising) are good enough for it.

I hope my benchmarks will help other developers to make a decision what tool to use in case they need performance.

As of me, I'm preferring TypeScript, as while performance is quite similar, it has many quality-of-life features that Unity lacks for 2D (like decent speed in project launching, instant live reload, way easier syntax in tools (PixiJS, Spine plugins, etc.), TS is based on ES same as AS3, etc. ).

pol2095

If you share a complete as3 code sample, I can test it.

UnderwaterBird

pol2095
It's the same as in first message, just with launch code:

package
{
    import flash.display.Sprite;
    import flash.text.TextField;

    public class Main extends Sprite
    {
        public function Main()
        {
            var txtResult:TextField = new TextField();
            txtResult.width = 400;
            txtResult.defaultTextFormat.size = 30;
            addChild(txtResult);
            new RtsBench(txtResult);
        }
    }
}

package
{
    import flash.text.TextField;

    public class RtsBench
    {
        var unitCount:Number = 2000;
        var units:Vector.<Unit> = new Vector.<Unit>();

        public function RtsBench(txtResult:TextField)
        {
            this.createUnits();

            var nowTs:Number = new Date().getTime();

            this.doStuff();

            var nowTs2:Number = new Date().getTime();

            var timeDifferenceTs:Number = nowTs2 - nowTs;

            var timeDifferenceSec:Number = timeDifferenceTs / 1000;
            var timeMaxSec:Number = 1 / 60;

            var resultMessage:String = "AS3 time passed: " + (timeDifferenceSec) + "sec.\n"
                    + "time max for 60 FPS: " + (timeMaxSec) + "sec.\n"
                    + "\ntest with " + this.unitCount + " units " + (timeDifferenceSec < timeMaxSec ? "PASSED" : "FAILED");

            if (txtResult) txtResult.text = resultMessage;
            else trace(resultMessage);
        }

        private function createUnits():void
        {
            var i:int = this.unitCount;
            while (i > 0)
            {
                var unit:Unit = new Unit();
                this.units.push(unit);
                --i;
            }
        }

        private function doStuff():void
        {
            unitCount = 2000;

            var i:int = this.unitCount;
            while (i > 0)
            {
                --i;
                var unit1:Unit = this.units[i];

                var j:int = this.unitCount;
                while (j > 0)
                {
                    --j;
                    var unit2:Unit = this.units[j];

                    if (unit1 === unit2) continue;

                    var distance:Number = Math.abs(this.distance(unit1.x, unit1.y, unit2.x, unit2.y));

                    var maxRadius:Number = Math.max(unit1.radius, unit2.radius);

                    if (distance < maxRadius)
                        unit1.moveAwayFrom(unit2);
                }
            }
        }

        private function distance(x1:Number, y1:Number, x2:Number, y2:Number):Number
        {
            return Math.sqrt((x2 - x1) * (x2 - x1) + (y2 - y1) * (y2 - y1));
        }
    }
}

class Unit
{
    internal var x:Number = Math.random() * 800;
    internal var y:Number = Math.random() * 600;
    internal var radius:Number = 2 + Math.random() * 3;

    public function moveAwayFrom(unit:Unit):void
    {
        var speedX:Number = (unit.x - this.x) * .1;
        var speedY:Number = (unit.x - this.x) * .1;

        this.x -= speedX;
        this.y -= speedY;

        unit.x += speedX;
        unit.y += speedY;
    }
}

HaimSh

This is rather pointless benchmark, what you would want to test are RTS possible bottlenecks.
First is the cost of rendering many units, buildings, trees and particles.
Second is garbage collection(tested in a real game not a tech demo).
Third is pathfinding .
Fourth is the cost of AI (another than pathfinding)

Considering the code, it would be ease to reduce the algorithmic complexity such as separating the units into grids .
If you really want to optimize AS3 code you would want to use Structure of arrays, so have the Unit class store only index and have the positions in Vector.<Number>

UnderwaterBird

HaimSh
for a generic RTS and for a first plan I would agree, but in my case it's extremely revealing, as I hope it might be for some other game genres that require mass calculations.

I've worked with a lot of graphics on mobile devices and I'm pretty sure that on almost any PC any GPU-rendering engine will easily handle few thousands of objects on screen with correct implementation of MeshBatches / Atlases. And I'm planning to fit all gameplay inside a single atlas, at worst one for all units, one for all buildings and one for all backs, so we have 4 draw calls (UI is 4-th), not an issue for any decent engine.

Garbage collection behavior is something we can predict: Unity incremental GC should do it's job without freezes and V8 GC is modern with reference counting, so I believe I have enough reasons to expect its decent behavior. Some performance optimizations in V8 sound mind-blowing, so I'm quite sure they did not mess up their GC.

Pathfinding is based on calculations, which this tiny benchmark represents well enough.

AI development is a mystery to me, but in worst case scenario I can postpone it, as my idea is more of a MP-oriented. Still, AI is affected primarily by code speed as well.

And one of the most important part personally for me, which I'm more silent about, is about fail-safe approach of picking a technology. It's smart to either use a technology you know good (which this test shows me it's not an option) or to pick a technology, learning which will benefit you in some other ways, because game might fail, so I have to be pretty sure at least my experience will benefit me in the job market, which is quite pessimistic right now, especially gamedev.

And my test shows me that NodeJs is indeed a viable option, as I had only code performance concerns left.

pol2095

I tried your code in OpenFL and your results are strange :
OpenFL html5 Chrome Win : 0.070 sec (Firefox : 0.100 sec)
as3 Air x64 app Win : 0.100 sec

UnderwaterBird

pol2095
Previously, I've corrected myself about flash exe build in IDEA, I was unable to imagine that just running non-debug build is 5 times slower than launching executable (0.113 - 0.135sec.), so only running SWF is that much slow (0.38 - 0.585sec.).

But as of Html5, my code, while is almost exactly the same, is noticeably quicker than in your result, so there should be some issue on your way of launching. Moreover, using hash sets, which are absent in raw AS3, improves html5 result even more.

Sadly, I can't edit my first post to add/correct AIR exe values and add Unity.

What's interesting, in-page Ruffle script (nightly builds, not even beta) took 5 sec. for 1000 units. Ruffle was quite good at emulating my games, but its code speed seem to suffer, at least as a page script in my little test. All my games work very good in Ruffle.

General

Interesting experiment. I tried running a simpler one, running this in AS3:

			var tm:uint = getTimer()
			var s:Number = 0;
			var count:int = 10000000;
			for (var i:int = 0; i < count; i++){
				s += i * i;
			}
			trace(getTimer()-tm)

I've got 325 ms
And then I've run this js in browser console:

	var tm = Date.now()
			var s = 0;
			var count = 10000000;
			for (var i = 0; i < count; i++){
				s += i * i;
			}
			console.log(Date.now()-tm)

And got 104 ms, 3 times faster.

However, as it was pointed earlier, for RTS many more things are needed.
And for example, I managed to get hordes of creatures in my AS3 game: https://www.youtube.com/watch?v=SgYpu1pkd-c

pol2095

@General
Are you using the debugger ?
I tried your code on flascc, the result is 0 ms (this test is not significant).

General

pol2095 Ah, looks like yes, I run the debug mode

pol2095

in this test (not significant)

as3 (900 ms) :

var nowTs:Number = getTimer();
var array:Vector.<int> = new Vector.<int>();
var i:int;
var j:int;
var count:int = 20000;		
for( i = 0 ; i < count; i++ )
{
	for( j = 0 ; j < count; j++ )
	{
		if( i < 300 && j == 10 ) array.push( j );
	}
}
var nowTs2:Number = getTimer();
var resultMessage:String = ( nowTs2 - nowTs ).toString();
txtResult.text = resultMessage;

JS Chrome (1 300 ms) :

var nowTs = Date.now();
var array = [];
var i;
var j;
var unitCount = 20000;
for( i = 0 ; i < unitCount; i++ )
{
	for( j = 0 ; j < unitCount; j++ )
	{
		if( i < 300 && j == 10 ) array.push( j );
	}
}
console.log(Date.now()-nowTs);

C++ (700ms)

#include <iostream>
#include <vector>
#include <chrono>

uint64_t timeSinceEpochMillisec() {
	using namespace std::chrono;
	return duration_cast<milliseconds>(system_clock::now().time_since_epoch()).count();
}

int main()
{
	uint64_t nowTs = timeSinceEpochMillisec();
	int count = 20000;
	std::vector<int> array;
	int i;
	int j;
	for (i = 0; i < count; i++)
	{
		for (j = 0; j < count; j++)
		{
			if (i < 300 && j == 10) array.push_back(j);
		}
	}
	uint64_t nowTs2 = timeSinceEpochMillisec();
	std::cout << nowTs2 - nowTs;
}

Artemix

These tests are mostly unrelated to what happens in a real game scenario. If you truly want to test, make a super simplified version of the RTS you want to build, and port that to other engines.

Most likely AS3 will not come on top.

I would probablly use Godot depending on the game.