If you have control over the video encoding, you can tune it quite a lot, to ensure maximum possible playback performance. In this case, you should try to target the lowest possible H.264 profile and level. The only downside to doing this, is a larger video file size, as more CPU and memory intensive algorithms are disabled.
Profile should be Baseline. This will disable features that are decoding intensive, such as CABAC, B slices, etc.
Figuring out the lowest possible Level is slightly trickier, but I'll give you a formula. You'll need to refer to the levels table on Wikipedia:
The columns I'm interested in here are Luma samples. There's one for "Max. decoding speed" and another for "Max. frame size".
To calculate your videos frame size , multiply it's pixel width, by pixel height. For example:
1920 x 1080 = 2073600
Looking at the Wikipedia chart, that number sits just under level 4 in the "Max. frame size" column (2,097,152).
Sometimes your framerate pushes you into the next level though, which is where "Max. decoding speed" comes in.
Multiply the previous result with your framerate. For example, if it were 30fps:
2073600 x 30 = 62208000
Again, this result sits just under the level 4 value in the "Max. decoding speed" column (62,914,560), which confirms level 4 is the minimum possible level. If you had a framerate of say 60fps, you'd be pushed up to level 4.2.
I use Handbrake (https://handbrake.fr) for encoding, but hopefully what ever you're using has that level of control too.