Improve Input Latency on A1U Killer Instinct?

[AtomicShroom]

Hey Code Mystics,

I was looking at this video that TwistedGamingTV posted of his Arcade1Up KI machine which looks at input latency, and upon closer inspection, it seems fairly big. I'm using this particular snippet here since we can clearly see when the button gets fully pressed in and when a movement occurs on screen:

https://youtu.be/0PWe4_PxYOY

As you can see, from the moment the button is fully pressed, we can count 5 frames of video before any movement is seen on screen. Since the video itself is encoded at 30fps on YouTube, and the emulator runs at 60fps, that translates to 10 frames of "in engine" input latency, which totals 166ms (1000/60*10). Relatively speaking this is immense when compared to, for example, Nintendo's SNES Classic which only has 5 frames (83ms) of input latency. It's double! It means that when a button is pressed, somehow, the emulator loop renders 10 frames of KI where the input is not taken into consideration. (Note that 1-2 of those 10 frames might be because of the monitor itself, but LCD monitors used in A1U cabinets generally have a very low inherent latency, unlike TVs which apply lots of post processing that induce lag.)

I'm curious to see if there's anything you guys could do to reduce or improve this? Generally this comes down to a variety of factors, but the biggest culprit is usually the method of VSync that is used combined with where in your loop the control polling occurs. What method of VSync are you guys using? Is it Triple Buffering? Because if so, while it is a good method for achieving a smooth framerate, it induces a lot of input latency since (as the name implies) it buffers 3 whole frames before one is displayed on the screen, meaning the frame you see reflects your input from 3 frames ago, not the current frame. The fact that Nintendo is able to achieve just 5 frames of input latency on the SNES Classic, which is also software emulation that runs on a Linux kernel, means that there *IS*, technically, a way to achieve lower input latency while maintaining VSync and avoiding screen tearing.

I trust you guys took care to minimize this as you developed this emulator, but can I implore you to take a second look and see if anything was missed or if anything could be tweaked in your rendering loop to bring this closer to 5 frames?

Many thanks!

[CodeMystics]

Hi,

Yes, through our work with SNK over years, we are acutely aware of the need to minimise input latency, especially in fighting games. We take pride in our work, and would never allow the emulator to waste 10 frames on anything. Certainly what you may understand about typical sources of input latency on PC or console is not necessarily applicable here, so we would caution you from jumping to conclusions. (No we do not triple buffer.)

For example, there are custom implementations of OpenGL, an uncommon GPU, serial communication between the control deck and PCBA, etc., all of which may contribute to latency and none of which we developed nor control. In fact, as developers purely of the emulation software (vs. OS or hardware), we have yet to see final hardware, so cannot comment fully on the contributions of these factors. That said...

We would suggest there are two significant problems with your comparison: 1. this is the arcade game, not the SNES game. SNES behaviour would not be applicable. 2. the Start screen is by no means representative of input lag. The original code will be doing many uncommon things at that stage (e.g. loading) which may take a few fractions of a second. The arcade developers would not have been concerned about optimising that delay in a non-critical phase of the game, nor would it be representative of input delay where it matters.

A more meaningful measure from the KI arcade game ROM is the time between gameplay action and response. We find the ROM itself is responsible for 2 frames of input latency between a joystick action and the beginning of a walk animation. Adding in the 1-frame contribution of the arcade machine's graphics chain, this would've presented as 3 frames of input latency on the genuine arcade machine. This would be the meaningful metric we should be comparing against.

Arcade1UP Killer Instinct can be controlled by a USB keyboard or game controller connected to the PCBA. That is how we develop and the only controller we currently have. Measuring the input latency between either keyboard or controller and beginning of the walk animation on the development PCBA using a slo-mo camera, we see a latency of 6 frames. This is 3 more frames than the arcade machine, which can only be attributed to a combination of graphics pipeline and/or the keyboard/controller driver.

Running Pong on the same hardware, we can switch between OpenGL and direct framebuffer writes and see a reduction in latency of 2 frames. Thus we conclude 2 of the 3 frames we net worse than the arcade machine are due to the OpenGL pipeline. The remaining frame will no doubt be keyboard-to-PCBA communication via USB, which is unsurprisingly going to take an extra frame vs. direct memory-mapped switch polling.

If there is any further input latency during game play, it would presumably have to be a result of control deck communication being slower vs. using a keyboard/game controller as a controller. We want to emphasise that we don't know that to be the case, nor that any such delay exists as we do not have a control deck to measure. If any such delay did exist, we would not be able to comment on the viability of improving the input latency between control deck and PCBA, as that was developed by a different company.

We hope this reply is informative.

[AtomicShroom]

Hey guys,

Many thanks for this very detailed response, it's very much appreciated! I have no doubt that you guys did everything in your power to minimize input latency in the software. Measuring it based on someone else's recording is certainly not the most accurate way to do it, and I'll have a more thorough look at it when I actually get the cab in my hands.

I'd just like to touch on one point that you brought up, which is that you say that the specific screen used in the video, the title screen, may not have been optimized for input latency as the game is possibly loading things, etc. However TwistedGamingTV also included the very same test on his real KI arcade machine, and as you can see from the video here:

https://youtu.be/rrMaQb8iCqo

The real arcade machine, on the very same screen, shows a screen update on the very next frame that the button is pressed. So we can safely rule out the possibility that the game itself is not responsive on that screen.

Cheers!

[AtomicShroom]

So there was something that could be done after all, eh?

Kudos on the latest update! It feels fantastic now!

[CodeMystics]

So there was something that could be done after all, eh?

Kudos on the latest update! It feels fantastic now!

Yes, the problem turned out to be exactly as predicted:

If there is any further input latency during game play, it would presumably have to be a result of control deck communication being slower vs. using a keyboard/game controller as a controller. We want to emphasise that we don't know that to be the case, nor that any such delay exists as we do not have a control deck to measure. If any such delay did exist, we would not be able to comment on the viability of improving the input latency between control deck and PCBA, as that was developed by a different company.

We weren't able to see the control deck until it was released to the public and we were able to get our hands on a consumer unit. At that point we were able to look at it ourselves and could explore the issue and suggest refinements to the OS team's code.

[AtomicShroom]

Thank you so much for the insight and your commitment! Nothing short of amazing!