Mark Rejhon – aka Chief Blur Buster – is now W3C Invited Expert
As creator of Blur Busters as well as TestUFO, I also have industry standards writing experience (XMPP extension XEP-0301) and a peer reviewed conference paper (display pursuit camera technique now in use by multiple monitor review websites).
Several years ago, I created TestUFO Motion Tests, the first precision display motion-test website. With this, I began benchmarking web browsers, with the first 120Hz Web Browser Tests, beginning a few years ago.
As of April 2017, I am now an Invited Expert at W3C Web Platform Working Group with a recent accepted modification to HTML 5.2 via a github commit. This is the HTML standard that web browsers aims to comply with.
This article aims to begin a serious discussion with the industry about further work on “Better Than 60Hz” support within HTML 5.2 (or 5.3) and beyond.
Earlier Discovery of VSYNC Limitations in Certain Browsers
W3C Animation Timing Standard (end of section 5) recommends that browers synchronizes animations to refresh rates.
Chrome was already compatible with 120Hz+ monitors in 2012. Then in 2013 FireFox fixed this (#1052354, #707884). In testing, I discovered Internet Explorer had a frame-rate capping behaviour (a behavior that persists in Edge today).
I have also collaborated with Duckware who owns www.vsynctester.com, who have also discovered flaws with VSYNC-related support in web browsers.
As Invited Expert status, this gives Chief Blur Buster the ability to directly submit edits to HTML 5.2 (in this specific sphere of expertise) for W3C peer review to the HTML 5.2 standard after appropriate consultations.
Why Is This Important?
There is little in web browser standards to accomodate “Better Than 60Hz” monitors, which leads to very inconsistent behaviours between different web browsers.
However, in the last few years, this topic has become extremely important, and harder to ignore:
- High refresh rate monitors
They are now being sold at mainstream stores like Staples and Best Buy. In addition, the new 8K standard requires a 120Hz refresh rate.
- New variable refresh rate displays
These includes VESA Adaptive-Sync, HDMI 2.1 VRR, FreeSync, G-SYNC. Microsoft’s Scorpio Project (the next XBox) is already planned to support variable refresh rate. Variable refresh rates can also provide cinema-smooth playback (e.g. 24fps, 48fps, HFR 120fps).
- Smooth playback of unusual framerates
Variable refresh rate displays (which can seamlessly change refresh rates dozens of times a second) allows applications to choose their own refresh rate simply by playing a specific framerate. An application can then successfully choose to perfectly smooth 48fps at 48Hz playback (cinema HFR playback), or 125fps at 125Hz playback (divisor of common mouse cursor update rates)
- Multi-Monitor Setups with Different Refresh Rates on Different Monitors
If not implemented properly, strange behaviours can occur when different displays run at different refresh rates. Some browsers behave properly, while others do not.
- Virtual reality headsets / WebGL
Oculus Rift and HTC Vive headsets at a 90 Hz refresh rate. They behaves as a 2nd monitor running at a different odd refresh rate than the primary monitor. HTML work is being done with WebVR and WebGL, but there can still be unusual behaviours for browser developers unaware of strange refresh rates.
- Motion Testing / Display Reviewers / Science
There are many reviewers using motion tests to benchmark displays such as gaming monitors, GSYNC monitors, and FreeSync monitors. Several reviewers now use a free Blur Busters invention for testing displays. Even major websites such as TomsHardware and ArsTechnica have have used some of our test techniques. Formerly, this was the domain of ultra-high-priced lab equipment or native executable files.
- eSports gaming / Scientific / Low Latency
Emerging applications require more browser control of the latency of screen updates. There are many subtle intractions between update rate and latency. For example, oscilloscope tests show that VSYNC OFF 1000fps at 60Hz have very noticeably far less latency than 60fps at 60Hz for special-purpose latency-critical applications. eSports gaming has become a billion-dollar industry. Microsoft Research confirms 1000fps makes a noticeable difference in the latency of user input.
Why Invited Expert?
Most software developers are not familiar with the existence of high-refresh-rate displays, nor are aware of special considerations (eSports, scientific motion tests, etc).
Nor are most writers of industry standards, either.
I found myself in the unusual position of simultaneously having:
(A) industry standards writing experience.
(C) software developer experience; and
(D) expert in displays and monitors — including a peer reviewed paper.
Being one of the few such individuals in the world to simultaneously meet all the above, someone at W3C asked me to apply for Invited Expert status, and my application got accepted.
This streamlines my ability to submit changes to HTML 5.2 for W3C peer review. As the year progresses, I’d like to open a public discussion about further improvements to the HTML standard.
Introduction of VSYNC limitations in current browsers
With the need for a stronger VSYNC to synchronize animations to the display’s refresh rate.
HTML API for Uncapped Framerate Support
HTML API for Variable Refresh Rate Support
HTML API for Bypass Windows Compositing During Full-Screen Mode
In full screen mode (not windowed mode, or “fullscreen borderless windowed mode”) it is possible for applications of multiple platforms to bypass WDM compositing and decrease input lag by another full frame. Windows 10 Creator’s Update also have a low-lag Game Mode which may be taken advantage of, for reduced latency between browser and display.
HTML API for Discovery Of Above
Historically, end of section 5 of W3C timing standard — https://www.w3.org/TR/animation-timing/ — recommends synchronization to refresh rate (VSYNC) — and work on #785 has fixed some shortcomings of the wording in HTML 5.2. This is great for many applications, but not all.
Many displays support other refresh rates, such as 75Hz. Also, there are many displays other than 60Hz, including gaming displays (120Hz, 144Hz). And VR displays such as Oculus and HTC Vive operate at 90Hz which Chrome supports (360 degree YouTube videos). Also, future TV standards (e.g. 8K 120Hz) may introduce 120Hz as a more standardized refresh rate in coming years.
It is extremely important to standardize how animations behave at refresh rates other than 60Hz, as synchronizing animations exactly to refresh rate is important, regardless of refresh rates. (Even 115fps @ 120Hz results in unsmooth motion), but also still provide an API to optionally decouple update rates from display refresh rate.
Browser comparison on refresh rate support:
Browser-based motion tests for analysis (test these at refresh rates other than 60Hz):
- NOTE1: Don’t forget to consider there are multi-monitor desktop computers. Different refresh rates for different monitors is possible. In other words (for multi monitor systems), the refresh rate of the specific display that the browser window is running on. Windows already changes the desktop manager update rate everytime the window moves to a different-refresh-rate display. Some browsers adapt correctly (automatically changes callback rate of requestAnimationFrame() when this happens) when windows are moved, but others mess up (suddenly goes janky / stuttery).*
- NOTE2: Don’t forget emerging variable refresh rate displays including G-SYNC, FreeSync, VESA Adaptive-Sync, and HDMI 2.1 VRR. (The next XBox supports variable refresh rate — Microsoft confirmed) Instead of software waiting for the display refresh, the display waits for the software. The display immediately refreshes the frame delivered by software — so refresh rate runs at frame rate. E.g. If you get 53fps, the display is 53Hz. Etc. The refresh rate of the display changes dynamically dozens of times per second to exactly match the refresh rate to the framerate, for stutter-free changes to framerate (simulated animation — test this on Windows/Mac with all other browser tabs closed for the most accurate visual simulation). Eventually, there may need to be a long-term path for browsers becoming “VRR-aware” at least in full-screen mode, very useful for WebGL 3D games.
- NOTE3: WebGL games can use wglSwapIntervalEXT(1) and glSwapIntervalEXT(0) to enable/disable synchronizing to VSYNC. But, they are still hampered by the latency of the window compositing manager. Most operating systems permit a full-screen mode that bypasses windows compositing delays, so this could be a feature (flag). In this particular case, if you both (A) bypass compositing and (B) use glSwapIntervalEXT(0) … for WebGL applications, you’d get the ‘tearing’ effects (while gaining useful “eSports-friendly” latency-reducing effects) of running framerates far higher than refresh rate.
For wording fixes to requestAnimationFrame() rate, see #785. This is related wording clarifications to existing VSYNC-driven requestAnimationFrame() — but this is separate from the additional features listed here.
VSYNC API Requirements for HTML 5.2
Modifications to a VSYNC API needs to solve the following, at least several or all of:
- Ability to detect whether requestAnimationFrame() or canvas (WebGL, etc) is dropping frames
In other words, frames missing the current refresh cycle of the monitor. Aka dropped/skipped/throttled frames during refresh rate synchronization. Many games and certain kinds of applications need self-awareness to know if frames have missed their refresh rate. Websites created by multiple people including www.testufo.com (my creation) and www.vsynctester.com (@duckware creation) use useragent detection & heuristics. The heuristics is needed to make a guess whether or not a frame was missed. And to make a wild guess whether requestAnimationFrame() is running at a rate different from the display refresh rate. Understandably, this is kind of a witchcraft of clever coding that should be avoided. The existence of this API would also double as confirming that the browser supports VSYNC (currently www.testufo.com/browser.html uses useragent detection to achieve the same thing).
NOTE: An accurate VSYNC time in requestAnimationFrame() callback parameter is another way to allow useragents to detect dropped frames.
- Ability to enable framerates far higher than refresh rate
REASON 1 — Optionally letting framerate vary as performance allows, makes things more compatible with future VRR displays. Letting browsers choose to decide to render as fast as possible, and then letting the display automatically decide to raise/lower refresh rate to match the current render-rate / update-rate. Browser vendors can then decide whether or not they want to support the emerging variable refresh rate displays such as G-SYNC / FreeSync / VESA Adaptive-Sync / HDMI 2.1 VRR. This could piggyback on existing permission-grant mechanisms (e.g. like Full Screen Mode permission) since faster rendering can use more battery power.
- Ability to query whether or not Windows’ compositing is bypassed
- Ability to to return the current maximum refresh rate of the display that the browser window is currently on
While this can be roughly achieved by counting requestAnimationFrame() (except on IE/Edge — see #785), there should be an easier mechanism to query. Should ideally be must be multimonitor aware — different displays can run at diverging refresh rates. (As one example, windows’s desktop manager changes update rate to the refresh rate of the monitor that the largest surface area of the window is on). Several browsers have very flawed behaviour on multimonitor systems with divergent refresh rates (including systems with two different ‘kinds’ of displays: A 60Hz monitor and a 90Hz Oculus VR goggles). Divergent refresh rates are still niche but getting common enough to merit departure away from the “one-refresh-rate” assumption prewritten into earlier standards. Also, if querying the refresh rate of a VRR display (Variable Refresh Rate such as G-SYNC, FreeSync, VESA Adaptive-Sync, and HDMI 2.1 VRR) — that automatically matches refreshrate to current framerate — then it should return the maximum refresh rate that the current VRR display is capable of.
NOTE: This one might be a reasonable substitute/alternative to missed-frame detection. Knowing the refresh rate means we can count frames per second within a browser page, and compare to refresh rate — although it’s possible to have a smooth 60fps@60Hz and a stuttery 60fps@60Hz when frames don’t exactly line-up with display refresh cycles.
Invitation to Software Developers & Industry Representatives
As owner of Blur Busters, and mentioned before, I am now an Invited Expert in W3C Web Platform Working Group. I welcome gaming industry programmers / those in this industry to collaborate on this topic.
You can also collaborate on this bug-tracking entry for adding a HTML VSYNC API:
HTML 5.2 VSYNC API: Support for VSYNC OFF / 120Hz / variable refresh displays / etc
By collaborating here, you can help influence wording of the changes to the HTML 5.2 standard, necessary to introduce a VSYNC API.
I can also be directly reached at firstname.lastname@example.org for ideas, suggestions, and technical discussions. Alternatively, you can also follow up at github #375 if you are a github member. Or follow up in the Comments below.