Blur Busters Working On Changes To HTML 5.2

Published May 1, 2017 by in: Industry Standards

Mark Rejhon – aka Chief Blur Buster – is now W3C Invited Expert

As creator of Blur Busters as well as TestUFO, I also have industry standards writing experience (XMPP extension XEP-0301) and a peer reviewed conference paper (display pursuit camera technique now in use by multiple monitor review websites).

Several years ago, I created TestUFO Motion Tests, the first precision display motion-test website. With this, I began benchmarking web browsers, with the first 120Hz Web Browser Tests, beginning a few years ago.

As of April 2017, I am now an Invited Expert at W3C Web Platform Working Group with a recent accepted modification to HTML 5.2 via a github commit. This is the HTML standard that web browsers aims to comply with.

EDIT June 2017: The new Apple announcement of an iPad with 120Hz and variable refresh rate support now adds extra urgency to standardization this within HTML 5.2 as soon as possible.

This article aims to begin a serious discussion with the industry about further work on “Better Than 60Hz” support within HTML 5.2 and beyond.

Earlier Discovery of VSYNC Limitations in Certain Browsers

W3C Animation Timing Standard (end of section 5) recommends that browers synchronizes animations to refresh rates.

Chrome was already compatible with 120Hz+ monitors in 2012. Then in 2013 FireFox fixed this (#1052354, #707884). In testing, I discovered Internet Explorer had a frame-rate capping behaviour (a behavior that persists in Edge today).

I created github #375 and #785 on W3C’s HTML bug tracking system. After several collaborations, I was invited to apply to become an Invited Expert at W3C Web Platform Working Group.

I have also collaborated with Duckware who owns www.vsynctester.com, who have also discovered flaws with VSYNC-related support in web browsers.

As Invited Expert status, this gives Chief Blur Buster the ability to directly submit edits to HTML 5.2 (in this specific sphere of expertise) for W3C peer review to the HTML 5.2 standard after appropriate consultations.

Why Is This Important?

There is little in web browser standards to accomodate “Better Than 60Hz” monitors, which leads to very inconsistent behaviours between different web browsers.

However, in the last few years, this topic has become extremely important, and harder to ignore:

  • High refresh rate monitors
    They are now being sold at mainstream stores like Staples and Best Buy. In addition, the new 8K standard requires a 120Hz refresh rate.
  • New variable refresh rate displays
    These includes VESA Adaptive-Sync, HDMI 2.1 VRR, FreeSync, G-SYNC. Microsoft’s Scorpio Project (the next XBox) is already planned to support variable refresh rate.  Variable refresh rates can also provide cinema-smooth playback (e.g. 24fps, 48fps, HFR 120fps).
  • Smooth playback of unusual framerates
    Variable refresh rate displays (which can seamlessly change refresh rates dozens of times a second) allows applications to choose their own refresh rate simply by playing a specific framerate. An application can then successfully choose to perfectly smooth 48fps at 48Hz playback (cinema HFR playback), or 125fps at 125Hz playback (divisor of common mouse cursor update rates)
  • Multi-Monitor Setups with Different Refresh Rates on Different Monitors
    If not implemented properly, strange behaviours can occur when different displays run at different refresh rates. Some browsers behave properly, while others do not.
  • Virtual reality headsets / WebGL
    Oculus Rift and HTC Vive headsets at a 90 Hz refresh rate. They behaves as a 2nd monitor running at a different odd refresh rate than the primary monitor. HTML work is being done with WebVR and WebGL, but there can still be unusual behaviours for browser developers unaware of strange refresh rates.
  • Motion Testing / Display Reviewers / Science
    There are many reviewers using motion tests to benchmark displays such as gaming monitors, GSYNC monitors, and FreeSync monitors. Several reviewers now use a free Blur Busters invention for testing displays. Even major websites such as TomsHardware and ArsTechnica have have used some of our test techniques. Formerly, this was the domain of ultra-high-priced lab equipment or native executable files.
  • eSports gaming / Scientific / Low Latency
    Emerging applications require more browser control of the latency of screen updates. There are many subtle intractions between update rate and latency.  For example, oscilloscope tests show that VSYNC OFF 1000fps at 60Hz have very noticeably far less latency than 60fps at 60Hz for special-purpose latency-critical applications.  eSports gaming has become a billion-dollar industry. Microsoft Research confirms 1000fps makes a noticeable difference in the latency of user input.

Why Invited Expert?

Most software developers are not familiar with the existence of high-refresh-rate displays, nor are aware of special considerations (eSports, scientific motion tests, etc).

Nor are most writers of industry standards, either.

I found myself in the unusual position of simultaneously having:
(A) industry standards writing experience.
(C) software developer experience; and
(D) expert in displays and monitors — including a peer reviewed paper.

Being one of the few such individuals in the world to simultaneously meet all the above, someone at W3C asked me to apply for Invited Expert status, and my application got accepted.

This streamlines my ability to submit changes to HTML 5.2 for W3C peer review. As the year progresses, I’d like to open a public discussion about further improvements to the HTML standard.

Introduction of VSYNC limitations in current browsers

With the need for a stronger VSYNC to synchronize animations to the display’s refresh rate.

HTML API for Uncapped Framerate Support

This is also called “VSYNC OFF”. More than 80% of users in the now-$1-billion-dollar eSports gaming industry use “VSYNC OFF” because it reduces input lag. Scientific applications also sometimes require this for low-latency tests. Photodiode oscilloscope measurements show that 1000fps @ 60Hz has at ~8ms less input latency than 60fps @ 60Hz. (Almost half a frame-time less lag, because waiting-on-vsync can force a frame to be delayed). In some cases, lag can spike to a full frametime less lag, e.g. 16ms less input lag. Chrome already supports this via a command line option “–disable-gpu-vsync” but is not enabled via JavaScript API

HTML API for Variable Refresh Rate Support

At least during full screen there needs to be a path of future support of optional JavaScript-controlled decoupling of updatesrates from fixed frequencies to allow useragent apps to support emerging variable refresh rate display technologies (e.g. VESA Adaptive-Sync, FreeSync, G-SYNC, HDMI 2.1 VRR, etc) for Canvas2D/WebGL/etc. Instead of apps waiting for VSYNC, the monitor immediately refreshes upon completed frame. If you’re running at 47fps the monitor is essentially 47Hz. Refresh rate changes to match framerate! It allows stutterless transitions between framerates (simulated animation via interpolation). In fact, Microsoft XBox Scorpio (the one after XBox One) is planned to support variable refresh rates.

HTML API for Bypass Windows Compositing During Full-Screen Mode

In full screen mode (not windowed mode, or “fullscreen borderless windowed mode”) it is possible for applications of multiple platforms to bypass WDM compositing and decrease input lag by another full frame. Windows 10 Creator’s Update also have a low-lag Game Mode which may be taken advantage of, for reduced latency between browser and display.

HTML API for Discovery Of Above

Even today, there’s no reliable way to detect the refresh rate of a monitor in a web browser (except with clever rAF() heuristics in Chrome, FireFox, Opera and Safari). There needs to be JavaScript ability to easily query for refresh rate, VSYNC ON/OFF support, variable refresh rate support, and low-lag full screen mode (bypass WDM compositing).

Historically, end of section 5 of W3C timing standard — https://www.w3.org/TR/animation-timing/ — recommends synchronization to refresh rate (VSYNC) — and work on #785 has fixed some shortcomings of the wording in HTML 5.2. This is great for many applications, but not all.

Competition gamers/eSports like using “VSYNC OFF” in full screen video games. The ability to do a W3C compliant HTML 5.2 Javascript API equivalent of turning on/off “–disable-gpu-vsync” would be highly favoured by full-screen WebGL game developers, to bring browser latencies (buttons-to-pixels) down. Ideally, this should be made possible in both canvas2D and with WebGL.

Many displays support other refresh rates, such as 75Hz. Also, there are many displays other than 60Hz, including gaming displays (120Hz, 144Hz). And VR displays such as Oculus and HTC Vive operate at 90Hz which Chrome supports (360 degree YouTube videos). Also, future TV standards (e.g. 8K 120Hz) may introduce 120Hz as a more standardized refresh rate in coming years.

It is extremely important to standardize how animations behave at refresh rates other than 60Hz, as synchronizing animations exactly to refresh rate is important, regardless of refresh rates. (Even 115fps @ 120Hz results in unsmooth motion), but also still provide an API to optionally decouple update rates from display refresh rate.

Browser comparison on refresh rate support:
www.testufo.com/browser.html

Browser-based motion tests for analysis (test these at refresh rates other than 60Hz):
www.testufo.com
www.vsynctester.com
www.testufo.com/photo
www.testufo.com/eyetracking
www.testufo.com/animation-time-graph

  • NOTE1: Don’t forget to consider there are multi-monitor desktop computers. Different refresh rates for different monitors is possible. In other words (for multi monitor systems), the refresh rate of the specific display that the browser window is running on. Windows already changes the desktop manager update rate everytime the window moves to a different-refresh-rate display. Some browsers adapt correctly (automatically changes callback rate of requestAnimationFrame() when this happens) when windows are moved, but others mess up (suddenly goes janky / stuttery).*
  • NOTE2: Don’t forget emerging variable refresh rate displays including G-SYNC, FreeSync, VESA Adaptive-Sync, and HDMI 2.1 VRR. (The next XBox supports variable refresh rate — Microsoft confirmed) Instead of software waiting for the display refresh, the display waits for the software. The display immediately refreshes the frame delivered by software — so refresh rate runs at frame rate. E.g. If you get 53fps, the display is 53Hz. Etc. The refresh rate of the display changes dynamically dozens of times per second to exactly match the refresh rate to the framerate, for stutter-free changes to framerate (simulated animation — test this on Windows/Mac with all other browser tabs closed for the most accurate visual simulation). Eventually, there may need to be a long-term path for browsers becoming “VRR-aware” at least in full-screen mode, very useful for WebGL 3D games.
  • NOTE3: WebGL games can use wglSwapIntervalEXT(1) and glSwapIntervalEXT(0) to enable/disable synchronizing to VSYNC. But, they are still hampered by the latency of the window compositing manager. Most operating systems permit a full-screen mode that bypasses windows compositing delays, so this could be a feature (flag). In this particular case, if you both (A) bypass compositing and (B) use glSwapIntervalEXT(0) … for WebGL applications, you’d get the ‘tearing’ effects (while gaining useful “eSports-friendly” latency-reducing effects) of running framerates far higher than refresh rate.

For wording fixes to requestAnimationFrame() rate, see #785. This is related wording clarifications to existing VSYNC-driven requestAnimationFrame() — but this is separate from the additional features listed here.

VSYNC API Requirements for HTML 5.2

Modifications to a VSYNC API needs to solve the following, at least several or all of:

  • Ability to detect whether requestAnimationFrame() or canvas (WebGL, etc) is dropping frames
    In other words, frames missing the current refresh cycle of the monitor. Aka dropped/skipped/throttled frames during refresh rate synchronization. Many games and certain kinds of applications need self-awareness to know if frames have missed their refresh rate. Websites created by multiple people including www.testufo.com (my creation) and www.vsynctester.com (@duckware creation) use useragent detection & heuristics. The heuristics is needed to make a guess whether or not a frame was missed. And to make a wild guess whether requestAnimationFrame() is running at a rate different from the display refresh rate. Understandably, this is kind of a witchcraft of clever coding that should be avoided. The existence of this API would also double as confirming that the browser supports VSYNC (currently www.testufo.com/browser.html uses useragent detection to achieve the same thing).
    NOTE: An accurate VSYNC time in requestAnimationFrame() callback parameter is another way to allow useragents to detect dropped frames.
    –
  • Ability to enable framerates far higher than refresh rate
    Basically, a JavaScript method of being able to enable/disable the equivalent of Chrome’s “–disable-gpu-vsync” flag. At the absolute minimum during full screen mode, maybe as an additional parameter for the existing HTML fullscreen API. This is very important for two emerging major things:
    –

    REASON 1 — Optionally letting framerate vary as performance allows, makes things more compatible with future VRR displays. Letting browsers choose to decide to render as fast as possible, and then letting the display automatically decide to raise/lower refresh rate to match the current render-rate / update-rate. Browser vendors can then decide whether or not they want to support the emerging variable refresh rate displays such as G-SYNC / FreeSync / VESA Adaptive-Sync / HDMI 2.1 VRR.  This could piggyback on existing permission-grant mechanisms (e.g. like Full Screen Mode permission) since faster rendering can use more battery power.

    REASON 2 — JavaScript developer access to access an ultra low latency mode for gaming. In eSports gaming (where crazy gamers play CS:GO at 1000fps on NVIDIA GTX Titans) — the use of 1000fps@60Hz has 8 milliseconds less input lag than 60fps@60Hz. This is consistent with experience by gaming authors. Also, latency were recently done in “–disable-gpu-vsync” and some GPUs such as NVIDIA GTX1080 was able to run the web browser at framerates in excess of 1,000 frames per second on several test pages. This means browser performance is reaching eSports territory. High speed video / input latency benchmarks showed less lag with buttons-to-pixels when framerates far exceed refresh rate. This is because at 1000fps, the frame seen by eyes was rendered only 1ms ago — the overkill framerate meant that whatever hit the screen was the freshest possible pixel, thus reducing mouse lag & keyboard lag for specialized low latency applications. In fact, in the billion-dollar eSports gaming industry — Bloomberg News, Forbes, New York Times. I have attended some of these events myself and noticed that more than 90% of paid competition game players use VSYNC OFF (disable GPU vsync) during gaming such as Counterstrike:GO, etc. If we were able to enable Javascript APIs to do this, browsers would more easily (over long term) enter the niche market of olympics-style “milliseconds-matters” races of tight reaction times and simultaneous-draw situations.–

  • Ability to query whether or not Windows’ compositing is bypassed
    Full screen mode can bypass window compositing manager (adds 1 frame of lag — 16ms). This is important for reducing latency during high-precision applications, eSports gaming applications, Mac can do compositing in sub-frame latency (Safari has 1 frame of lag), while Windows enforces an additional wait (Chrome has 2 frame of lag). Being able to let applications query for this lag, would enhance latency awareness for web applications, and allow “eSports-ready” benchmarking in pure Javascript. If compositing is false then “tearing artifacts” will appear during VSYNC OFF (–disable-gpu-vsync) which is normal. Tearing artifacts look like this (simulated). Generally, the higher the framerate (e.g. 1000fps@60Hz), the less visible tearing artifacts becomes during true compositing-bypassed direct-to-display VSYNC OFF operation (eSports league lowest latency mode).
    –
  • Ability to to return the current maximum refresh rate of the display that the browser window is currently on
    While this can be roughly achieved by counting requestAnimationFrame() (except on IE/Edge — see #785), there should be an easier mechanism to query. Should ideally be must be multimonitor aware — different displays can run at diverging refresh rates. (As one example, windows’s desktop manager changes update rate to the refresh rate of the monitor that the largest surface area of the window is on). Several browsers have very flawed behaviour on multimonitor systems with divergent refresh rates (including systems with two different ‘kinds’ of displays: A 60Hz monitor and a 90Hz Oculus VR goggles). Divergent refresh rates are still niche but getting common enough to merit departure away from the “one-refresh-rate” assumption prewritten into earlier standards. Also, if querying the refresh rate of a VRR display (Variable Refresh Rate such as G-SYNC, FreeSync, VESA Adaptive-Sync, and HDMI 2.1 VRR) — that automatically matches refreshrate to current framerate — then it should return the maximum refresh rate that the current VRR display is capable of.
    NOTE: This one might be a reasonable substitute/alternative to missed-frame detection. Knowing the refresh rate means we can count frames per second within a browser page, and compare to refresh rate — although it’s possible to have a smooth 60fps@60Hz and a stuttery 60fps@60Hz when frames don’t exactly line-up with display refresh cycles.

There are other APIs that may be needed, but this is the type of “update-rate and latency knowledge” that high-performance JavaScript applications need. This will be required for both WebGL and non-WebGL modes (e.g. HTML5 canvas).

Invitation to Software Developers & Industry Representatives

As owner of Blur Busters, and mentioned before, I am now an Invited Expert in W3C Web Platform Working Group. I welcome gaming industry programmers / those in this industry to collaborate on this topic.

You can also collaborate on this bug-tracking entry for adding a HTML VSYNC API:
HTML 5.2 VSYNC API: Support for VSYNC OFF / 120Hz / variable refresh displays / etc

By collaborating here, you can help influence wording of the changes to the HTML 5.2 standard, necessary to introduce a VSYNC API.

I can also be directly reached at [email protected] for ideas, suggestions, and technical discussions. Alternatively, you can also follow up at github #375 if you are a github member. Or follow up in the Comments below.

UPDATE: WHATWG Standards Collaboration

A new issue has been submitted to WHATWG as of January 2020. While it may take time for broad standardization, more consistency such as requestAnimationFrame() need to be accelerated before the arrival of popular 120 Hz smartphones.