Making the web “just work” with any input: Mouse …
We recently announced support for the Touch Event API in Internet Explorer for Windows Phone 8.1 Update as a part of our commitment to make the Web “just work” for our users. This means IE now supports all three major pointing input APIs: Mouse Events, Touch Events, and Pointer Events. Mobile browsing experience gets better, but at the same time developers start wondering which API they should use.
To help developers understand when to use which API, we wanted to share our updated guidance, as well as our forward-looking thoughts on where we think the web platform should go next, considering Pointer Events is nearing support in Firefox while Google recently decided not to implement the specification.
We want developers to use the input technology that is right for them. We provide interoperable Touch Events for the broadest reach across browsers today and leading-edge Pointer Events for simpler coding patterns, more capabilities, more device support, easier responsive design, and best hardware acceleration. Here’s a breakdown of qualities and capabilities for each model.
|Mouse Events||Touch Events||Pointer Events|
|Supports pen, Kinect, and other devices||2|
|Provides over/out/enter/leave events and hover|
|Asynchronous panning/zooming initiation for HW acceleration|
|Usable cross-browser on mobile devices||4|
|Usable cross-browser on desktop devices||4|
1 In at least some Android browsers, touch events may fire for mouse input. However, this mouse input cannot be differentiated from actual touch input when using touch events.
2 All browsers fire some mouse events for single-touch input in order to preserve compatibility with mouse-based Web pages. However, it’s not currently possible to differentiate touch input from actual mouse input when using touch events. Similarly, Kinect gesture input on IE for Xbox will also fire some mouse events for basic compatibility.
3 Pointer Events are currently a Candidate Recommendation on track for Recommendation soon. Touch Events is a Recommendation. In 2012, the Web Events Working Group resolved to discontinue work on Touch Events Version 2 and close the WG in favor of standardizing Pointer Events.
4 Full support for pointer events is available in IE and on the way in Firefox. Partial support (the touch-action API) is available in Chrome, Firefox, and Opera and is being considered by Safari WebKit. Through polyfills, like Google Polymer Pointer Events and Hand.JS, the pointer event model can be used across browsers.
Check out this HTML5 Rocks article that explains some of these differences and issues in further detail.
Performance is critical for input, especially touch. The reality is you’re just sliding your fingers across a sheet of glass—but we want it to feel like you’re moving the Web page. This is why we invested 3 years of engineering to re-architect IE’s touch pipeline to be quad-threaded and GPU accelerated (starting in IE10 and improving in IE11), and we designed pointer events to leverage these types of architectures which have since also started to appear in other modern browsers.
Panning & zooming are typically the most critical of touch interactions, accounting for more than 60% of all touch gestures on IE and Chrome for Windows. Just about every native app, Web app, or site uses panning and/or zooming. It’s a gesture that’s very revealing when it comes to input performance. If panning is slow to start (jumping to your finger after a delay) or choppy (dropping frames), then you notice and remember it’s just glass.
A demo of possible pan & zoom perf impact from Touch Events (both on an Android LG Nexus 5 and Windows Phone HTC 8x).
Pointer Events enable hardware accelerated, asynchronous touch actions, which often improves performance.
In this basic example test page (configured with a 1000ms delay), developed by Rick Byers on the Chrome team, with touch events in IE we can see the first frame of panning doesn’t render until after the touch contact has moved for 1020ms and travelled 4.8cm, which is clearly perceivable by the user. Simply replacing Touch Event handlers with Pointer Event handlers reduces this latency to just 31ms and 0.3cm—a greater than 32x improvement.
Although the delay in this test page is artificially introduced, it was authored to demonstrate a fundamental performance issue that’s common on the web. In practice, we’ve seen pages better and worse than the delay exhibited by the 1000ms value. This variable unbounded dependency on the browser’s primary thread leads to unpredictable user experiences and limits progress towards the optimal 1ms response time. Microsoft Research’s Advanced Sciences researchers have demonstrated the incredible user impact of this type of latency reduction. We’re still on the road to 1ms pan/zoom response time and see Pointer Events as a crucial step towards this.
Pointer Events also have additional functionality not present in Touch Events, such as the over/out/enter/leave transition events developers have asked for, and additional functionality comes with some additional cost. But we’ve invested in reducing those costs. The added hit-testing required for these features typically amounts to a mere ~0.15ms cost, and we’ve got ideas on how we might improve that further should we start to see evidence of this impacting developers’ ability to achieve 60FPS performance.
In our post announcing Touch Events support, we mentioned issues on the desktop Web that prevent us from compatibly bringing this API to those devices. For example, if Touch Events are enabled on desktop then the following common coding pattern breaks mouse support:
In this incorrect pattern, crucial mouse event handling is not wired up on any browser that supports Touch Events (regardless of whether the device actually has a touchscreen). This causes major functionality issues like menus, drop downs, share icons, and more not correctly functioning with mouse on sites like Web MD, Macys, Onet, Pages Juanes, Globo, Samsung, Taobao, Huffington Post, and more.
To make both the user touch experience and developers’ life easier, we want to do the following:
- Add Touch Event support to IE on Windows Phone (done)
- Work with site developers to fix the issues mentioned above in hopes that one day we can enable the APIs in a responsible way (work in progress: watch out for emails or tweets from our outreach team and webcompat.com).
- Bring Touch Events to all of our platforms to make the support matrix less confusing for developers (work in progress)
- Contribute to the implementation of Pointer Events on other platforms (work in progress on Firefox and Chrome)
- Actively engage with site and framework developers to hear their feedback and desired direction for the web platform
- Share our internal design docs, architectural diagrams, testing methodologies, and our code to other browsers (work in progress)
Many developers will find it easiest to use Pointer Events with a polyfill for other browsers, like Google Polymer Pointer Events or Hand.JS. But if you choose to use Touch Events, just be sure to also provide proper mouse support and never assume the existence of Touch Event APIs means the device has a touchscreen.
Input events remain one of the bigger interoperability pain-points across browsers and devices. Like most interop issues, this inevitably leads to both developer and user pain in the form of complex coding patterns and compatibility issues. We’re committed to making this better through interoperability work in our browser and standardization work at the W3C.
We want to build a platform that works for Web developers and our users. So give us your feedback—what works for you? What doesn’t? What would you like to see next? Sound off at @iedevchat and the Chrome and Firefox issue trackers.