Roadmap feedback

Feb 12, 2010 at 12:52 AM

Hello,

My name is Josh Blake. I'm the lead developer of InfoStrat.VE and blog about natural user interfaces. I'm also writing a Manning book about multi-touch development.

I took a look at the roadmap and have some feedback.

*       Beta Release of core Scroll and TranslateRotateScale behaviors to developer community. <-- You are here
*       Revised Scroll and TranslateRotateScale behaviors
*       Hold Menu (Single Finger Touch & Hold Context Menu)
*       Select (Single Finger Tap)
*       Select Lasso (Single Finger Draw Lasso)

The above goals seem fine. Hopefully the context menu is easily skinnable.

The ones below are all very specific gestures that are non-standard. In addition, the wording assigns specific gestures specific meanings, but the suggested meanings may not be appropriate in all contexts.  As one example, applications running on hardware that only allows two touches could not use the Clone, Global Rotate, and Global Perspective Rotate gestures, so designers would likely want to use a different gesture for those actions. Also the difference between the gesture and meaning of Global Rotate and Global Perspective Rotate is unclear.

If I were designing an application and evaluating this library for use, I would be a bit put off by this list. Instead what would be useful if a more generic framework for gesture and manipulation recognition with associated events. The meaning of the gesture and implementation of the event handlers should be left up to the application designers.

*       Draw (Three Finger Pen Grasp)
*       Erase (Fist Wipe)
*       Create (Single Finger Draw '+')
*       Clone (Two Finger Double Tap)
*       Delete (Single Finger 'X' Drawn Over Target)
*       Open / Edit (Single Finger Double Tap)
*       Global Rotate (Five Finger Grab and Rotate of Application)
*       Global Perspective Rotate (Five Finger Pressure to Rotate Application Perspective)

Just my two cents!

Coordinator
Feb 12, 2010 at 4:58 AM

Hi Josh,

Thank you for reaching out. Our goal in posting the roadmap was to solicit discussion just like this.

We spent a lot of time on the code we posted (although it can always improve) to insure it was flexible and easy to implement. I think making sure the context menu is skinnable is a great idea. Truth be told we don't like menus (or buttons for that matter). I might be preaching to the choir but buttons and menus, in our opinion, are not part of the future of NUI. They are, however, a necessary evil and we would be remiss to not include them.

With regards to your statement about non-standard gestures, I'm wondering if you can point us to the standard alternatives? We'd love to compare and consider them. We stole the three finger pen grasp from Apple. See it [url:here|http://news.softpedia.com/news/More-Multitouch-from-Apple-Pretend-to-Write-Draw-79301.shtml]. Their approach to erase didn't feel right so we looked for something more natural (like you use your fist to erase a whiteboard). We're certain the fist gesture won't work on a 7" touchscreen which leads me to the next point...

Your point about hardware is well taken. While this code is meant to be a jump-starter until WPF 4 releases, we hope to use this platform as an opportunity to explore the boundaries of touch both from a hardware perspective as well as natural adoption. We are also exploring these gestures in touchless contexts. Love [url:this|http://touchless.codeplex.com/] project btw. Global Rotate (grabbing the entire application and spinning it to be right-side-up for the viewer for example) and Global Perspective Rotate (Measuring changes in the imprint size of each finger point to determine changes in pressure and using that to tilt the entire application in a 3D environment) would only be available today on technology like the Surface. We don't know how long it will be before the hardware vendors unlock the throttled drivers they are shipping with some units today, but we hope not long.

Ultimately we hope the discussion will happen here. We don't intend to put anyone off but hope instead to start the dialog and offer our vision / suggestions as a starting point. The great thing about open-source is you can take it and make it your own. I'm not sure if you or Marc also mentioned converting the fine work done [url:here|http://expressionblend.codeplex.com/] to realize the delete and create gestures mentioned in the vision, but that's awesome!

Thanks again Josh!