Emulating multi-touch device for Windows 7

Comments (20) | Share

Short while after I wrote about resources for multi-touch on Windows 7 Daniel D left a comment that got me very excited:

MultiTouchVista now has a driver that emulates multitouch hardware for Windows 7

I got to try it out myself! Bit later I got everything up and running and I’m happy to report that it’s all true: you can now effectively emulate multi-touch devices under Windows 7. You can see it yourself on this video. In fact it works so well that I was able to demonstrate it last Saturday at 4Developers conference.

That being said, the process to get it running involves few steps so I thought it would help to have a walkthrough to follow for anyone who would like to try it. So here it goes.

Hardware

Of course you need a PC. Note that computer vision puts CPU to heavy use, and I was able to get barely 15 FPS on my laptop with 1.6Ghz, and thus the interaction wasn’t very smooth. In fact Windows 7 specification requires multi-touch devices to report at least 50hz per finger.

The software that we are going to use allows to work with number of protocols. One option is to simply connect multiple USB mice to emulate multi-touch. But it is much more fun to build your own surface-like table. For start you can built a MTmini table designed by Seth Sanders – it takes about 20 minutes to build and all you will need is:

• Webcam – people try different kinds of webcams for this, but I think even something simple with decent video should work. I have great results using Logitech QuickCam Pro 9000 because it has excellent video quality in 640x480 and most important you can set the focus and other parameters manually.
• Tracing paper – it’s purpose is to diffuse the light coming to the camera so only objects very close to the surface (i.e. your fingers) look sharp. Initially I used just ordinary printer paper, but tracing paper gives more even image.
• Photo frame – no, I’m not talking about any fancy LCD frames. It’s just plain wood frame (word of advice: bad things will happen if you try to repurpose frame from your wedding photo). Frame I use has A4 paper format (210mm × 297mm) and fits nicely on top of the carton box from the packs of A4 office paper.
• Cardboard box, duck tape, and scissors.

When you have all the materials here is what you do:

1. Put the camera at the bottom of the box facing up, and glue it with the duck tape. You might need to cut out the hole for the camera’s USB cord.
2. Remove the back board from the photo frame and put only the tracing paper on the glass.
3. Put the frame on top of the box.

You can also watch this video for step by step instructions.

Here is my completed setup:

Software

Obviously you need to have Windows 7 Beta installed. So hopefully you was able to download it while it lasted.

Next thing to download and configure is tBeta. Here is description of it from official website:

The Beta, tbeta for short, is a open source/cross-platform solution for computer vision and multi-touch sensing. It takes an video input stream and outputs tracking data (e.g. coordinates and blob size) and touch events (e.g. finger down, moved and released) that are used in building multi-touch applications. tbeta can interface with various web cameras and video devices as well as connect to various TUIO/OSC enabled applications and supports many multi-touch lighting techniques including: FTIR, DI, DSI, and LLP with expansion planned for the future (custom modules/filters).

Download the software from the above website and unzip to folder you choose. Before you run it you might want to change configuration to use higher resolution if your webcam supports it. Go to \tbeta\data folder and open config.xml file. Specify the correct values for  WIDTH and HEIGHT in CAMERA_0 node.

One more thing I did to get more predictable results was to turn off all automatic adjustments in camera software. For Logitech QuickCam this includes switching to manual focus, and turning off Automatic settings for RightLight, exposure and gain. We are not going to move the camera around or dramatically change the lightning conditions so it will be best to make all settings constant.

When you run tbeta.exe you should see something like this:

The tricky part is calibration, which means adjusting the sliders for all filters at the bottom so that you get best recognition accuracy. Unfortunately I don’t know any rules that would help you with this. You would have to play with the settings a bit to get a feeling how it works. If you make it right you should see an outline and unique id associated with each of your fingers as seen on the above screenshot. When all works well you can minimize this window (press spacebar) so it won’t consume CPU for rendering.

The last piece of this puzzle is MultiTouchVista project that I mentioned in previous post as well. This framework, which actively developed by Daniel D himself (aka nesher), adds support of multi-touch to the current version of WPF. But for purpose of this article the important thing is the recently added Windows 7 multi-touch driver. It’s not even officially released yet, so you would need to go directly to Source Code tab and download latest bits (I run it on changeset 18685).

You can follow instruction in this video on how to install the driver and run the services. Here is a short summary of the same:

1. Compile all projects for MultiTouchVista. Follow instructions on this page.
2. From Windows Explorer go to MultiTouchVista\Main Source folder, and while holding down Shift right click on Multitouch.Diver folder and select “Open Command Window Here”.
3. In this window type: install driver.cmd, press Enter and ignore all warnings about not certified drivers.
4. Open Device Manager and confirm that new driver called “Universal Software HID device” is installed under “Human Interface Devices”. Video suggests to disable and then enable the driver to ensure it is working correctly.
5. To verify the driver is installed you can open “Pen and Touch” applet from Control Panel. It should now contain the Panning tab.
6. Back in Explorer go to MultiTouchVista\Main Source\Output folder and run  Multitouch.Service.Consol.exe. By default it loads the MultipleMiceInputProvider so if you should see red dots indicating “virtual cursor” for each USB mouse you have attached. But to use your MTmini table you would need to change configuration so it connects to tBeta through TUIO interface.
7. From MultiTouchVista\Main Source\Output run Multitouch.Configuration.WPF.exe. From list of Available devices select Tuio and click the arrow button in middle to make it Active device. Then click “Restart service” button to apply the new configuration.
8. Finally go to MultiTouchVista\Main Source\Multitouch.Driver.Console\bin\debug and run Multitouch.Driver.Console.exe.
9. Now if you put finger on surface, you should see the input messages coming to the console window. And of course now you should be read to use it as input device for Windows.

Try it on several touch enable applications that I listed in my last post (XPS Viewer and Paint are great for quick demo). You can also find some C++/C# examples on MSDN Code Galery.

Great thanks for Daniel D for leaving this inspiring comment, and even bigger compliments for developing such great library. Please don’t hesitate to let me know if I can help with anything.