Interactions with Large Ubiquitous Displays Using Camera-Equipped Mobile Phones

Researchers: Seokhee Jeon, Jane Hwang, Gerard J. Kim, and Mark Billinghurst

In the ubiquitous computing environment, people interact with everyday objects in ways different from the usual desktop user interface — one typical situation being interaction with large displays such as televisions, mirror displays, and public kiosks, where the usual keyboard and mouse are not viable. This research introduces user interaction techniques using a camera-equipped hand-held device for large shared displays, considering two situations: (1) sharing the display from a distance and (2) interacting with a touch screen display at close distance. Using motion flow and marker recognition, a camera-equipped hand-held device can effectively replace a mouse to share, select, and manipulate 2D/3D objects and navigate within the environment presented through the large display.

Related Publications:

• Seokhee Jeon, Jane Hwang, Gerard J. Kim, and Mark Billinghurst, "Interaction with Large Ubiquitous Displays Using Camera-Equipped Mobile Phones," Personal and Ubiquitous Computing, vol. 12, no. 2, pp. 83-94, 2010.
• Seokhee Jeon, Gerard J. Kim, and Mark Billinghurst, "Interacting with a Tabletop Display Using a Camera Equipped Mobile Phone," Lecture Notes on Computer Science (HCI International 2007), vol. 4551, pp. 336-343, 2007.
• Seokhee Jeon, Jane Hwang, Gerard J. Kim, and Mark Billinghurst, "Interaction Techniques in Large Display Environments using Hand-held Devices," ACM Symposium on Virtual Reality Software and Technology, pp. 100-103, 2006.

Jeon, S., Hwang, J., Kim, G. J., & Billinghurst, M. (2010). Interaction with Large Ubiquitous Displays Using Camera-Equipped Mobile Phones. Personal and Ubiquitous Computing, vol. 14, no. 2, pp. 83-94. https://doi.org/10.1007/s00779-009-0249-0