Text to Texture: Visual and Haptic Textures from a Text Prompt
Describe a texture in words, see it as an image, and feel it as a vibration: a complete text-to-haptic pipeline (IEEE World Haptics Conference 2025).
The framework generates both a visual and a haptic texture from a text description: describe a surface in words, see it as an image, and feel it as a vibration. To our knowledge this is the first complete text-to-haptic pipeline that produces both visual and haptic outputs from a single text prompt; a user study in the paper supports its perceptual quality and usability.
How it works
- Text to image. Stable Diffusion makes a photorealistic, top-down image of the described texture.
- Image to attributes. AttributeNet predicts how rough and how hard it would feel (Rough-Smooth, Hard-Soft, 0 to 100).
- Attributes to vibration. An interpolation-based authoring algorithm mixes real textures from a library into a matching vibrotactile signal, played on a haptuator / stylus.
AttributeNet = a frozen, ImageNet-pretrained ResNet-50 (2048 features) + a small trained head (2048 → 512 → 128 → 128 → 2). Only the head is trained.
Results
Papers
Text-Driven Generative Framework for Multimodal Visual and Haptic Texture Synthesis
Myrah Naeem, Mudassir Ibrahim Awan, Seokhee Jeon
IEEE World Haptics Conference (WHC) 2025, pp. 140-146, 2025
