MSEmbGAN : Multi-Stitch Embroidery Synthesis using Region-Aware Texture Generation


Xinrong Hu, Chen Yang, Jin Huang, Lei Zhu, Ping Li, Bin Sheng, Tong-Yee Lee, Senior Member, IEEE



Abstract

Although convolutional neural networks (CNNs) have been widely used in embroidery synthesis, these methods tend to fail in predicting diverse stitch types, which are often contained in real embroidery images. The reasons behind this are that these network architectures do not have an independent module to capture the stitch features of an embroidery image, and there is few multi-stitch embroidery datasets labeled with the stitch type for embroidery synthesis task training. In this paper, we propose a multi-stitch embroidery generative adversarial network (MSEmbGAN) by considering the influences of stitch types on predicting embroidery image synthesis. To the best of our knowledge, our work is the first convolutional neural network for synthesizing multi-stitch embroidery images. To achieve this, we devise the region-aware texture generation network. It detects multiple regions of the input image using a stitch classifier and generates stitch texture for each region based on its shape features. In addition, a structure generation network with a structure discriminator is developed to further optimize the whole image. It maintains the structure consistency between input images and the output embroidery image to make the shape and color features of the result close to the input image. Meanwhile, we present a multi-stitch embroidery dataset which is labeled with multiple stitch types (three single-stitch types and a multi-stitch type). To the best of our knowledge, this dataset is also the largest embroidery dataset currently: more than 30K high-quality multi-stitch embroidery images (over 13K aligned content-embroidery images and over 17K unaligned images). Experimental results including a user study show that our MSEmbGAN clearly outperforms both the current state-of-the-art traditional method on embroidery synthesis and the popular state-of-the-art style transfer methods.




Introduction video
Video Download ( .mp4 / 143M )
Related works comparison
Quantitative evaluation
{{column}}
{{row[column]}}
{{row[column]}}
User study
{{column}}
{{row[column]}}
{{row[column]}}
dataset
data_distribution
download






威斯尼斯人app下载安装-首頁(欢迎你)