Research/Blog
Visual Product Search through Artificial Intelligence
- February 18, 2020
- Posted by: CellStrat Editor
- Category: Artificial Intelligence Computer Vision Deep Learning
![](http://www.cellstrat.com/wp-content/uploads/2020/02/CS-AIConclave-Poster-Session-FashionSynthesis-1024x295.png)
AI enabled visual product search was demostrated by AI Lab members team comprising of Darshan CG, Gouthaman Asokan and Pushparaj Muthu on 8th. Feb’20 in Bengaluru.
Abstract
Visual search is changing the way people shop online and it is fast becoming a must for all fashion e-commerce brands. Why? because that’s what people want. There are many times when we find it difficult to recall the exact name of a product or cannot describe it clearly to search for it or would love to reorder what we saw someone else wearing. In all these cases and many more, visual product search is what comes to the rescue. Instead of describing, we could just take a picture of the product and search for it.
GANs can be extensively used to translate the original input Image to suitable image which further goes to the Search Engine. Further, it’s possible to cut clothes from the image, and besides that to get the main colours of the clothes, as well as skin and hair colour. Using GANs with the ability to visualize and even personalize how products are shown is a big step toward increasing purchase intent and reducing returns.
![](http://www.cellstrat.com/wp-content/uploads/2020/02/Visual-Product-Search.png)
Algorithm
Mask-RCNN: First the person is segmented from the input image using the Mask-RCNN model and the mask it produces.
Pix2Pix GAN: Segment background, skin, hair and all clothes and accessories on the person.
HSV colour space: The cut off process for different classes from the segmented image is done by colour thresholding, thus we get masks for them. HSV space describes colours similarly to the way the human eye tends to perceive them.
Implementation
To run visual product search, we need to have a database of images that can be encoded into feature vectors. We then get a query image and compare it’s vector to the entire database to get the closest match.
Visual Product Search is widely used by E-commerce platforms to give users the option to search by using their smartphone camera. Google has been aiming at enhancing the overall fashion shopper experience, The Echo Look is Amazon’s “style assistant” that takes a photo of your outfit and makes fashion recommendations that are conveniently available for sale on Amazon.
Pinterest’s visual search feature, called Lens, allows users to search for items they’ve captured in a photo with their phone’s camera. Users can also upload existing photos from their camera roll.
View complete Poster here.