Multilingual semantic sourcing using product images for cross-lingual alignment

In online retail stores with ever-increasing catalog, product search is the primary means for customers to discover products of their interest. Surfacing irrelevant products can lead to poor customer experience and in extreme situations loss in engagement. With the recent advances in NLP, Deep Learning models are being used to represent queries and products in shared semantic space to enable semantic sourcing. These models require a lot of human annotated (query, product, relevance) tuples to give competitive results which is expensive to generate. The problem becomes more prominent in the emerging marketplaces/languages due to data paucity problem. When expanding to new marketplaces, it becomes imperative to support regional languages to reach a wider customer base and delighting them with good customer experience. Recently, in the NLP domain, approaches using parallel data corpus for training multilingual models have become prominent, but they are expensive to generate. In this work, we learn semantic alignment across languages using product images as an anchor between them. This overcomes the necessity of parallel data corpus. We use the human annotated data from established marketplace to transfer relevance classification knowledge to new/emerging marketplaces to solve the data paucity problem. Our experiments performed on datasets from Amazon reveal that we outperform state-of-the-art baselines with 2.4%-3.65% ROC-AUC lifts on relevance classification task across non-English marketplaces, 34.69%-51.67% Recall@k lifts on language-agnostic retrieval task and 6.25%-13.42% Precision@k lifts on semantic neighborhood quality task, respectively. Our models demonstrate efficient transfer of relevance classification knowledge from data rich marketplaces to new marketplaces by achieving ROC-AUC lifts of 3.74%-6.25% for the relevance classification task in the zero-shot setting where the human annotated relevance data of target marketplace is unavailable during training.