DeepMMATE: Deep learning based multimodal architecture for audit taxability classification with XAI

Review of non-taxable products is an important internal audit which is carried out by majority of e-commerce stakeholders. This process usually cross checks the initial taxability assignments to avoid any unnecessary penalties incurred to the companies during the actual audits by the respective state compliance teams/tax departments. In order to handle millions of products sold online on e-commerce websites, we can adopt a machine learning solution to scale up the processing of products and make faster taxability predictions. However, a fine-grained classification cannot be achieved by visual analysis alone(product images). Often, the relevant information is present in the form of text on the product title, description & feature bullets etc. In this paper, we put forward a Multimodal Siamese based deep neural network which is capable of taking inputs from both product images and other textual content associated with it and predict the final output taxability. We show that this Multi-modal architecture outperforms single modality networks which are only based on vision or language by a margin of at least 5-6%. Furthermore, we reinforce confidence in our taxability outputs by incorporating an explainability wrapper around our model. This feature aids in establishing trust in the accuracy and reliability of our taxability predictions.