AI-Powered Question Generator to Support Automated Chatbot in Japanese

AI chatbots are the latest solution to speed up customer services, marketing, sales, HR, supply chain, and IT help desks. However, developing an AI chatbot involves setting up questions and answers. This task might require lots of effort to analyze thousands of pages of instruction manuals for businesses with complex infrastructure like banking or logistics. Therefore, we want to shorten the process by training this AI model to analyze any Japanese instruction manuals. It will scan the document and automatically generate questions and answers accordingly. Then, businesses can use this output to develop their chatbot in any industry or any business function. 

We wanted this AI to be able to analyze text from different domains. However, it would be labor-intensive to use a traditional machine learning model for each domain. Therefore, we needed to work on Transfer Learning, which can transfer knowledge from one context to another.  Transfer Learning is more complex when applied in NLP than in Visual Learning. We used some of the latest language models, including T5 (Text-to-Text Transfer Transformer) by Google, BERT, GPT2, etc. There has been little to no application of these models for question generation. Therefore, our team needed to spend more time researching and finding solutions. Meanwhile, Japanese is a sophisticated language, which also makes it more challenging.  

The AI model needs to identify and select topically important sentences from the document. First, we get it to process the whole paragraph to identify sentences. Next, the AI processes each sentence with a POS Tagger. After that, it performs feature extraction, classifying sentences by their feature, selecting important sentences on which questions can be generated.  The AI will then identify question words with the selected questions and rearrange the sentence to generate questions.  Every week, the client reviews the AI for its usefulness and usability. Then, they give us feedback to improve the model. We are deploying this AI to analyze banking instruction manuals. After months of development, the AI has given more precise output. 

Developing a chatbot is never simple. But we have strived to work on this AI to make it easier for businesses to build their chatbots in Japanese. Our goal is to make it work for any business function or industry soon.

1. GEM Corporation is an IT Outsourcing company experienced in developing AI-based applications. We have worked on developing AI projects for various Japanese corporations across industries.

2. Our domain expertise includes Finance, Banking, Insurance, Healthcare, Manufacturing, Retails, Logistics, etc

3. We have more than 9 years of experience. Our offices are based in Hanoi, Vietnam, and Tokyo, Japan.

4. We have successfully built more than 100 successful projects for our clients in the US, UK, Europe, Japan, Korea, Singapore, and many more.

5. Let us know how we can help you build your banking solution. Contact us now and get a quote for your project.