HiTZ has found an innovative way to create chattbots in Basque and other small languages

The Centre HiTZof the University of the Basque Country/Euskal Herriko Unibertsitatea has created a new way of doingthe\u00A0 chat in small languages such as the Euskera, based on the multilingual open language model built by the Meta Research Center, Llama.

The usual way would be to feed Llama with texts and examples in Basque, but that manual work is very expensive. "Only big companies have been able to do it so far," explainsEneko Agirre, director of the research center HiTZ.

In order to avoid this work, HiTZ members have found an "innovative and efficient way" to adapt the chat to the Euskera. With the new method, it is enough to continue training with the Euskera text mass Llama , but the key to this is to be able to apply techniques to deal with the problem known as "catastrophic oblivion".

The work done opens up new avenues. On the one hand, the method itself can be applied to open models stronger than the Llama, and on the other hand, it can be done in other languages with a similar volume of text.

In fact, with regard to the number of documents on the open Internet, there are 1000 times more documents in English than in Basque, and 100 times more in Spanish. So far, the question has been whether conversations with chattbots in small languages can achieve the same good results as those in English or Spanish.