Sure! Here’s the translation in American English:
—
Samsung Electronics has launched TRUEBench, an innovative evaluation standard designed by Samsung Research to measure the productivity of artificial intelligence (AI) in workplace environments. This tool provides a comprehensive set of metrics that allow for the assessment of large language models (LLMs) across various productivity applications, encompassing different dialogue scenarios and multilingual conditions.
The development of TRUEBench addresses the growing need to evaluate the effectiveness of LLMs in common business tasks such as content generation, data analysis, summarization, and translation. This benchmark is characterized by its 10 categories and 46 subcategories, including a total of 2,485 test sets in 12 languages. Unlike other standards that tend to be English-centric and limited to simple question-answer structures, TRUEBench allows for interaction between languages, thereby enriching the evaluation.
Paul (Kyungwhoon) Cheun, CTO of Samsung Electronics’ DX Division and head of Samsung Research, emphasized the importance of the company’s practical AI experience, stating that TRUEBench is expected to set a new standard in evaluation and reinforce Samsung’s technological leadership in this sector.
The evaluation process proposed by TRUEBench goes beyond simple accuracy in responses. Recognizing that user intentions are not always expressed explicitly, the system also takes implicit conditions into account. This approach is based on a collaboration between humans and AI, aiming to ensure precise evaluation criteria, thereby minimizing subjective biases and ensuring consistency in results.
Additionally, the data samples and rankings of TRUEBench will be available on the open-source platform Hugging Face, offering users the ability to compare up to five different models. This transparency in performance is complemented by information on the average response length, providing a comprehensive view of the efficiency and effectiveness of AI models in the current market.
—
If you need any adjustments or further information, feel free to ask!
Source: MiMub in Spanish
