Here’s the translated text in American English:
<div itemprop="text"><p>Multimodal fine-tuning has emerged as an innovative strategy for customizing foundational models, especially for tasks that require integrating both visual and textual information. While base multimodal models are capable and demonstrate notable overall performance, they often show limitations in specialized visual tasks, domain-specific content, or particular formatting requirements. The fine-tuning technique aims to overcome these deficiencies by adapting models to specific datasets and use cases, resulting in improved performance on tasks that are critical for businesses.</p><p>Recent experiments have shown that fine-tuned Meta Llama 3.2 models can achieve increases of up to 74% in accuracy on specialized visual comprehension tasks compared to their base versions, thanks to optimized prompts. In this context, Amazon Bedrock has introduced fine-tuning capabilities that allow organizations to tailor Meta Llama 3.2 multimodal models to their specific needs. This process is based on applying best practices and scientific insights, supported by extensive experiments using public benchmark datasets for various tasks that combine language and visualization.</p><p>Suggested use cases for fine-tuning include visual question answering, where customization improves accuracy in image interpretation; graph interpretation, enabling models to analyze complex data representations; and image captioning, which increases the quality of the produced text. The ability to extract structured information from images in documents is also highlighted, such as identifying key elements in invoices or technical diagrams.</p><p>To leverage these functionalities, users need to have an active AWS account and enable the Meta Llama 3.2 models in Amazon Bedrock, currently available in the AWS US West (Oregon) region. It is essential to prepare appropriate training datasets in Amazon S3 to maximize fine-tuning results, prioritizing both the quality and structure of the data.</p><p>Experiments have been conducted using representative multimodal datasets, such as LlaVA-Instruct-Mix-VSFT, ChartQA, and Cut-VQAv2. These studies have revealed how performance scales with the size of the data used. Data quality and organization are key elements for the success of fine-tuning, with a recommendation to use a single image example per record. Although larger datasets tend to yield better results, it is advisable to start with small samples of around 100 high-quality examples before increasing the volume.</p><p>Configuring parameters such as the number of epochs and the learning rate plays a vital role in optimizing performance. Research suggests that for small datasets, a greater number of epochs facilitates optimal learning, while for larger datasets, a reduced number may be sufficient. Additionally, the choice between the 11B and 90B Meta Llama 3.2 models involves a balance between performance and associated costs, with the 90B model standing out for applications requiring maximum accuracy in complex visual reasoning tasks.</p><p>Fine-tuning the Meta Llama 3.2 multimodal models in Amazon Bedrock represents a significant opportunity for organizations to develop customized artificial intelligence solutions capable of integrating both visual and textual information. With a focus on data quality and appropriate customization, considerable performance improvements are attainable, even when using modest datasets, making this technology an accessible tool for various businesses.</p></div>Source: MiMub in Spanish