Amazon has launched a fully managed service called Amazon Bedrock, which offers a selection of high-performance foundational models from leading artificial intelligence companies such as AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon, accessible through a single API. This service also provides a broad set of capabilities necessary to build generative artificial intelligence applications with security, privacy, and responsible AI practices.
Some of these models are publicly available, allowing for customization for specific use cases and domains. However, securely implementing and scaling custom models is a complex task, due to the selection of instance type and deployment parameters necessary to host large models.
To address this challenge, AWS has announced the preview of the new Amazon Bedrock Custom Model Import, a feature that allows importing custom models created in other environments such as Amazon SageMaker, Amazon EC2, and on-premises installations to Amazon Bedrock. This feature simplifies the deployment process through simple APIs for model deployment and invocation.
The ability to customize foundational models can unlock significant value by adapting their capabilities to specific domains or tasks. In this instance, the process of fine-tuning the Meta Llama 3 8B model in SageMaker to specialize it in generating SQL queries (text to SQL) is detailed. Meta Llama 3 8B is a relatively small model that balances performance and resource efficiency.
AWS customers have explored fine-tuning Meta Llama 3 8B for SQL query generation, especially in non-standard SQL dialects, and have requested methods to import their custom models into Amazon Bedrock. This allows them to benefit from the managed infrastructure and security of Amazon Bedrock.
The customization process is divided into several steps: a training job is invoked in SageMaker to fine-tune the model with QLoRA and store the weights in Amazon S3. Once the fine-tuning is completed, the user runs the model import job using the Amazon Bedrock console. The service initiates an import job in an AWS-operated deployment account, copying the model artifacts from the user’s account to an S3 bucket managed by AWS. Once the import is completed, the fine-tuned model will be accessible for invocation.
All data remains in the selected AWS region, using a VPC endpoint, and it is possible to encrypt the model data with Amazon Key Management Service (AWS KMS) keys. Scripts for fine-tuning and evaluation are available in the Amazon Bedrock GitHub repository.
The import of the fine-tuned model into Amazon Bedrock is completed by configuring various options in the service console, importing the data stored in S3, and ensuring the model complies with the supported parameters. An evaluation of the fine-tuned model showed superior performance, generating correct SQL queries in most cases.
This release and its customization capabilities provide a powerful tool for data scientists and application developers looking to deploy generative AI models at scale, enabling more efficient and secure operations. By exploring the best practices and methodologies provided, users can maximize the potential of fine-tuned models for specific tasks such as code generation.
via: MiMub in Spanish