Managing costs in the cloud and understanding resource usage can be a daunting task, especially for organizations with complex AWS deployments. AWS Cost and Usage Reports (AWS CUR) provide valuable data, but interpreting and querying that raw data can be challenging.
Recently, a solution has been explored that utilizes generative artificial intelligence (AI) to generate a SQL query from a question posed in natural language by the user. This solution can simplify the process of querying CUR data stored in an Amazon Athena database by generating SQL queries, executing them in Athena, and displaying them in a web portal for easier understanding.
The solution uses Amazon Bedrock, a fully managed service that offers high-performance base models from leading AI companies such as AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API. This service also provides a wide range of capabilities for building generative AI applications with security, privacy, and ethics.
Challenges addressed by this solution include the complexity of SQL queries, data accessibility, and ease of use. These factors can prevent organizations from effectively analyzing their CUR data, leading to inefficiencies, excessive spending, and missed opportunities for cost optimization. With the help of Amazon Bedrock’s generative AI, these processes are aimed to be simplified.
The solution is presented as a chatbot web application that allows users to ask questions related to AWS costs and usage in natural language. The application generates SQL queries based on user input, executes them against an Athena database containing CUR data, and presents the results in a friendly manner. The architecture of the solution is deployed as follows:
1. CUR data is stored in Amazon S3.
2. Athena is configured to access and query CUR data stored in Amazon S3.
3. The user interacts with the Streamlit web application and submits a question in natural language related to AWS costs and usage.
4. The Streamlit application sends the user input to Amazon Bedrock.
5. The LangChain code uses the BedrockChat class to interact with Amazon Bedrock and generate a SQL query based on the user input.
6. The generated SQL query is executed against the Athena database.
7. The query results are returned to the LangChain application, which sends them back to the Streamlit application.
8. The Streamlit application displays the SQL query and query results to the user in a formatted and friendly manner.
This solution offers multiple benefits, including simplifying data analysis, increased accessibility, time savings, and better visibility into AWS costs and usage. Thanks to the web interface, even non-technical users can leverage the valuable data provided by cost and usage reports without needing to write complex SQL queries. This enables organizations to make more informed decisions, optimize cloud spending, and improve resource management.
The AWS CUR chatbot solution leverages advanced models in Amazon Bedrock to automate the generation of SQL queries and offers a user-friendly web interface that simplifies data analysis. With this tool, both technical and non-technical users can gain valuable insights into costs and resource usage on AWS, promoting better optimization and data-driven decisions.
Referrer: MiMub in Spanish
