Prediction of Spot Instances prices in AWS Automated solution for cost optimization

by Ankiola BEU Amanda KOTE

Abstract
Developing applications for managing and optimizing cloud resources is a necessity in the modern era, especially for startups looking to increase performance with a limited budget. This paper focuses on creating a system for predicting Spot Instances prices on Amazon Web Services (AWS), using a data-driven approach and artificial intelligence to help users make automated decisions on the use of cloud resources.
The problem addressed by this study is related to the volatility and unpredictability of Spot instance prices, which, although offering a low-cost alternative to on-demand instances, can lead to unexpected service outages if not managed properly. To address this challenge, this paper explores the possibility of building a reliable ML model that aims to predict Spot Instances prices by analyzing historical data. The proposed approach aims to assess whether the use of this model can help Albanian startups optimize the cost of cloud infrastructure and make more informed decisions about the use of instances.
To achieve this objective, a methodology was followed that combines historical data analysis, training a regression model with the XGBoost algorithm, and its implementation through cloud-native technologies. The model was packaged with Docker and distributed on AWS via Elastic Container Registry (ECR), while execution was performed in an AWS Lambda function connected to Amazon API Gateway via an HTTP API. Local testing and Postman testing were also performed to guarantee functionality and accuracy.
This paper contributes to the construction of a practical prototype for smart decision-making on cloud cost management and shows that the use of artificial intelligence can bring efficiency, scalability, and cost-effectiveness in the use of cloud infrastructure by startups and small businesses.

Keywords: AWS, Spot Instances, Docker, Lambda, API Gateway, ML, XGBoost, cloud-native, cost optimization, Postman.