Training a Huggingface Model on AWS Sagemaker (Without Tears)
Liling Tan · 2025
Read Training a Huggingface Model on AWS Sagemaker (Without Tears) by Liling Tan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24098.txt.
Introduction
Liling Tan writes for researchers new to cloud platforms. Many know language models well. Few know how to train them on Amazon SageMaker. The service offers computing power that many labs lack. But its learning curve is steep. Official documents leave gaps. Users end up hunting scattered answers across the web.
Tan's demo paper gathers the essentials in one place. It walks through a first training run from scratch. The model is a Hugging Face model. The platform is SageMaker. The goal is stated plainly. Smaller teams should be able to train in the cloud without tears. The guidance reflects what newcomers actually struggle with. It turns a trial-and-error process into a guided path.
Catalog record
| CREATOR | Liling Tan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-24098
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Training a Huggingface Model on AWS Sagemaker (Without Tears) Liling Tan Abstract Developments of Large language models (LLMs) have been mainly led by resource-rich research groups and industry partners. Without the luxury of on-premise computing resources to train LLMs of growing complexity, researchers are adopting cloud machine learning services like AWS Sagemaker to train Huggingface models. While a competent researcher can train a model well on a local machine, AWS Sagemaker’s learning curve might be more challenging for non-seasoned cloud users. As a result, researchers end up scouring the web for information to fix issues and knowledge gaps in existing documentation to use Huggingface models in AWS Sagemaker. In this demo paper, to bring broader democratization of cloud adoption, we put the information to a single point such that research can start from zero to train their first Huggingface model. 1 Introduction State-of-the-art large language models (LLMs) have grown in size and training difficulty ( 21 ; 10 ; 22 ; 23 ) . While 90% of the global IT spending is still on-premises and yet to migrate to the cloud ( 6 ) , many research teams without heavy funding support do not have the luxury of on-premise computing cluster to train an LLM from scratch nor fine-tune a competitive state-of-the-art model with physical GPU machines. Unlike bare-metal cloud instances, products like Amazon Web Services (AWS) Sagemaker, Google Colaboratory, and Microsoft Azure ML Studio provide fuss-free access to Jupyter notebooks on managed virtual servers. These “Jupyter as a Service" products abstract the users away from infrastructure setup and let researchers focus on data and model improvements with minimal Infra-/DevOps (Infrastructure or Development Operations) knowledge ( 15 ; 16 ) . Machine Learning Operationalization (MLOps) is the process of bringing machine learning models into production ( 19 ; 5 ; 8 ) . An end-to-end MLOps framework on the cloud would start from research and development tools support like “Jupyter as a service" and the ability to measure, monitor and maintain the models created from researchers’ notebooks in an automated manner. Figure 1: Sample code to train a Huggingface model using AWS Sagemaker While there are a lot of books, documentation, blog posts, and a canonical online course 1 1 1 https://huggingface.co/docs/sagemaker to educate users on HuggingFace with AWS Sagemaker, there’s a lack of a comprehensive guide to training a HuggingFace model for new cloud users. Figure 1 shows the options on the Huggingface model card pages to create sample code to train a model using AWS Sagemaker. The sample code is helpful as a start, but researchers soon find that the access to features in Sagemaker is different from how one would code the same model on a local machine. This demo paper closes the knowledge gap between training a model on a local machine’s Jupyter Notebook and training on AWS Sagemaker. We will present a demo to train a HuggingFace model using Huggingface on AWS Sagemaker. 2 AWS Sagemaker AWS Sagemaker is a managed cloud specifically used for machine learning. It comes with a suite of tools that ranges from data processing, Jupyter Notebook hosting, model provenance management, model training, hyperparameter tuning, and model inference hosting. In this demo, we will focus on model training and hyperparameter tuning and briefly touch on model deployment for the models we’ve trained in AWS Sagemaker. 2.1 Amazon SageMaker > Domains > Domain: mtsdemo > Jupyter Notebook To begin, a researcher would navigate to the AWS Sagemaker tool on the search bar in the AWS console. However, unlike tools like Google Colab and Kaggle Notebook, the user is not brought directly into a Jupyter Notebook but is required to Setup a Sagemaker Domain to configure permissions and security controls. For simplicity, a new user should use the quick setup; it takes 5-10 mins for the domain to be ready after the setup, after which it can be accessed from the left panel of the console through the Domains menu. After clicking on the domain name, e.g., ‘mtsdomain‘, there will be an option to open a Studio page. And that will bring the user to Sagemaker Studio’s launcher page similar to Jupyter Lab on a local machine. Next, on the top bar of the page, the user will need to create a new Jupyter notebook file using the File > New > Notebook and that will pop-up a Set up notebook environment window and user should configure the (i) image, (iii) instance type and (ii) kernel settings that fit the needs accordingly. • Image: This is the Docker image from which you want to load your notebook. 2 2 2 If unsure, choose the latest CPU-optimized PyTorch and Python version compatible with the Huggingface transformer latest major/stable version • Instance: This is the instance type that you would like your Jupyter notebook to be hosted on. Note that this is not the instance type that you will train your model on in this demo, so a CPU instance works well. 3 3 3 If unsure, choose ml.t3.medium , the comprehensive list of instance can be found on https://aws.amazon.com/ec2/instance-types/ • Kernel: It is always Python since we work with Huggingface libraries. Finally, after setting up the notebook environment, users will see the familiar Jupyter notebook page as one would experience on a local machine, Google Colab, or Kaggle notebook. Figure 2: Shutting down virtual machine instances hosting the Jupyter notebook(s) After the Jupyter notebook runs, the charges to host the ml.t3.medium instance will be initiated, even if the notebook is not running anything. Stopping the notebook functions or ending the Jupyter runtime does not stop the charges since the virtual machine 4 4 4 For experienced cloud users, AWS calls them “ Amazon Elastic Compute Cloud Instances ”, Microsoft Azure calls them “ Azure Virtual Machines ”, Google Cloud Platform (GCP) calls them “ Compute Engine: Virtual Machines ”. We will refer to virtual hardware that AWS Sagemaker uses as “ instances ” and “ virtual machine ” interchangeably is still running and hosting notebook. To stop the charges, frugal users are advised to open on the instance, kernel, apps, and terminal session control panel on the left bar, then click on the power-off button in the RUNNING INSTANCES and a pop-up will warn the user that shutting down the instances will stop all notebooks running on that instance and all unsaved data will be lost. 5 5 5 Note that there can be multiple notebook kernels running on the same instance, esp. when you select the same instance type when starting a new notebook. As a good practice, turn off the KERNEL SESSIONS you do not need before shutting down the RUNNING INSTANCES or RUNNING APPS The instructions in this section might seem trivial to seasoned AWS and Sagemaker users, but it can be daunting to new users and students learning to use cloud resources. While the instructions and user interface (UI) will change over time, the concepts on Jupyter services in MLOps remain; there will be some (i) hardware ( virtual machine ) hosting the notebook, i.e., the compute resources needed to run a Jupyter notebook and (ii) software ( Docker image ), i.e., the operating system and libraries that we will