Intent-driven scheduling of backup jobs
Souvik Dutta and Suri Brahmaroutu · 2025
Read Intent-driven scheduling of backup jobs by Souvik Dutta and Suri Brahmaroutu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-14763.txt.
Introduction
Souvik Dutta and Suri Brahmaroutu present a framework for scheduling backup jobs by intent in a paper published in 2024. Backup jobs must fit around existing schedules in data centers. Pure global optimization can clash with time-based limits that matter more in this setting.
Their method adds new backup jobs while honoring intent-based rules and leaving current schedules untouched. Intents cover varied goals and constraints that operators state in advance. Tests on real Veritas NetBackup customer policies cover many backup scenarios. The framework held up across those cases, which supports its use in live operations.
Catalog record
| CREATOR | Souvik Dutta and Suri Brahmaroutu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-14763
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Intent-driven scheduling of backup jobs Thanks: Citation : Dutta S., Brahmaroutu S.,. Intent-driven scheduling of backup jobs, Pages…. DOI:xxxxxx/yyyy. Souvik Dutta and Suri Brahmaroutu Abstract Job scheduling under various constraints to achieve global optimization is a well-studied problem. However, in scenarios that involve time-dependent constraints, such as scheduling backup jobs, achieving global optimization may not always be desirable. This paper presents a framework for scheduling new backup jobs in the presence of existing job schedules, focusing on satisfying intent-based constraints without disrupting current schedules. The proposed method accommodates various scheduling intents and constraints, and its effectiveness is validated through extensive testing against a variety of backup scenarios on real-world data from Veritas Netbackup® customer policies. K eywords Backup job scheduling \cdot Kernel Density Estimates 1 Introduction In the data backup and recovery domain, scheduling backup operations to minimize disruption to standard business processes is critical. However, as highlighted in [ 1 ] , users tend to schedule backups at specific hours of the day or week, leading to server overload during peak times and extended periods of inactivity otherwise (see Fig. 1 ). This clustering of backup jobs often exceeds server concurrency limits, resulting in a significant number of job failures. Indeed, [ 2 ] attributes the majority of these failures to inefficient scheduling practices. Further complicating this issue is the dynamic nature of server workloads, which must accommodate a fluctuating number of clients (physical and virtual machines). These challenges underscore the need to shift from user-defined scheduling to approaches that align with user intent . Figure 1: Left: Backup client-server topology with c clients; if the number of simultaneous backups is more than the concurrency limit of the server, backup jobs for some clients may fail. Right: Backup schedule for a customer, where 72% backup jobs are clustered around midnight (from a Veritas customer’s Netbackup policy configuration. The objective of our research is two-fold: • rigorously define the constraints inherent in scheduling backup jobs, and • enable backup administrators to incrementally develop backup policies that accurately reflect strategic intent . In § 2 , we explain why scheduling backup jobs needs to be handled differently from standard constrained optimization problems. In § 3 , we formalize the problem statement and systematically list implicit and explicit constraints stemming from the user intent. In § 4 , we present the scheduling algorithm and explain the rationale for the prescription. Finally, in § 5 , we evaluate the effectiveness of our approach by applying it to backup schedules for customers with PIIs redacted. 2 User-intent for scheduling backup jobs Scheduling jobs is a well-studied problem in computer science and operations research, often approached with methods that prioritize jobs based on some metric such as importance (e.g., priority scheduling), queued time (e.g., first-come-first-served), or expected duration (e.g., shortest-job-first). These approaches work well for optimizing the order of independent jobs but fall short in enterprise backup scheduling, where an initial queue of jobs is typically predefined, and new jobs must be integrated with minimal disruption to existing schedules. Global optimization techniques, such as genetic algorithms, simulated annealing, or integer programming, aim to optimize the schedule globally, but may lead to an undesirable reconfiguration of pre-existing schedules. Furthermore, backup administrators impose domain-specific constraints (outlined in § 3 ), such as maintaining affinity between related schedules or limiting overlaps between consecutive backups. These requirements demand specialized approaches that maximally preserve schedule integrity while adapting to dynamic workloads. Figure 2: From the intent, a small language model extracts parameters for the scheduler, which in turn integrates with the existing job windows, to create a new schedule that is designed to match the user intent. In this paper, we propose an algorithm that translates the intent and constraints specified by the backup admin into a schedule that minimally disrupts existing jobs. For example, an administrator might specify: “ Backup asset VM16as_v1 four times per week with minimal overlap with other backup jobs, and no more than twice on any day. " or “ Backup VM16as_v1 during lull periods for normal jobs. " As in Fig. 2 , this intent is processed through a language model that extracts key scheduling parameters required for scheduling, such as the number of new jobs requested, time period over which they will be distributed, average intended overlap, etc. The scheduling algorithm then relates these parameters with the distribution of existing jobs to generate a new schedule that best aligns with the specified intent. 3 Problem set-up and constraints Assume we have a topology with c clients connected to a single backup server as in Fig. 1 L. Over a length of time, these clients are being backed up via n\geq c jobs. Each job is assigned a time-window w_{i} of length \delta_{i} , centered at \tau_{i} : w_{i}=(\tau_{i}-\delta_{i}/2,\tau_{i}+\delta_{i}/2),\qquad (1) within which the backup must be completed – an example is shown in Table 1 . Job window Client Start Time End Time \tau \delta w_{1} client 1 M 13:00 M 17:00 M 15:00 2.00 w_{2} W 15:00 W 18:00 W 16:30 1.50 w_{3} client 2 Su 23:00 M 00:30 Su 23:45 0.75 Table 1: Example schedules for 2 clients and 3 jobs. We want to schedule k\in\mathbb{Z}^{+} new non-overlapping windows \{w_{n+1},\ldots w_{n+k}\} for an incoming client subject to both implicit and explicit (user-defined) constraints. Implicit constraints may include hardware limitations (such as the server’s capacity to process a maximum number of concurrent jobs) or the periodic nature of all jobs within a week. Explicit, user-defined constraints can involve requirements like a minimum spacing between consecutive jobs for the same client or the need for affinity with existing schedules. In this paper, we assume that the window width \delta_{i} is fixed as \Delta>0 for all i\in[1,n+k] only for simplicity of notation. However, the logic is adaptable to arbitary \delta_{i} . 3.1 Implicit constraints 1. Concurrency limit : Let f(n;t) be the number of existing jobs ( \leq n ) at time t and \mathcal{J} be the maximum number of concurrent jobs the backup server can process. Then, 0\leq f(n;t)\leq\mathcal{J},\qquad\forall n,t. (2) 2. Disjointedness : Unless backup jobs have long durations, the k new backup windows must be mutually disjoint, meaning that w_{n+1}\cap w_{n+2}=\phi , w_{n+2}\cap w_{n+3}=\phi , and so on. Alternately, \tau_{n+i+1}-\tau_{n+i}\geq\Delta,\qquad\forall i\in\{1,\ldots,k-1\}. (3) 3. Periodicity : Enterprise backup schedules are usually periodic in time, that is: f(n;t+P)=f(n;t),\qquad P>0. (4) This requirement closes the circle on ( 3 ) as (\tau_{n+1}-\tau_{n+k})(\text{mod }P)\geq\Delta . 3.2 Explicit constraints 1. Min spacing: The backup admin may adjust the spacing in ( 3 ) to: \tau_{n+2}-\tau_{n+1}\geq\epsilon,\qquad\epsilon>\Delta. (5) This may be necessary to minimize cloud costs from performing backups, write costs into storage, and the user’s recovery point objective requirements. 2. No. of windows: The admin selects k , the number of backup windows required for the new client. Generally, increasing k adds more constraints to the scheduling problem. To ensure the problem is well-defined without violating ( 5 ), we require k\times\epsilon<P. (6) 3. Expected overlap : The admin may choose an expected overlap \alpha\in[0,1] with existing windows. For instance, if \alpha\sim 1 , the algorithm will attempt to schedule new jobs highly correlated with the time windows of existing jobs. Conversely, when \alpha\sim 0 , the new