SmartGridDrive: an integrated adaptive Q-learning framework for precision self-parking and reverse navigation in dynamic grid environments—a proof-of-concept study Authors: Revati Raman Dewangan, Deepali Thombre, Vivek Parganiha, Monika Verma, Bhupesh Kumar Dewangan, Amit Pimpalkar, Nilesh Shelke Published: 2026 ABSTRACT AbstractThis research proposes an adaptive Q-learning framework for precision self-parking and reverse maneuvering in dynamic grid-based environments. The vehicle state includes position, orientation, and velocity and the environment model incorporates moving obstacles and dynamically changing parking slot availability. Key technical contributions are: (i) adaptive exploration where the epsilon-greedy parameter decays only when recent reward improves; (ii) look-ahead obstacle prediction using a Markov transition model with directional persistence; and (iii) priority-based Q-value updates that focus on state-action pairs with large temporal-difference errors. The hybrid control architecture combines Rapidly-exploring Random Trees (RRT) for global path planning, Model Predictive Control (MPC) for smooth trajectory tracking and Q-learning for local corrective adjustments. In simulation with a 10 × 10 dynamic grid and moving obstacles, the proposed method achieves a success rate of 92.4% ± 1.2%, collision rate of 1.8% ± 0.5%, and parking alignment error of 6.7 ± 0.8 cm. Ablation studies confirm that each module contributes significantly (p < 0.01). The method proposed in this paper is shown to be more efficient than DDPG, PPO and DQN baselines all evaluated under the same simulation settings. These results confirm the proof-of-concept (POC) viability of light-weight tabular Q-learning with predictive extensions in a simulated dynamic grid-based parking domain.Keywords: Reinforcement learning, Q-learning, Autonomous driving, Self-parking systems, Dynamic environment navigation, Obstacle prediction, Real-time path planning, Automated parking and reversingSubject terms: Engineering, Mathematics and computing Introduction The autonomous driving technology not only makes the road safer but also eases traffic congestion and makes smarter transport systems. Nevertheless, deploying self-driving cars is challenging in high-density cities. Such tasks as self- parking and reversing are especially challenging because of tight parking spaces, moving obstacles, and environmental obstacles1. Traditional path planners and controllers tend to lack the flexibility necessary for reverse manoeuvers, as well as anticipating dynamic obstacles in our application2. Deep Reinforcement Learning (DRL)3 frameworks such as DDPG and PPO4–6 achieve strong performance but require substantial computational resources, limiting real-time deployment and multi-vehicle scalability7–11 proposed a modified tabular Q-learning framework designed for grid-based environments with dynamic obstacles, specifically targeting self-parking and reverse maneuvers. Conventional path planning techniques often struggle to detect dynamic obstacles. Although autonomous driving has achieved significant progress, most of the existing technologies have serious shortcomings when deployed in such a complex environment. Older path planning techniques usually have problems with the ability to detect dynamic obstacles and to act in real time, so the approach cannot be used in unpredictable urban environments12,13. Similarly, classical control systems do not maintain steady states and are not capable of successful administration of unforeseen traffic variations or obstacle designs14. Also, some models are based on physics-driven methods for learning component degradation, but are not intended to be used in controlling the vehicle in real-time. Although traditional forms of Q-learning in RL operate well in the controlled environment they are usually not suitable in changing environments in the accuracy of such actions as self- parking and reversal. The recent deep learning methods, e.g. Temporal Convolutional Networks (TCNs) and transformers, are predominantly devised for the temporal prediction problem. These solutions provide an easy, straightforward and powerful solution to driving independently. The autonomous driving demands fine navigation in dynamic conditions, especially in manoeuvre like parking and reversing. These activities demand immediate response to the dynamic barriers and path arrangements. Reinforcement Learning (RL) and fuzzy based RL15, particularly Q-learning, provides strong potential for acquiring such behaviours through interaction with the environment. Borole6 suggested a Q-learning algorithm that involves look-ahead obstacle prediction, prioritized learning and adaptive exploration. Moreover, a well-developed reward system helps a driver to conduct safe, effective, and accurate decisions on their driving16. The ability of the improved version of Q-learning method to do self-parking and reversing behaviours is tested in simulations of other dynamic grid cases. A demonstration of the agent capabilities and to perform the specific maneuvers even under the complicated conditions was framed by Gao17. Such results support the promise of the use of RL to solve the problems of autonomous driving in dynamic conditions. Moveable grid display records all the live variations in obstacles and parking plan, thus enhancing situational knowledge9. Autonomous navigation in dense cities involves rapid re-planning based on obstacles that are in motion, and varying road conditions. The outcomes of simulations prove real-time flexibility to dynamic situations, avoidance of obstacles, and energy-saving maneuvering18. Conventional motion control techniques and traditional Q-learning find it challenging to deal with reverse maneuvers, quick adaptation, and prediction of object movement19. The DRL methods are also effective in such a case, but they are multifaceted to apply and require high performance computing and have large temporal-difference errors10,20. That is, they are not employed in either simple or time sensitive systems. Moreover, most models tend to neglect the need of quick policy change and predicting obstacles and results in unrealistic or risky behaviour. These weaknesses underscore the need to have a more efficient and adaptable learning paradigm that is proposed in this paper by the means of an adapted version of the Q-learning algorithm that would suit the task of autonomous navigation in dynamic settings. To address the challenges identified above, this work makes the following contributions: (i).Integration of Q-learning with RRT and MPC, along with extensions including adaptive exploration, look-ahead prediction, and priority updates: (a) adaptive exploration where epsilon decays only when recent reward improves; (b) look-ahead obstacle prediction using a Markov transition model with persistence bias; (c) priority-based Q-value updates focusing on state-action pairs with large temporal-difference errors. Integration of Q-learning with RRT and MPC, along with extensions including adaptive exploration, look-ahead prediction, and priority updates: (a) adaptive exploration where epsilon decays only when recent reward improves; (b) look-ahead obstacle prediction using a Markov transition model with persistence bias; (c) priority-based Q-value updates focusing on state-action pairs with large temporal-difference errors. (ii).Hybrid control architecture combining RRT for global collision-free path planning, MPC for smooth trajectory tracking, and Q-learning for local corrective adjustments in dynamic scenarios. Hybrid control architecture combining RRT for global collision-free path planning, MPC for smooth trajectory tracking, and Q-learning for local corrective adjustments in dynamic scenarios. (iii).Reward function for low-speed precision maneuvers that penalizes collisions and excessive steps while rewarding parking alignment using an exponential alignment term. Reward function for low-speed precision maneuvers that penalizes collisions and excessive steps while rewarding parking alignment using an exponential alignment term. (iv).Open-source dynamic grid simulation with configurable obstacle motion patterns, parking slot availability changes, and reproducible evaluation metrics. Open-source dynamic grid simulation with configurable obstacle motion patterns, parking slot availability changes, and reproducible evaluation metrics. The primary novelty lies in system-level integration and regime-based performance characterization rather than in proposing a new standalone learning algorithm. The proposed Q-learning framework is designed to operate in a dynamic grid-based simulated environment, enabling more precise maneuvers such as automated parking and reversing. The reward system has been well planned to be based on safety, efficiency and accuracy where the collision and superfluous movements are punished, to promote the best navigation. The proposed approach improves precision by introducing dynamic re-planning and backward navigation in confined spaces in comparison to the existing techniques used, including DDPG and MPC. Our system uses a dynamic grid which helps self-driving cars to accomplish a task like autonomous parking. The reliability of predictions achieved in our system is not simply based on estimations by the other approaches like the one used in the adoption of predictions like computer-based prediction systems that come before the car. This integration enables adaptive decision-making within the simulated navigation environment by combining RL-based control with grid-based planning. Simulation experiments show that the agent is able to respond to changes occurring in the real-time, to circumvent the obstacles, and to perform precise maneuvers in a high success rate. This paper addresses the paradigm shift represented by RL and the particular problems of automated parking and reversing, providing a simulation framework tested in large dynamic grid-structure military simulation environments in the last few years as related work. This work is positioned as an integration and regime-characterization study that combines established methods (Q-learning, RRT, and MPC) into a unified framework. Rather than introducing a fundamentally new algorithm, the contribution lies in analyzing their combined behaviour and performance under dynamic grid-based scenarios. This work is an integration and characterization study, where well-known techniques (Q-learning, RRT, MPC) are used for precise self-parking and reverse maneuvers. Instead of proposing another learning paradigm, this work contributes by analyzing their collective behaviour in dynamic grid environments and by characterizing performance in different navigation conditions. Tabular Q-learning is particularly useful when state representations can be discretized, real-time simulated performance is crucial, and we are in a structured, low-dimensional environment such as grid-based navigation. On the other hand, deep RL methods are better suited for high-dimensional continuous spaces but come at a huge computational and training cost. The remainder of the manuscript is structured as follows: Section "Related work" presents the related work along with a comparative analysis of autonomous driving methods. Section "Methodology" outlines the methodology, including problem formulation, the proposed dynamic autonomous driving architecture, pseudocode for simulation environments, RL for navigation optimization, reversing and self-parking mechanisms, and the hybrid integration of RRT, MPC, and Q-learning. Section "Results and discussion" discusses the results, covering ablation and sensitivity analysis, computational efficiency, and statistical error estimation. Finally, Section "Conclusion and future work" concludes the study and highlights directions for future work. Related work Gharbi7 provide an adjusted version of the Q-learning algorithm for self-driven moving in a changing environment with a focus on more complex tasks, such as self-parking and reversing. The system reacts to the ongoing changes in real-time by modeling the driving environment as a dynamic grid and using new extensions including look-ahead processes, priority updates, and adaptive exploration. The safe and efficient navigation is supported by a reward setup, and the ability to generate a dynamic obstacle prediction ensures resilience to dynamic settings. Simulations indicate the ability of the agent to react efficiently, prevent crashes and produce perfect parking compatibility in challenging situations. The potential of multi-agent reinforcement learning (MARL) has been studied for autonomous parking systems, and the model of parking environments as Markov decision processes (MDPs). The construction of more complex MDPs was done through a lean and iterative process that starts with simple settings with Q-learning and advances to DRL with PPO2,21,22 produced a vision-assisted DDPG framework that can handle parking in real-world situations, although it is still quite computationally intensive and has to be significantly tested on real vehicles. The most benefits of learning-based planning, as well as the use of trajectory control (via the RRR and MPC) in a single work, are introduced by the only one that does not use deep neural networks or multi-agent systems. This increases its chances of application in real-time in congested parking cases. These models did not have the ability to backtrack and manoeuvrability on the fines of the manoeuvrability regardless of the training time and incentives used. The researchers developed a customizable parking environment based on the model of an MDP with the application of technologies like Unity3D with machine learning agents, which enables testing the scales. The findings display the high success of parking up to seven vehicles at 98.1%, and this success significantly exceeds conventional individual agent methods. New behaviours, e.g., leaky collaboration and competitive interactions, were found, and this underscores the complexities of multi-agent systems. The discussion has limitations such as scalability and implementation of the simulation in the real world, which reflects a chance of additional research. To create a purposeful simulation environment based on a simplified version of the original control and input data of a seven-sensor set was created with PyGame. This was made possible by the environment facilitating repeated DQN testing and a variant of DQN with priority-based action selection. The method involves learning models with different action spaces (left, right, or forward), employing normalized sensor inputs as the observation space, and creating a reward system to encourage smooth and collision-free driving23. Vanilla neural networks and customized DQN were used to carry out experiments. The findings revealed the modified DQN was better than the baseline models with an average reward of 40 versus 25 using normal DQN and reduced training time to 10 to 4 h with the assistance of a GPU. These findings outline the importance of specialized action selection systems and the potential of RL in self-driving simulations. The tabular Q-learning, as compared to DQN, is computationally less, and it includes reverse motion and self-parking accuracy. Zhang11 explores the application of RL to solve the issues that are posed by conventional route planning and tracking policies, in autonomous parking systems (APS). Existing APS systems are likely to generate errors due to car nonlinear dynamics and imprecision of trackers. The paper gives an RL-powered end-to-end parking system, which employs the algorithm of DDPG to obtain optimal parking in complex and constrained conditions. The system identifies a parking slot in real-time by using a combination of vision-based sensors and car chassis information, which is facilitated by extended Kalman filters. To avoid the limitation of action-based agent training, DDPG algorithm was used to teach the agent using continuous instruction in steering. Training and validation would be supported by a custom simulation and real-vehicle environment, with human directioning in the early exploration phase and late control tactics to support learning. The RL-based model improved over the conventional APS techniques by reducing the tracking errors and generating less jagged parking paths. Shahi24 proposed autonomous rear parking (ARP) system comprises of many components like OpenAI gym environment is a simulation-based model and it is used to train the RL agent. MPC-based route makes the vehicle pass the preferred route in a smooth and safe manner and the MDP-based RL appraises and optimizes the ARP system by teaching the agents about the use of value and policy iteration methods. Work done recently in RL-based autonomous parking in a number of directions. (Dewangan25) suggested a privacy-sensitive26,27 and secure hyper parameter-tuned RL algorithm to find robot paths in dynamic environments.20,28 suggested evolutionary-guided RL to optimize both policy and hyper parameters jointly in the context of parking tasks. Sharma et al.29 proposed a real-time RL that uses dynamic obstacle management which has been validated in high-fidelity simulation. We have suggested that Jiang et al.30 can make it possible to navigate and park cars with the help of hybrid and lightweight RL methods. They produced the hybrid policy RL planner to address various parking scenarios. Zhang et al.31 created deep RL approaches to the generation of continuous trajectories in the parking domain. Yang et al.32 introduced a safe, efficient, and generalizable parking, RL, which can learn based on pre-collected parking demonstrations. Wang et al.33 integrated Light Detection and Ranging (LiDAR)-based representations with hybrid RL to enhance transfer to reality. Almousa et al.34 showed a reverse parking system35 on a set-wide challenging ware prototype, and this filled a simulation-reality gap between the challenging ware prototype and simulation. Despite this integration improving real-time replanning, it could not operate at low speeds and relied on fixed assumptions. Obstacle-aware RL models ( with dynamic updates were investigated in recent studies on dynamic obstacle prediction and adaptive planning36. Nevertheless, they either lack simulation features for parking alignment or concentrate on single-agent control. Although current research has significantly advanced, several shortcomings still persist. Many methods struggle with scalability when processing high-dimensional or fast-changing data settings37. Some are less efficient for actual use, as they lack flexibility under dynamic circumstances or ignore real-time limitations. Some techniques also depend on static setups or idealised assumptions that conflict with the complexity of the real world. These limitations motivate the proposed framework. While Soft Actor-Critic (SAC) demonstrates stability in continuous control, our discrete action space makes it less suitable. DDPG and PPO were selected as deep RL baselines due to their prevalence in prior parking studies. Unlike recent efforts that use DRL frameworks like DDPG for continuous control in parking applications or MARL via PPO to achieve collaborative self-parking. Table 1 shows the comparative analysis between our work and existing works. The analysis shows that the improved Q-learning method performs well in dynamic environments due to its adaptive exploration and obstacle prediction capabilities. Methods such as DDPG and MPC have advantages in continuous control and path tracking, respectively, but have weaknesses in dynamic adaptation and backward navigation. PPO demonstrates its performance in multi-agent scenarios. Traditional APS lacks the real-time adaptability of RL methods. This paper employs a lightweight modified tabular Q-learning approach well-suited for navigation environment. In the prior work, using somewhat naive DQNs, adaptive behaviour was not afforded with such simplicity, which is required in dynamic and obstacle-rich settings. Here, adaptive exploration has been introduced along with look-ahead mechanisms for dynamic obstacle prediction and priority updates for high-variability states so that the agent can make decisions in a safe and quick manner. Integrated control has previously been demonstrated by combining RRT with MPC for trajectory following but typically under static conditions. Our contribution lies in bridging planning and control through Q-learning enhancements, enabling efficient self-parking and reversing even in volatile, partially known grids without relying on deep networks or collaborative systems. Our work differs from these by focusing on a lightweight tabular approach with explicit look-ahead and priority updates, targeting resource-constrained embedded systems rather than high-performance computing platforms. Table 1Comparative Analysis of Autonomous Driving Methods: Our Modified Q-Learning Approach in Relation to Alternative Techniques.FeatureModified Q-Learning (Dynamic Grid)DDPG3MPC35PPO10Traditional APS38Primary ApplicationSelf-parking, reversing in dynamic gridsContinuous control (steering), parkingPath tracking, parkingMulti-agent scenarios, parkingGeneral parking manoeuvresEnvironment RepresentationDynamic grid with obstaclesContinuous-state spaceGeometric or discretisedComplex simulation environmentSensors and vehicle chassis dataStrengthsAdaptive exploration, obstacle prediction, priority updates, backward navigationContinuous-action space, end-to-end solutionAccurate path trackingHandles complex multi-agent interactionsMeets standards for accuracyWeaknessesScalability to very large grids, requires careful reward tuningStruggles in discrete-action spaces, dynamic obstacle predictionLacks backward navigationComputationally intensive, single-agent focusNonlinear dynamics, tracking errorsSuccess Rate/AccuracyHigh (simulations show adaptability)Variable94.5% accuracyHigh (98.1% in specific simulations) ± 3° inclination accuracyAdaptation to DynamicsReal-time updatesLimitedReal-time control adaptationDynamic re-planningRequires recalibrationCollision AvoidanceEffective (reduced collisions)Can be challengingImplicit through path planningFocus on safety, collision avoidanceReliant on sensor accuracySpecific FeaturesLook-ahead function, priority updates for dynamic statesVision-based sensor fusionRRT for path planningCustomisable parking environmentKalman filters Comparative Analysis of Autonomous Driving Methods: Our Modified Q-Learning Approach in Relation to Alternative Techniques. Methodology Problem formulation The proposed study characterizes coordinated behaviour by combining path planning, control, and RL modules for maneuvering the robot autonomously in dynamic environments. We formulate the autonomous parking and reverse maneuvering problem as a finite-horizon discounted Markov Decision Process (MDP) defined by the tuple: Where is the state space, is the action space, is the state transition probability function, is the reward function, is the discount factor and T is the episode horizon. Let the environment be represented as a discrete grid: where N=10 in this implementation and each cell at simulation time step t belongs to one of three mutually exclusive sets. With : free space (traversable), : obstacle-occupied space (non-traversable) and : parking slot space (traversable, terminal). These sets satisfy , , , and . The vehicle state at time t is defined as: where : discrete grid coordinates, (four cardinal orientations: east, north, west, south) and : speed, with grid cell per time step. The total number of discrete states is:, so for N=10, |Θ |=4, : . The vehicle can execute one of five discrete actions at each simulation time step: The transition effect of each action depends on the current orientation : The Table 2 captures how motion and rotation are decoupled: translational actions (forward/backward) affect only position, while rotational actions (left/right) affect only orientation, and the stay action results in no state change. The backward action enables reverse maneuvering, which is essential for self-parking in confined spaces. The transition from state to under action is deterministic with respect to vehicle kinematics but probabilistic with respect to obstacle movement. Let denote the vehicle position after applying action ignoring obstacles. The actual next state is: Table 3Hyper parameters Summary.ParameterSymbolValueGrid sizeN10Discount factorγ0.95Learning rateα0.1Exploration (initial)1.0Exploration (final)0.01Exploration decay0.995 per episodeStep penalty0.1Collision penalty10.0Alignment reward5.0Goal reward100.0Alignment sensitivityα2.0Episode horizonT200Obstacle persistence0.7New obstacle probability0.05 Hyper parameters Summary. Upon collision, the episode does not terminate but incurs a large negative reward. Each obstacle moves independently according to a Markov process. Let denote the position of an obstacle at time t. The transition probability to is: where is the movement direction vector, is the previous movement direction of the obstacle, is the persistence probability and if no previous direction exists, for all . An obstacle that moves outside the grid boundaries is removed from . New obstacles may appear at boundaries with probability per time step. The immediate reward is defined as: Each penalty component is defined as follows: Distance penalty (negative quadratic) where is the target parking slot positions. Step penalty (per-action cost) where and 1[· ] is the indicator function. Collision penalty with . Parking alignment reward where , α =2.0, is the angular difference modulo 2π and is the target orientation (typically for perpendicular parking). Goal achievement reward with and radians (10 degrees). The agent seeks a policy that maximizes the expected discounted cumulative reward over an episode of length T: The optimal policy satisfies: The corresponding optimal state-action value function is defined by the Bellman optimality equation: An episode terminates when one of the following conditions is met: Success: The vehicle reaches the target parking slot with orientation error . Success: The vehicle reaches the target parking slot with orientation error . Timeout: The episode reaches the maximum horizon T=200 steps. Timeout: The episode reaches the maximum horizon T=200 steps. Stagnation: The vehicle has not moved for 50 consecutive steps (excluding ). Stagnation: The vehicle has not moved for 50 consecutive steps (excluding ). The success rate is defined as: The important hyper parameters applied in the proposed model can be summarized in Table 3 and they are grid size, learning and exploration parameters, reward and penalty parameter, and environment dynamics that control obstacles and length of an episode. Figure 1 illustrates the proposed autonomous driving architecture that is hierarchical, which is dedicated to dynamic grids. It begins with the complex dynamic grid world, which is the input in which stationary and moving barriers, such as pedestrians are projected to a high-dimensional state of the environment as a state vector. It starts with an upper-level planning of RRT in the plans of a feasible path. SmartGridDrive: A Dynamic Autonomous Driving Architecture. Path planning is used to process this data, producing a global collision-free path using an RRT Planner. To implement, the control module employs an MPC controller to deal with trajectory tracking subject to constraints. MPC then makes the vehicle adhere to this path in a smooth manner with physical limitations being observed. In order to dynamically adjust to the varied environmental conditions, the adjusted Q-learning algorithm enables the RL agent to improve its navigation performance. Simultaneously, the learning module uses adaptive Q-Learning to optimize actions, with epsilon-greedy exploration, look-ahead prediction and priority-based TD error updates. The architecture ends with optimised output behaviour, demonstrating effective results such as collision avoidance and accurate parking. This is confirmed by the performance metrics such as alignment error and reward convergence as being a resource-efficient approach that is real time and can operate autonomously. The aim of this hybrid is to achieve the achievement of safe, efficient and accurate self-parking and reversing in both static as well as dynamic grid layouts. The suggested method effectively optimises the behaviour of vehicles through a combination of path planning, control, and RL. A custom simulation environment is built based on the 2D grid representation. Each grid cell represents a parking target, obstacle, or free space. The car is described as a moving agent, which has a state space consisting of position, orientation, and speed. Real-world uncertainties are represented by dynamic obstacles, including the change in the availability of parking slots and moving obstacles. A reward system is used to strengthen the desired behaviours, including achieving the goal, and discourages collisions, delays and wasteful paths. Model incorporates real-world complexity by introducing moving obstacles and dynamically changing parking spaces into the model. The state space is defined by the vehicle’s position, orientation, and speed, while the action space consists of control inputs such as steering angles, acceleration, and reversing. This would mean developing a reward system that would promote the best behaviours. Positive reinforcement is created by reducing the space to goal and making successful parking. The negative rewards are the collisions, out of course directions, and ineffective reverts. The reward system stipulates the broad architecture on how rewards and penalties are awarded to individuals when learning but the immediate reaction is the reward that is made available. It also directs the agent to favour effective, riskless, and goal-oriented behaviours. In particular, the last reward is defined as: where captures the fundamental RL reward, and I(s,a) is the incentive function that adaptively adjusts the reward according to urgency and resource availability. This addition enhances the transparency of the framework. where 1(·) are indicator functions and λ terms are weighting coefficients controlling step cost, collision penalty, alignment reward, goal achievement, and time-critical bonus. Initial route planning by RRT is used as a reference path to generate collision-free paths. The algorithm scans the grid to generate a tree of potential paths that the vehicle will take to reach the point of destinations. Reconfiguration the RRT-based pathway in regard to grid changes, e.g., a shift of obstructions or a shift in parking slot position gives the tracking of the motion smooth and steady by tracking reference path given by the RRT. The strategy is based on the physics of the vehicle including turning radius and speed limit, which provides realistic and realistic driving experience39. MPC adapts the control inputs in simulated real-time in response to any grid variations or route changes caused by external sources. This process adaptively adjusts the exploration rate according to the agent’s recent learning performance in various areas, so that more intelligent balance between exploration and exploitation is achieved by optimizing e-greedy policies. There is exploration that evolves with the learning of the agent. We view success of learning as an outcome of improving in the past k tries this ensures that the exploration rate decreases only when performance improves, while preventing unintended increases due to negative reward variations. To improve the decision-making process in a dynamic grid framework, a look-ahead40 function is used to predict the consequences of potential actions before they are executed. This function simulates future grid states within a fixed depth. This enables the system to evaluate not only the immediate results but also the downstream impacts, such as resource contention, load balancing, or potential deadlocks. By incorporating this predictive capability, the model selects actions that optimise long-term performance rather than short-term gains. This look-ahead scheduling mechanism improves the robustness and efficiency of grid-based task scheduling and allocation under dynamic conditions. The look-ahead function predicts the outcome of actions up to depth d by simulating probabilistic obstacle transitions. For each action an in state st​: where is derived from a Markovian obstacle transition model . The augmented Q-update becomes: where represents the expected future, reward estimated via the look-ahead mechanism and is incorporated as an auxiliary reward term rather than replacing the standard Q-value. While traditional DRLapproaches will model the entire environment, this light-weight probabilistic look-ahead mechanism does not need to construct the whole but focuses on capturing future state transitions. Through the use of an efficient Markovian obstacle transition model, it allows for well-informed real-time decision making under risk with much lower computational cost. This constitutes a key methodological contribution of the proposed framework. We use a priority-based update scheme, where state-action pairs are updated based on the size of their temporal difference error. This prioritisation speeds up convergence by focusing on high-impact updates. The incentive adjusts the reward according to whether a task is urgent or not and the performance of the node to ensure that the agent does not overload resources but executes those tasks that require urgent attention. Additional rewards or penalties by this function are based on obstacles like urgency of the task, availability of the node and previous performance. The framework motivates nodes to follow important activities, avoid resource hoarding by ensuring there are incentives within the reward system and avoid workload imbalances. This is an improvement of the system of rewards by changing values dynamically based on urgency and availability of resources thus maximizing effectiveness in making decisions in highly transforming environments. The algorithm described above prepares an environment of a 2D grid-based simulation to recreate a real-life navigation environment. The grid contains a goal purpose, an automobile, hindrances and free spaces. Basic functions, which are required by the algorithm, include the following: (i).reset(): to restart the simulation. reset(): to restart the simulation. (ii).step(action): to award prizes based on the cell type (obstacle, target or free) and to refresh the position of the vehicle. step(action): to award prizes based on the cell type (obstacle, target or free) and to refresh the position of the vehicle. (iii).render(): It displays the grid onto which we can see the vehicle, any obstacles, and the location of the target. In this manner, we are able to observe the action as the computer simulation process unfolds. render(): It displays the grid onto which we can see the vehicle, any obstacles, and the location of the target. In this manner, we are able to observe the action as the computer simulation process unfolds. The grid values represent the various types of cells, that is: 0 = Free Space (white), 1 = Obstacle (black), 2 = Target (green). The x and y value are grid indices. RL for Navigation Optimisation An MDP defines the system. The agent, through the interaction with the grid, learns its policy of navigation through the use of Q-learning. The agent can make up of the position, orientation and the immediate environment of the agent. The action space involves direction movement and reversal. Rewards are given due to safe and effective movement and successful parking with fines taken on collision, excessive movements and any violation of the boundaries. An e-greedy strategy advances the Q-values to strike a balance between exploration and exploitation (Lowe41). The above algorithm teaches an RL agent to travel from a starting location to a target location while avoiding obstacles. It utilises a ε-greedy technique for action selection and changes the Q-values depending on the reward feedback. The abstraction of a 10 × 10 dynamic grid creates a POC for precision maneuver decision-making in a controlled dynamic environment. While this is a simplified representation of self-parking and reversing actions such as obstacle avoidance, alignment and constrained navigation, the MPC module provides kinematic constraints that further enhance the realism of the motion. Reversing and Self-Parking Mechanisms Special attention is paid to the reversing and self-parking which are two difficult maneuvers at low speed and demand precision. In case forward movement is interfered with the RL agent will use a reversing policy to navigate backwards. Other incentives of the correct alignment in parking edges as well as punishment against improper alignment or ineffective parking efforts will also serve to perfect the parking conduct. Such systems make sure that the agent trains to properly and safely navigate the car into tight areas (Alamri42). The above algorithm enables the car to self-park, which means visiting a designated parking point with the help of Q-learning. The reward of perfect parking and the penalty of crashing are used to encourage the agent to reach an optimal method of parking. That shows the possibility of using RL in the case of autonomous self-parking, focusing on proper alignment, avoiding obstacles, and optimizing the route. The above algorithm based on Q-learning, which trains the agent to reverse into a parking position or move out of a narrow environment. It examines retrogressive movement, counter shading and optimal alignment. It supplements the concept of forward moving and parking with the backward moving movements, providing total two-way control in difficult situations, like in the city. Path planning using rapidly exploring random trees and control using MPC Initial path planning uses the RRT algorithm to build a tree of paths that are feasible by sampling random points and linking them with motions without collisions to avoid obstacles this procedure is known as initial path planning; upon finding a path to the destination the algorithm is used as a guide to the vehicle43. The RRT is continuously updated within a dynamic environment in order to address any changes in the positions of the obstacles or the goal point so that the vehicle is able to continuously adjust during execution. MPC is used in tracking of a trajectory by tracking the reference path calculated by RRT with optimal control inputs in form of steering angle and speed calculated on a prediction horizon, constraints such as acceleration limits and turning radius are considered, and control actions are adjusted in real time to adapt to environmental changes and dynamic obstacles to ensure minimal deviation of the planned path. The pseudocode implements RRT to find a collision-free route between the vehicle’s start and target points. By picking random sites and extending pathways from the closest existing node, it repeatedly expands a search tree. RRT first gives a route plan that serves as a reference trajectory in the simulation. It handles path finding in situations with multiple obstacles and dynamic changes. This algorithm enables the vehicle to follow a predetermined course smoothly and securely. It employs a kinematic model of the vehicle and generates optimum control inputs (steering angle, velocity) across a future prediction horizon, utilising optimisation. MPC ensures the vehicle follows the intended RRT course with realistic motion, taking into consideration vehicle characteristics like turning radius and speed constraints. This facilitates the reliable and exact execution of pathways in simulation. Priority update mechanism and obstacle prediction module Prioritized experience replays samples experiences from a buffer and does not use real-time Temporal Difference (TD) error; instead, its samples errors from past experiences. We employ online priority updates that do not require replaying historical data. This approach is particularly applicable to dynamic grid scenarios, such as self-parking, where it is crucial to promptly update critical state-action pairs in response to rapidly changing environments. Unlike experience replay, updates are performed online using the TD error: Priority is proportional to TD error magnitude: The weighted update is: This ensures that states with large errors receive stronger updates in real time. Though in model-based RL, the full transition model of the environment is usually being learned, we do not do so and instead rely on the shallow look-ahead module. It is a predictive near future movement of obstacles based on probabilistic transitions on the basis of the patterns observed recently, which allows making safer decisions without voluminous modelling. Consequently, it is suitable to lightweight and real-time navigation. When paired with the ability to predict the future obstacles (foresight) and dedicated learning (priority updates), the agent is resilient and responsive. This algorithm is a gradient strategy, and it relies on attractive and repulsive forces to steer a robot (vehicle) to a destination and avoids moving impediments. The next direction of the move at any step is determined by the entire gradient vector. This makes them safer and more responsive, dynamically changing the path of the vehicle in real time, making sure it does not hit anything that is moving which is extremely important in autonomous driving scenarios in the real world. An online update mechanism that prioritizes state-action pairs with significant TD errors is adopted for faster convergence in SOC MDN, but an experience replay buffer is not needed. Because updates are limited by the learning rate α and TD error , this method retains convergence properties analogous to those of standard Q-learning under normal assumptions. Hybrid integration of RRT, MPC, and Q-learning An effective system for self-driving in crowded environments must be capable of planning, executing, and adapting in real time; this is achieved through an integrated approach where RRT determines a feasible route, MPC ensures accurate trajectory tracking, and Q-learning enables corrective actions when deviations or uncertainties arise. The last control applied on the vehicle combines both trajectory tracking of the MPC and correction of the Q-learner thus, ensuring global feasibility (RRT), smooth control (MPC), and simulated dynamic responsiveness (Q-learning). A discrete Q-learning agent produces corrective actions, such as local steering or reversing adjustments, which are treated as control corrections and combined with the continuous MPC outputs via the final control law. MPC continuously delivers nominal inputs for trajectory tracking, while Q-learning prepares adaptive local adjustments to dynamic obstacles, forming a two-layer closed-loop control structure. Figure 2 shows how RRT, MPC, and Q-learning work together to control self-driving cars. RRT planning makes the main path, and the MPC controller keeps the car on track smoothly. Then, the Q-learning part makes small changes to fix obstacles as needed. In the end, all these controls are put together to steer the car well. RRT makes a path without collisions from where you start to where you want to be. Global collision-free path . This path is just a list of points that work around obstacles that do not move and obstacles that do move. At each timestep t, MPC computes a control input to follow. MPC minimizes a cost function: where H is the prediction horizon, Q and R is weighting matrices, and is the RRT reference. The Q-learning agent observes the dynamic grid around the vehicle. The Q-learning module monitors a local grid (a 5× 5 window around the vehicle) to detect unexpected obstacles or parking slot changes. When unexpected events occur (moving obstacle, blocked slot), it computes a corrective control adjustment , where is the learned policy for local manoeuvres (e.g., reversing, fine alignment). The final control applied to the vehicle combines MPC’s trajectory tracking and Q-learning’s correction , where ​ ensures smooth trajectory tracking and provides simulated real-time corrective adjustments in dynamic scenarios. This ensures that the system respects global feasibility RRT, smooth control MPC, and online adaptation Q-learning. Hierarchical Control System for Vehicle Dynamics. Figure 3 indicates the control architecture in levels of different degrees of hierarchy. In this, global collision free path is produced by RRT module which is followed by MPC based controller with a finite prediction horizon (H). Q-learning module suggests local corrective measures as based on real time grid observations. Q-learning with MPC has been incorporated into MPC and is flexible by combining new Q-value modifications and probing inputs. Integrated Architecture of RRT–MPC–Q-Learning Framework. Evaluation metrics Performance is measured as a number of several measures: (i)Success rate: Percentage of the cases when the car reaches the parking goal. Success rate: Percentage of the cases when the car reaches the parking goal. (ii)Average steps: Measures the efficiency of the navigation. Average steps: Measures the efficiency of the navigation. (iii)Collision rate: Represents safety and strong. Collision rate: Represents safety and strong. (iv)Alignment error: measures the accuracy in which the car fits in the parking slot. Alignment error: measures the accuracy in which the car fits in the parking slot. These parameters are tested and studied in the static and dynamic grid topology to validate the flexibility and effectiveness of the proposed technique. The proposed approach is first evaluated through extensive computer-based simulations under a variety of operating conditions to assess its robustness and performance. Following the training of the RL model, its effectiveness motivates future real-world validation by testing the vehicle’s performance using actual sensors and control systems, as demonstrated in Cleverciti44. All reported metrics were evaluated over repeated runs to enable statistical significance testing. All experimental results were averaged over 100 independent runs, each using a different random seed, unless otherwise noted. Results and discussion All performance metrics (success rate, alignment error, collision rate and uncertainty) are reported respectively as mean ± standard deviation over 100 independent runs with different random seeds according to a consistent evaluation protocol to make results in the manuscript more comparable unless specified otherwise. To convey variability across runs we report mean ± standard deviation (SD), and do not report confidence intervals (CIs) because the focus of the paper is comparative consistency rather than estimation of intervals. The simulation employs a reward function to evaluate movements and penalize collisions. As discussed in earlier sections the RRT based path planning algorithm is a useful method of generating collision free paths within a grid-based environment since it progressively develops a tree by iteratively searching at random points. It makes sure that it has a valid navigation and there are no obstructions and backtracks to form an ideal route to the target. The visualisation shows the growth of the trees and their eventual route. The method is appropriate in real time autonomous navigation problems in structured environments. Figure 4 shows a grid environment that models’ environment of 2D navigation with obstacles, a target, and a mobile vehicle. The environment offers ways of resetting, running operations (up, down, left, right), and visual representation of the environment with Matplotlib. Figure 5 give an implementation of path following with MPC of a vehicle that follows a reference trajectory. Each grid cell is color-coded based on dynamic obstacle probability, where lighter colours indicate lower movement probability and darker regions represent highly dynamic obstacles. Blue → static (P = 0), Yellow → moderate (P≈0.5), Red → high dynamic (P≈1). The MPC-based path-following algorithm employs a kinematic vehicle model to project the vehicle state on the basis of control inputs (steering angle and velocity) and economic the controls on a prediction horizon with the aim of minimizing the difference between the predicted and reference path. This is a cyclic update of the controls to facilitate the tracking of the trajectory. The results plot the actual route of the vehicle in reference to the reference path. Visualisation of a 10 × 10 2D Grid Environment for Autonomous Navigation. Path Planning Using Rapidly Exploring Random Trees. The figure also shows iteration count and cumulative path cost, illustrating convergence of the RRT-based planner toward an optimal collision-free path. Figure 5 shows the kinematic model of the vehicle which is utilized in the generation of the trajectory taking into consideration the vehicle motion constraints like steering angle and wheelbase. The blue route is the reference route that has been planned without considering any dynamic obstacles whereas the red route is the actual traffic route that is adjusted to the dynamic aspects while navigating. As Fig. 6 reveal, the optimization of navigation with the help of Q-learning is effective to train an agent to navigate through a grid environment avoiding obstacles and reaching the target. With the help of the Q-values, continuously updated and an e-greedy policy, the agent is able to learn an optimal path by balancing exploration and exploitation. The algorithm is graphically presented and visually shows the path learned through interaction with the environment. This method is well-suited for autonomous navigation in structured and obstacle-rich environments. Figure 7 illustrates that the green dot would be the initial position of the vehicle, and the red flag is the desired position that the vehicle will travel to on the dynamic grid map. This module helps improve the navigation behaviour by allowing the agent to explore the most efficient/safe path during trial and error across many episodes. It offers basic navigation abilities in dynamic as well as structured surroundings. The latter can be further maintained to more sophisticated techniques, including MPC, or by implementing libraries such as the Robot Operating System (ROS) to detect and avoid obstacles in real time. Path Following Using MPC. The prediction horizon (H) and cost minimization behaviour are illustrated, showing convergence of the control inputs toward the reference trajectory. RL for Navigation Optimisation Using Q-Learning. The environment is represented as a grid with a start point, obstacles and a parking spot as shown in Fig. 8. Q-learning modifies the Q-table to find the best navigation policy. The e-greedy approach balances exploration and exploitation to allow the efficient exploration, and testing makes use of the trained Q-values to perform parking. The coordinates that have been depicted in the route to the parking space are in (x, y) format which is compatible with the Cartesian coordinate notation. Speaking of the total reward plot, the non-convergent behaviour in episodes 100 to 1,000 is due to the high level of variability of dynamic environments, thus leading to variations even after primary learning. Figure 8 orange ovals show a high-concentration obstacle area that the planning algorithm is keen to avoid, which means a dynamically moving area with uncertain movement of objects. Figure 9 shows which measures the percentage of episodes in which the agent successfully reaches the target, indicating its overall effectiveness. Reflects the efficiency of the agent’s learned policy in completing tasks, while captures the agent’s overall performance based on accumulated rewards. Dynamic Obstacle Avoidance. Agent’s Path Tracking for Self-Reversing Using Q-Learning. Figure 10 provide the Q-learning applied to self-reversal in a grid environment, in which an agent learns to go toward a target and avoid obstacles. The characteristics of visualization are the optimal policy of the agent, the frequency of actions, and the route that the agent follows in training. With these improvements, one gains a better insight into how the agent learns and how its exploration and exploitation approaches perform. As seen in Fig. 10, the reward amounts do not smooth out and become level quite as they should. We did all tests several times starting with various points and plotted the average reward in a line demonstrating the possible outcomes. Implementation of a Grid-Based Self-Parking Simulation Using RL (Q-Learning). The reward curve is shown with shaded variance across multiple runs, highlighting convergence behaviour and stability of the learning process. We flattened the reward graphs to view the general trend with the ups and downs. We experimented on two methods of doing so. The predictability of the actions of the agent and the frequency with which it wins were other measures of whether it had a consistent strategy. What we discovered is that although the reward each round varies (since the environment is random and sometimes obstacle configurations change unpredictably), the smoothed-out rewards and win rate increase and then levels off at approximately 700 rounds. Moreover, the activity of the agent becomes more predictive and less random, i.e. it is stabilizing upon a particular approach. We are then less likely that the reward changes that we continue to observe, such as the position of obstacles or the position of the vehicle, initiate the change due to the environment being random, rather than because the agent was still exploring the environment. We included the information on the number of tests (1000 runs) we performed, the way we smoothed the graphs and measured the predictability in the extra info so that you could replicate this. Figure 11 shows that throughout the 100 episodes, the agent achieved a success rate of 99%, consistently meeting the goal. The average number of steps per episode was only 8, indicating efficient navigation. The agent identified the optimal course of action, successfully avoiding obstacles and effectively reaching the objective, with an average reward of 93.00 and no penalties. Figures 11 and 12 illustrate trajectories and obstacle densities using a color-coded representation, where the blue lines denote the vehicle’s predicted path, while green and pink regions indicate obstacle zones with varying dynamic intensity, with red corresponding to highly dynamic areas and yellow representing moderately dynamic regions. Table 4 represented the evaluation results of both test cases as discussed earlier. Performance of the Trained Agent in the Self-Reversing Task (Test-1). Measuring the Agent’s Performance in Terms of Its ability to Navigate to the Target, Avoiding Obstacles, and Avoiding Inefficient Paths (Test-2). Table 4Evaluation results of Test Performance.Evaluation ParameterTest-1Test-2Total episodes100100Success rate100%90%Average steps per episode8.008.00Average reward per episode93.0093.00Average penalty per episode0.000.00 Evaluation results of Test Performance. Figure 12 shows that in the controlled single-task self-reversing experiment, the agent achieved 100% success. (The 99–100% success value corresponds to a controlled self-reversing test case, whereas 92.4% represents the overall benchmark performance under dynamic grid scenarios.) This result represents a task-specific benchmark and differs from the overall dynamic-grid success rate of 92.4% reported in Tables 5,6 and 7. Table 5Ablation Study Comparing the Performance of the Full Q-Learning Model against Variants with Key Components.VariantSuccess rate (%)Average stepsAlignment error (cm)Collision rate (%)Full model (all modules active)92.4 ± 1.2%38.2 ± 1.2 steps6.7 ± 0.8 cm1.8 ± 0.5%Without adaptive exploration84.6 ± 2.143.9 ± 1.58.2 ± 1.04.5 ± 0.7Without priority updates85.1 ± 1.941.7 ± 1.38.5 ± 0.93.9 ± 0.6Without look-ahead prediction80.2 ± 2.346.3 ± 1.89.7 ± 1.15.1 ± 0.8 Ablation Study Comparing the Performance of the Full Q-Learning Model against Variants with Key Components. Table 6Benchmark Tests on NVIDIA A100 GPUs.MetricDynamic Grid Q-learningPPO baselineImprovementDQN baselineImprovementTraining time (hrs)4.28.7 ± 0.552% faster9.8 ± 0.557% fasterMemory footprint (GB)1.73.1 ± 0.245% lower3.4 ± 0.250% lowerSteps/second2,1481,256 ± 10071% higher1,203 ± 10078% higher Benchmark Tests on NVIDIA A100 GPUs. Table 7Summary of related work performance and matched baseline comparisons.AlgorithmSuccess rateCollision rateAverage stepsAlignment error (cm)Proposed QL92.40%1.80%38.26.7DDPG3688.10%3.20%41.98.3PPO4085.60%4.70%45.19.1A3C System4383.20%5.10%47.610.4 Summary of related work performance and matched baseline comparisons. Ablation and sensitivity analysis We performed ablation research (All results use identical units and harmonized uncertainty notation for consistency.) by turning off each of the three suggested enhancements from the modified Q-learning framework adaptive exploration, priority updates, and look-ahead obstacle prediction one at a time to assess each one’s unique contribution. Table 5 displays the outcomes of training and testing each version under the same dynamic grid circumstances. This confirms that each module contributes independently and synergistically to performance improvements. All the uncertainty values in Tables 6, 7 and 8 are always shown in mean ± standard deviation, while the alignment error is consistently written in cm (centimetres), and the orientation error is defined separately in ° (degrees) if relevant. All results are reported as mean ± SD over 100 independent runs with different random seeds to ensure statistical reliability. Each module contributed significantly to the overall system performance. In particular, adaptive exploration improved both convergence speed and stability, while the look-ahead component had the strongest impact on obstacle avoidance. A sensitivity analysis was conducted by varying two key hyper parameters viz, the collision penalty and the exploration decay rate (ϵ-decay). Results showed that insufficient collision penalties led to unsafe, high-risk behaviour, whereas stronger exploration settings enabled faster convergence, as illustrated in Fig. 13. The best performance was achieved with a collision penalty of 10 and an ϵ-decay of 0.995. This is shown in the right half of Fig. 10 in which an appropriate punishment of crashes, without affecting speed, leads to safer navigation. The ablation study also supported the fact that with severe dynamics or the absence of look-ahead prediction the performance sharply declined emphasizing the significance of each of the modules when managing failure prone situations. Table 8Comparative Analysis of the Proposed and Recent RL-Based Approaches, Grouped by Simulation-Based and Real-World Applicability.MethodDynamic obstacle handlingSelf-parking accuracyAdaptive learning (RL)Simulation realismMulti-agent capabilityReal-world applicabilitySimulation-Only MethodsProposed (Modified Q-learning + RRT + MPC)Look-ahead and real-time updatesFine-grained alignment and reversingAdaptive exploration, priority updatesModerate (grid + dynamic entities)Not addressedIn progress (simulation validation only)MARL based2Collaborative movementHigh (98.1% success rate)Deep RL (PPO)High (Unity3D)YesSimulation onlyModified DQN (PyGame, 2024)Priority-based action selectionLimited to forward motionSensor-based input + custom rewardsLow (simplified controls)NoSimulation onlyRRT + MPC + RL24Hybrid real-time re-planningHigh MPC-based path trackingCombined value and policy learningModerate (OpenAI Gym)NoNot validated outside simulationReal-World Validated MethodsDDPG with Vision + EKF22Vision-assisted obstacle detectionISO-aligned performanceContinuous control (DDPG)High (real-vehicle test bed)NoReal-world test bed Comparative Analysis of the Proposed and Recent RL-Based Approaches, Grouped by Simulation-Based and Real-World Applicability. Investigation of the Improved Q-Learning Model’s Sensitivity. Quantitatively, these results affirm that each module teaches different tasks independently and in concert to boost performance. The considerable decrease in success rate when removing the look-ahead module is especially noteworthy, as it demonstrates its crucial importance to handling dynamic uncertainty. The robustness of the framework across different hyperparameter settings is also confirmed in the sensitivity analysis. These results show that the proposed enhancements are effective both individually and collectively, resulting in robust and adaptive behaviour when faced with dynamic circumstances. Moreover, results from substitution-style ablations (replacing individual components) further validate that each module adds unique value to the model’s performance gains relative to simplified baseline variants. Computational efficiency Dynamic grid Q-learning demonstrates competitive performance compared with the PPO and DQN baselines across the evaluated metrics. It is faster, more memory efficient, and more computationally efficient, making it a highly scalable and resource-friendly RL approach. Table 6 shows the computational efficiency tested on NVIDIA A100 GPUs. The results in Table 6 for the PPO and DQN baselines are our own reimplemented baseline models, evaluated in the same simulation environments, reward settings, and hardware. Where literature-derived values are used, they are explicitly stated and used solely for the purposes of qualitative comparison. Where exact reproduction was not possible, values are from literature for similar conditions and normalized where appropriate for comparison. The results demonstrate that the proposed framework successfully operates at a competitive computation efficiency as compared to the evaluated deep RL baselines within the simulation environment studied here. This makes it ideal for studies of real-time, simulation-based navigation in resource-limited environments. These improvements across various metrics substantiate the reliability of the proposed method. This highlights the suitability of tabular RL for resource-constrained and simulated real-time applications. Comparative algorithm evaluation The proposed Q-learning-based methods demonstrate a comparable performance with respect to DDPG, PPO, DQN, and A3C-based methods on all the metrics reported in the literature. Direct superiority assertions are only applicable to baselines evaluated under matching simulation settings whilst results reported scarce literature are used for qualitative comparison only. Chen et. al.43 in terms of accuracy (alignment error) and efficiency (average steps), achieving the highest success rate (92.4%) and lowest collision rate (1.8%). This demonstrates the Q-learning method’s exceptional control accuracy and dependability in challenging driving situations. Table 7 highlights the capabilities of different approaches with respect to key objectives, including dynamic obstacle handling, self-parking accuracy, adaptive learning, simulation realism, multi-agent support, and real-world applicability. It should be noted that results for PPO and DQN are for matched in-house baseline implementations, while some DDPG and A3C values are from similar but not identical published studies. Figure 14 provides the performance of the suggested Q-learning SmartGridDrive model and its comparison according to the findings shown in Tables 4, 5, 6 and 7. Subplot (a) emphasizes the test performance indicating that the success rates are high with the constant steps and rewards in the evaluation runs. Subplot (b) shows the ablation study which illustrates the contribution of key components through changes in success, alignment and collision measures. Subplot (c) compares benchmark performance based in NVIDIA A100 GPUs, showing a short training time, consumption of memory and better computational throughput. Subplot (d) opposed the proposed approach and existing methods and showed a higher success rate and fewer errors which confirms the efficiency of the model. Comprehensive Performance Evaluation and Ablation Analysis of the Proposed Q-Learning SmartGridDrive Framework. Comparative analysis The proposed method integrates Q-learning, RRT, and MPC for fine-grained, adaptive navigation in grid environment. Compared to previous model-based or experience-replay strategies, we present a simulated real-time, lightweight enhancement added directly to the tabular Q-learning process. It is designed specifically for self-parking and reversing in dynamic grids, without external memory buffers or learnt dynamics models reported in Table 8. In cases where baseline algorithms were reimplemented, simulation settings were matched for a fairer comparison. Baseline values from the literature were clearly differentiated and incorporated strictly for contextual reference. Statistical analysis and error estimation All uncertainty values reported in the manuscript (e.g., ± values in Tables 5, 6, 7 and 8) indicate mean ± standard deviation of 100 independent runs with different random seeds. These values reflect variability in performance across repeated experiments rather than confidence intervals. A comparison with baseline was done where allowed. Paired two-tailed t-tests were used to assess the statistical significance of the means, with Bonferroni correction for multiple comparisons. The results obtained from 100 independent simulation runs were statistically analyzed, and the error estimates of measurement were obtained to enhance the evaluation of the proposed method. The enhanced Q-learning agent achieved a mean parking orientation error of 2.1° ± 1.4°, mean episode reward of 85.3 (± 5.7) and a success rate of 92.4% ± 1.2%. Positional alignment error (cm) and orientation error (degrees) are reported separately to avoid ambiguity. These measures demonstrate the sure and correct performance in varying circumstances. Also, both efficiency and collision avoidance showed significant improvements (p < 0.01) compared to the baseline Q-learning method using a paired t-test. Statistical significance between the proposed model and ablation variants was assessed using two-tailed paired t-tests. For comparisons involving multiple ablation variants, Bonferroni correction was applied to control family-wise error, with adjusted significance thresholds maintained where applicable. These findings indicate the effectiveness of the advanced RL model in precision and uncertainty tasks, such as self-parking and reversing. This study reinforces the soundness of its results by incorporating measurement-error analysis and statistical validation, in line with the previous literature’s recommendations for best practices in RL assessments. The comparative evaluation further demonstrates that the proposed method consistently outperforms strong baseline approaches. In addition, the sensitivity analysis confirms the robustness and scalability of the framework across varying conditions. The results suggest that the proposed approach is valid and consistent within the analyzed dynamic grid-based simulation setup. To the best of our simulation system, there are also random motions of obstacles, and sensor-like input noise as well as kinematic vehicle constraints. These features were introduced to replicate the uncertainty and movement behaviour of the actual car-parking scenarios45. In order to deal with extreme-case performance, we used test conditions where obstacle density was high, where there was sudden change of direction and where there was very little clearance to park. When this occurred, the agent had reduced success rates, increased alignment error, and increased collisions. These failure instances were mostly associated with those cases when the dynamic obstacles prevented the passage of several critical points or the situation when the agent was affected by the unexpected changes in the target location. Discussion of key findings The results must be taken to be POC evidence in a highly simplified simulated environment. Thus, conclusions around the deployment of full-scale autonomous driving should be seen as provisional. It is still conceptual rather than full-scale real-world feasibility with autonomous vehicles, promising, nonetheless. Thus, the framework proposed here should be understood in that the ideas should be validated in a simulation but not yet used in the deployment of an autonomous driving system. Tabular RL is preferable when: (i) state and action spaces are small enough for a complete table (e.g., 800 states); (ii) interpretability and deterministic convergence are required; (iii) computational resources are limited (e.g., embedded systems). Deep RL becomes necessary for high-dimensional or continuous spaces. Our grid environment stays within tabular feasibility to serve as a reproducible baseline. Taken together, these results narrate a coherent story: Ablation study confirms that every module in the framework is essential. Ablation study confirms that every module in the framework is essential. Comparative evaluation demonstrates clear advantages over strong baselines. Comparative evaluation demonstrates clear advantages over strong baselines. Sensitivity analysis validates robustness and scalability. Sensitivity analysis validates robustness and scalability. Thus, the proposed approach is both effective and reliable for dynamic wireless sensor networks. While simplified, this abstraction is intended to evaluate algorithmic behaviour before transfer to higher-fidelity simulators in future work. Conclusion and future work A modified version of Q-learning framework to suit the dynamic grid-based setting where precision tasks, self-parking and reversing, are part of autonomous driving is presented in the work. Our approach combines the advantages of previous approaches such as obstacle prediction, continuous control, or collaboration into a lightweight, scalable framework that functions well in dynamic real-time simulated situations without the need for intricate multi-agent systems or deep architectures. The look-ahead functions to predict obstacles, adaptive exploration strategies and prioritized updates of highly variable states are included in the proposed system that can deal well with the uncertainties and challenges inherent in dynamic environments. Through RL along with RRT and MPC is a strong base of real-time vehicle path planning and control. The technique uses simulation settings and mathematical models to enhance vehicle performance in a variety of circumstances. When compared to earlier systems, the ARP system was found to be much more accurate and efficient. In simulations of six different situations, Under the matched simulation setup used in this study, the suggested technique achieved improved performance compared with the MPC-only baseline, with an average accuracy of 94.4% and an error rate of 5.5%. The use of RRT, MPC, and RL in this study provides a good basis for ARP. The technique not only ensures accuracy in parking maneuvers but also identifies potential uses in larger navigation and control systems for autonomous cars. These results demonstrate the capability of the model to adjust to the changing environments and prevent collision as well as do the correct parking and reversing procedures. The outcome of these results makes RL an interesting aspect of for autonomous navigation that can be pursued in simulations of dynamic environments. Instead of much of the state of the art being an entirely new RL algorithm, we show that integrating and characterizing complementary existing methods can yield a successful and robust adaptive autonomous navigation framework. Future work will include implementing all baselines under a unified simulation environment to enable fully matched experimental comparisons. This study emphasizes integration and performance characterization of established methods rather than algorithmic novelty. The results demonstrate how such hybrid combinations behave under varying dynamic conditions. However, since the simple grid abstraction does not fully account for vehicle dynamics, perception uncertainty, or interactions with other traffic, we consider our conclusions regarding deployment on-road to be preliminary. The findings in this study are to consider the potential of a dynamic system based on grids and to offer a POC assessment of how such a system could operate in the near future within an existing grid. Although the results are promising, the framework is yet to be tested in a simplified simulation environment (10 × 10 grid, single-agent), which is not realistic to the complexities of the real world such as sensor noise, perception uncertainty and continuous vehicles dynamics. The proposed framework shows better ability for self-parking and reverse driving but has not been evaluated on real autonomous vehicle platforms and a large-scale real-world driving database. Moreover, the table-based formulation of Q-Learning might not scale well to high-dimensional continuous spaces. Next, work is planned on hardware-in-the-loop testing, extensions of the continuous-control RL, and finally, selected real urban traffic scenarios. This results in a sim-to-real gap, which may inhibit easy application to real-world autonomous driving systems. The future work will enhance robustness and generalization via domain randomization, hardware-in-the-loop validation with ROS and benchmarks on real-world data (such as nuScenes Zhang et al.,40, Fu et al.46 and Xu et al.47. Additionally, the method used currently has issues with scaling to larger and multi-agent environments, where state complexity may affect performance. The further development should be aimed at the increased scalability, more realistic simulation, and new features, namely vision-based sensors to apply it to the real world. This POC simulation framework can potentially act as a basis for future research initiatives towards realistic, closed-loop autonomous driving and offers a substantial piece of information regarding designing adaptive and intelligent navigation models to make transportation less risky and efficient. The grid based dynamic environment can be supplied with sensor information including LiDAR and another sensor information. This leads to the ability of the agent to act in a manner of a real vehicle and be tested on noisy and uncertain conditions. Expanding the system to multi-agent cases will make it possible to have cooperative and competitive interactions in the environment like shared parking lots and congested roads. Acknowledgements Open access funding provided by Symbiosis International (Deemed University). Author contributions Conceptualization, Formal Analysis, RRD, DT; Investigation, VP; Methodology, MV; Software, RRD; Writing – Original Draft Preparation, RRD; Writing – Review & Editing, AP; Validation, RRD; Visualization, AP; Supervision, BKD, NS, AP; Project Administration, BKD, NS. All authors have read and approved the final manuscript. Funding Open access funding provided by Symbiosis International (Deemed University). Data availability The repository includes full implementation details, simulation scripts, and parameter configurations to facilitate reproducibility. The code of simulation and the implementation of our new Q-learning, path planning, and obstacle prediction modules is free to study and validate at the following site https://github.com/rev2004/autonomous-parking-and-reverse. Declarations Competing interests The authors declare no competing interests. Ethical approval and consent to participate Not applicable. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The repository includes full implementation details, simulation scripts, and parameter configurations to facilitate reproducibility. The code of simulation and the implementation of our new Q-learning, path planning, and obstacle prediction modules is free to study and validate at the following site https://github.com/rev2004/autonomous-parking-and-reverse.