The Cosmological Simulation Code OpenGadget3 – Implementation of Self-Interacting Dark Matter Moritz S. Fischer 1,2,∗, Marc Wiertel 3, Cenanda Arido 4,5, Yashraj Patil 6, Antonio Ragagnin 7,8,9, Klaus Dolag 2,10, Marcus Brüggen 11, Mathias Garny 5, Andrew Robertson 12, Kai Schmidt-Hoberg 13,14 Abstract Dark matter (DM) could be subject to non-gravitational self-interactions which is relevant to resolve potential problems of cold DM on small scales. Their impact on astrophysical objects such as galaxies and galaxy clusters allows for constraining the strength of this scattering and eventually further properties of the cross-section. To model self-interacting dark matter (SIDM), N N -body simulations are a crucial tool widely employed by the SIDM community. In this paper, we describe the SIDM implementation in the cosmological hydrodynamical N N -body code OpenGadget3 and release it to the public. It is capable of simulating elastic scattering for various differential cross-sections, including strongly anisotropic cross-sections. Beyond single-species models, the code also allows simulating a two-species model with cross-species interactions. In addition to describing the numerical schemes for modelling various flavours of SIDM, we discuss the technical challenges of implementing them. Moreover, we demonstrate through several test problems that OpenGadget3 can accurately simulate DM self-interactions. Furthermore, we assess the performance of the code and provide scaling tests. Lastly, we highlight remaining challenges in the context of SIDM and describe directions for improving the current state of the art. OpenGadget3 – SIDM Implementation keywords methods: numerical –- dark matter 1 Introduction Self-interacting dark matter (SIDM) is a class of particle physics models that provides candidates to explain the nature of dark matter (DM). In contrast to the collisionless cold dark matter (CDM) of the cosmological standard model, \Lambda CDM, SIDM involves interactions beyond gravity. DM particles can scatter off each other, which potentially alters the DM distribution on small scales, i.e. galactic scales. At the same time, the large-scale structure remains unchanged compared to that of CDM. Historically, SIDM had been introduced by 116 with the motivation of providing a mechanism to reduce the abundance of satellite galaxies and reduce the inner density of DM halos. 1 1 1 See also 15 for even earlier work in a more cosmological context. But SIDM has been studied in various contexts, and a variety of particle physics models have been explored. For reviews on SIDM, we refer the reader to 124 and 3 . Various observational probes have been employed to constrain the strength of DM self-interactions. This includes the size of DM density cores across a wide range of halo masses ( 110 ; 142 ; 25 , e.g.) , the shapes of DM halos ( 98 , e.g.) , and an offset between galaxies and DM in merging galaxy clusters ( 134 ; 58 , e.g.) . Deriving all these constraints made it necessary to model the effect of DM scattering. In order to derive constraints, often semi-analytical models are employed, as they are computationally cheap. At the same time, N -body simulations have been proven to be crucial for understanding the astrophysical phenomenology of SIDM from first principles and for validating and calibrating semi-analytic models. Collisionless N -body simulations are a widespread tool in astrophysics to study, for example, the formation of the large-scale structure or the evolution of galaxies and galaxy clusters. The matter distribution is represented by means of N particles, where each represents a patch of phase-space. Hence, it is typically assumed that a numerical particle represents numerous physical particles that are close in phase-space. The numerical particles are characterised by their mass, position, and velocity. Those quantities are evolved over time by discretising and solving the differential equations governing the physical system. In practice, this includes gravity, which allows one to compute the acceleration that each particle experiences. Numerical schemes for N -body simulations of self-gravitating systems have been developed for several decades ( 22 , for a review see, e.g.) . In contrast, SIDM N -body simulations are younger. The first one was run by 14 , where the modelling of the DM scattering is based on a stochastic process. A probability for a particle to interact with its nearest neighbour is computed. In detail, it depends on the local density and the particle’s velocity in the rest frame of the halo. The Monte-Carlo approach by 14 draws a random number and compares it to the probability to decide whether the particle interacts with the neighbouring particle. An important improvement at that time was made by 66 ; they used the actual phase-space distribution for the interactions, i.e., computed the relative velocity of the numerical particles and used it in computing the interaction probability. More similar schemes were quickly implemented ( 144 ; 21 ) and some already included a velocity-dependent cross-section ( 19 ; 28 ) . In addition to employing a Monte-Carlo scheme, very strong cross-sections have also been modelled using smoothed particle hydrodynamics (SPH) ( 86 ; 143 ) . Further SIDM implementations followed in various astrophysical N -body codes ( 67 ; 129 ; 107 ; 45 ; 104 ; 106 ; 130 ; 7 ; 139 ; 20 ; 35 ; 125 ; 126 , e.g.) . Although almost all SIDM N -body codes are based on a Monte-Carlo scheme, there is one notable exception by 60 . Many particle physics models are described by a differential cross-section that is velocity and angular dependent, and the two dependences are often not separable. The first implementation that allows one to model such a differential cross-section was written by 103 and applied to simulating merging galaxy clusters. Later, other implementations of full differential cross-sections followed, e.g. by 7 ; 20 ; 141 , or the one in OpenGadget3 that we describe here. A common problem in SIDM N -body simulations occurs when a particle undergoes multiple scatterings per time step without updating the particle’s velocity between the interactions. The exchange of kinetic energy between the numerical particles cannot be treated in a simple cumulative manner, as doing so leads to an error in energy conservation, introducing an artificial heating term ( 104 ) . Fully resolving this issue is difficult, as one needs to ensure that the velocity of a particle is updated after an interaction before undergoing another scattering event within the same time step. In particular, for parallel computations, this can require a fairly sophisticated parallelisation scheme. 104 have largely improved on this issue ( 105 , see also) . However, it has now been fully resolved by 35 , with 125 presenting an alternative approach that fulfils the same purpose. Another challenge that had persisted for a while was the simulation of strongly forward-dominated cross-sections. 2 2 2 Small-angle scattering needs to be much more frequent than large-angle scattering to have a comparable effect on the DM distribution. As a consequence, strongly forward-dominated cross-sections are also referred to as frequent SIDM (fSIDM) in contrast to rare SIDM (rSIDM). Within the usual SIDM schemes, they require extremely small time steps in order to keep the interaction probability well below unity, effectively leading to prohibitively high computational costs. An approach to avoid this problem is to simply exclude those small-angle scattering events, as, for example, done by 103 . However, those events could be so frequent that they contribute significantly to the momentum or energy transfer between the particles. An attempt to study such a cross-section in the context of a merging galaxy cluster was made by 61 . They had built a semi-analytic approach by combining an N -body code with an effective drag-like description for small-angle scattering. Another way of approaching the problem was undertaken by 71 . They aimed to model the evolution of an isolated halo by effectively describing small-angle scattering as heat conduction within an N -body simulation. However, the problem was only fully resolved by 35 , who presented the first scheme that allowed one to model such a cross-section from first principles within N -body simulations. It applies an effective drag force and perpendicular momentum diffusion to pairs of neighbouring N -body particles, building on analogous methods known from seminal works by 30 ; 131 related to Brownian motion. The drag force acting along the axis defined by the relative velocity leads to a deceleration that is applied deterministically in each time step, representing the cumulative effect of a large number of small-angle scatterings. The momentum diffusion within the plane perpendicular to this axis is described as a stochastic process, specifically regarding the direction along which a perpendicular momentum kick is applied, while its magnitude is fixed by energy conservation. Overall, this leads to a deterministic effective scattering angle that depends on the cross-section, relative velocity and density in accordance with the microscopic dynamics. Recently, 6 combined the schemes for large- and small-angle scattering. This allowed – for the first time – the efficient simulation of differential cross-sections that fall in both regimes. The underlying idea is to introduce a critical angle; the parts of the differential cross-section with scattering below this angle are treated with the small-angle scattering scheme, and the other ones with the larger-angle scattering scheme. This extends the range of models that can be practically simulated. Various SIDM models have been studied so far with N -body simulations. This does not only concern the exploration of various angular and velocity dependencies, such as for models of resonant scattering ( 121 ) . But it goes beyond the assumption of elastic scattering or having only a single DM particle species. A few simulations exist with models where the scattering is no longer elastic, but a fraction of the kinetic energy is lost per interaction. Such models have been studied by 60 ; 112 ; 113 ; 136 . Closely related are models with multiple states, where energy can be stored in excited states. While particles interact, they may change between the ground and excited state. 130 ; 18 ; 93 ; 72 ; 76 have investigated such models. Recently, there has also been interest in models with two species with interactions between the species ( 96 ; 138 ; 137 ) . These variations on the classic elastic SIDM model give rise to diverse phenomenology, with many interesting effects that can potentially be probed observationally. A fascinating consequence that many SIDM models have in common is the gravothermal collapse of DM halos. The self-interactions may cause the halo to lose energy, by transporting it to larger radii. As a consequence, the central DM density grows, and – given the negative heat capacity of gravitationally self-bound objects – the centre also heats up, accelerating the gravothermal evolution. Motivated by observations that may provide hints for surprisingly dense objects ( 127 ; 80 ; 83 ; 54 ; 26 ; 32 ; 74 ; 82 ; 120 ; 68 ; 128 , e.g.) , these later stages of gravothermal evolution have received a lot of attention recently ( 140 ; 89 ; 46 ; 63 ; 70 ; 77 ; 145 ; 149 ; 31 ; 44 ; 55 ; 73 , e.g.) . However, it has transpired that these stages are difficult to model with N -body codes ( 141 ; 150 ; 78 ; 94 , e.g.) . Not all of these problems are directly related to the modelling of SIDM, but some are also present in simulations of collisionless DM, but become more severe for the dense halos produced by the gravothermal evolution ( 41 ) . Despite some concerns, no fundamental problems were encountered in modelling SIDM with N -body simulations. Although modelling the late gravothermal evolution accurately is very expensive and requires increasing resolution and smaller time steps, the deeper one simulates into the gravothermal collapse ( 43 ) . The underlying problem of modelling SIDM, i.e. solving the Boltzmann equation, had been addressed well before any SIDM N -body simulation had been run. For problems rather different from SIDM, generally referring to rarefied gases, for example, in the context of the reentry of a spaceship, Direct Simulation Monte-Carlo (DSMC) schemes have been developed ( 10 ; 9 ; 92 ; 95 ; 49 ) . They commonly rely on grid cells instead of performing a neighbour search and employing kernel functions, as is done in state-of-the-art SIDM N -body simulations. As a consequence, adaptive meshing techniques have been developed for DSMC ( 65 ; 135 , e.g.) . However, there have also been studies that developed meshless DSMC methods ( 91 ; 114 , e.g.) , being more closely related to the schemes being usually explored for SIDM. A first study with a grid-based DSMC method in the context of SIDM was undertaken by 56 . They studied the gravothermal evolution of an isolated spherically symmetric SIDM halo and took advantage of the symmetries of the set-up by reducing the dimension of the DSMC grid from three to one. A closely related work with a similar scheme followed by 63 . The aim of this paper is to provide an overview of the SIDM implementation in the cosmological hydrodynamical N -body code OpenGadget3 (Dolag et al. in prep.). This was introduced by 35 and refined in several later studies. Initially, it only enabled simulating elastic velocity-independent scattering for isotropic or extremely forward-dominated cross-sections. With 36 , comoving integration was added, to enable cosmological simulations. Moreover, the implementation was extended to simulate velocity-dependent interactions and got a more efficient message passing interface (MPI) parallelisation ( 39 ) . 6 enhanced the capabilities of simulating anisotropic cross-sections. In particular, it became possible to simulate a differential cross-section that is forward-dominated but also has non-negligible larger-angle, a situation that is typically encountered for scatterings described by Coulomb or Yukawa potentials. Another recent extension is the introduction of a two-species model with interactions between the two species ( 96 ) . In addition to the publications that have improved or extended the SIDM implementation in OpenGadget3 , the module has also been used in several studies ( 37 ; 38 ; 42 ; 100 ; 108 ; 109 ) . In particular, ( 41 ) and ( 43 ) have investigated the numerical properties of OpenGadget3 in the context of the gravothermal collapse of SIDM halos and demonstrated its accuracy. Here, we will go beyond what has been presented in the above-mentioned publications and release 3 3 3 Until the full OpenGadget3 code is made public with the main release paper (Dolag et al. in prep.), we are happy to grant access on demand. For which a message can be sent to the corresponding author. a version that is additionally capable of simulating full differential cross-sections. This, in particular, includes models where the velocity and angular dependence are not separable. Moreover, we describe for the first time an Open Multi-Processing (OpenMP) parallelisation for the SIDM computations, which is part of the release. Beyond the code itself, we show several test problems, some of them for the first time, that can be used to test SIDM simulation codes. Importantly, we investigate how the SIDM implementation scales with the number or processors used and provide advice on choosing the simulation parameters. The remainder of the paper is structured as follows. In Sect. 2 we describe the numerical method on which the SIDM implementation of OpenGadget3 is based. Here, we also discuss the implementation and related challenges, in particular when it comes to the parallelisation of the computations. It is followed by a series of test problems in Sect. 3 , where we demonstrate that we can produce accurate results for problems with a known solution. In addition, we also simulate more astrophysically relevant set-ups to test the scaling of the SIDM module as well as comment on the particular challenge of simulating the collapse phase of the gravothermal evolution. In Sect. 4 , we discuss shortcomings of the current implementation and avenues for improving upon the current state of the art. Finally, in Sect. 5 , we summarise and conclude. Additional information is provided in the Appendix. 2 Numerical Method The systems we aim to describe consist of an enormous number of dark matter (DM) particles. In this regime, a statistical description becomes appropriate: instead of tracking individual particles, one may consider the smooth phase-space density f(\mathbf{x},\mathbf{v},t) , which is well defined only in the limit of many particles. In this continuum limit, the evolution of the system is deterministic and governed by the Boltzmann equation with self-gravity. The underlying goal of the numerical method presented in this section is to model the impact of DM self-interactions on astrophysical objects. Since their dynamics are primarily driven by gravity, we seek to solve the Vlasov–Poisson equation supplemented by a collision term, i.e. the Boltzmann equation including self-gravity, \frac{\partial f}{\partial t}+\mathbf{v}\cdot\nabla_{x}f-\nabla_{x}\Phi\cdot\nabla_{v}f=\left(\frac{\partial f}{\partial t}\right)_{\mathrm{coll}}\;. (1) Here, f denotes the phase-space density, \mathbf{v} the velocity, and \Phi the gravitational potential. The self-interactions are encoded in the collision term on the right-hand side, which depends on the differential cross-section specified by the underlying particle physics model. Although microscopic DM scatterings are stochastic processes, the macroscopic systems of interest behave deterministically in the limit of large particle numbers. Here, Eq. ( 1 ) provides a valid statistical description. Complications arise, however, when employing an N -body method to model SIDM. The discretization of the phase-space density into a finite number of simulation particles (see Sect. 2.1 ) reintroduces stochasticity at the numerical level. As a consequence, the collision term can no longer be implemented in a purely deterministic manner. Instead, the numerical scheme relies on a Monte-Carlo procedure whose stochastic realization converges, in the large-number limit, to the deterministic evolution of the phase-space density described by Eq. ( 1 ). 2.1 Numerical representation and kernel overlap The physical entities, in our case, the DM particles, are represented by N numerical particles. They represent all the mass, momentum, and energy of the DM. Each numerical particle is characterised by a mass, m_{\mathrm{n}} , 4 4 4 Here, we use the label “n” to distinguish the numerical particle mass from the mass of the physical particles. a position, \mathbf{x} , and a velocity, \mathbf{v} . One numerical particle can be interpreted as representing a phase-space patch of physical DM particles. In this picture, all the DM particles of the phase-space patch have the same velocity, as each numerical particle has only a single velocity. For SIDM, every numerical particle is additionally assigned a kernel. It is introduced to obtain a local density estimate used for the modelling of the self-interactions. Although the kernel function does not represent a fundamental property of the physical system, it can be useful for the derivation of the SIDM scheme to interpret it as a description of the spatial extent of the represented DM, i.e. the phase-space patch’s distribution in configuration space. The kernel has a positive finite size h , implying that the kernel function W(d,h) becomes zero at distances d\geq h . The density represented by a single numerical particle can be understood as \rho=W(d,h)\,m_{\mathrm{n}} . We determine h for each particle adaptively by searching for the N_{\mathrm{ngb}} nearest neighbours, i.e. h is a radius that includes the nearest N_{\mathrm{ngb}} particles. We illustrate the numerical representation and the interaction of two numerical particles in Fig. 1 . Figure 1: Illustration of the numerical representation and the scattering process for the SIDM scheme. The black circles denote numerical particles, with the black dots inside illustrating that they consist of numerous physical DM particles. The numerical particles evolve from the pre-scattered state (left) to the post-scattered state (right) by changing the direction of their momentum vectors in their centre-of-mass frame. The effect of self-interactions is computed based on pairs of neighbouring particles. Pairs that are close enough can interact; the criterion is |\mathbf{x}_{i}-\mathbf{x}_{j}|\uppi/2 . The momentum transfer cross-section is, for example, relevant in a model with two DM species where it is also the better-suited quantity to measure the strength of the interactions across the species compared to the viscosity cross-section, which is a common measure for interactions within a species ( 96 , e.g.) . We note that one can also express the forward-dominated limit for distinguishable particles as a differential cross-section, for example, \left.\frac{\mathrm{d}\sigma}{\mathrm{d}\cos\theta_{\mathrm{cms}}}\right|_{\mathrm{fwd}}=\lim_{\epsilon\to 0}\frac{\sigma_{\mathrm{T}}}{4\,\ln(\epsilon^{-2})}\frac{1}{\left(\epsilon^{2}+\sin^{2}{(\theta_{\mathrm{cms}}/2)}\right)^{2}}\,. (9) In the following two sections we consider the two algorithms provided by the code, adapted to differential cross-sections dominated by large- and small-angle scatterings, respectively. The former leads to a large momentum transfer per scattering, and is typically characterized by a relatively low total scattering rate, while the latter efficiently describes frequent small-angle scatterings with a large total scattering rate but small momentum transfer per scattering. 2.3 Rare self-interactions In the following, we describe the modelling of rare self-interactions. By this we refer to a regime in which the DM scattering rate is low enough such that one can assume that a physical particle represented by a numerical particle will not scatter more than once per pair-wise interaction of numerical particles. 7 7 7 In this context, we want to highlight the discussion given under point four in sect. 4 by 41 related to this assumption. But it could still scatter multiple times per time step as multiple numerical pairs are considered. We note that most SIDM implementations require a stricter criterion, where a particle is allowed to interact only once per time step. But thanks to its parallelisation, the implementation in OpenGadget3 allows relaxing this condition; it must hold only on the basis of a single pair and not all pairs. In the picture of a numerical particle representing a phase-space patch, physical particles would scatter in all directions when two phase-space patches interact. The physical particles represented by numerical particles are located next to each other in phase-space before the interaction. After the interaction, they could be distributed over a substantial part of the phase-space. Some of them remain unchanged, i.e. the ones that do not scatter. Unfortunately, this post-interaction distribution of physical particles can no longer be represented by a single numerical particle, as it represents only a single velocity. As a consequence, we employ a Monte-Carlo approach by picking only one of the possible post-interaction locations in phase-space. Each of these locations is associated with a probability density that we should satisfy when selecting the post-interaction location. This is done in two steps: Firstly, we decide if the phase-space location changes at all, and if so, we secondly pick the new location following the probability density implied by the differential cross-section. The first step consists of computing an interaction probability and drawing a random number to decide whether the pair of numerical particles under consideration interacts. The second step is to randomly choose the scattering angles and update the velocities of the particles. When the number of numerical particles is sufficiently large, this can provide a good representation of the phase-space and how it evolves under the influence of self-interactions. For the first step, the probability that two particles interact can be expressed as, 8 8 footnotemark: 8 † † footnotetext: We note that Eqs. ( 10 ) and ( 12 ) can be viewed as a special case of the more general description presented by 40 . For a derivation, we refer the reader to appendix A of that paper (the Equations can be obtained by setting f_{\mathrm{bary}}=1 and \mu=1 ). P_{ij}=\frac{\sigma}{m_{i}}\,m_{\mathrm{n},i}\,|\Delta\mathbf{v}_{ij}|\,\Lambda_{ij}\,\Delta t\,. (10) Here, \Delta\mathbf{v}_{ij} is the relative velocity of the particles i and j , and \Delta t denotes the time step. Moreover, we employ the total cross-section \sigma , given by Eq. ( 4 ). The physical DM particle mass of the species represented by particle i is m_{i} . Whereas the numerical particle mass is m_{\mathrm{n},i} . Importantly, the interaction probability is symmetric, P_{ij}=P_{ji} , and the time step should be kept small enough such that P_{ij}\ll 1 ( 6 , but see also appendix G by) . A pseudo-random number x\in[0,1] is drawn, and when x0 ), i.e. the perpendicular velocity dispersion of the phase-space patch increases. Although the scheme described in the previous section can, in principle, simulate any angular dependence, it faces a severe challenge when the scattering rate becomes very large. In order to keep the interaction probability small in that case, it requires tiny time steps. Unfortunately, this does not necessarily imply that the self-interactions are strong enough that one could apply a fluid description. In the case of small-angle scattering, each scattering event has only a small effect, and the overall influence of the self-interactions might be moderate only. Since the scheme of rare scatterings requires tiny time steps, a different formulation is necessary. Strongly forward-dominated cross-sections, which are characterised by small scattering angles, can be simulated by employing an effective description derived from first principles ( 35 ) . This effective description is based on a drag force decelerating the two particles ( 61 , see also) and perpendicular momentum diffusion. We illustrate this in Fig. 2 . The drag force acting on two numerical particles that arise in the limit of Eq. ( 2.2 ) can be expressed as, † † footnotemark: F_{\mathrm{drag},ij}=\frac{1}{1+r}\frac{\sigma_{\mathrm{\tilde{T}}}}{m_{i}}\,m_{\mathrm{n},i}\,m_{\mathrm{n},j}\,|\Delta\mathbf{v}_{ij}|^{2}\,\Lambda_{ij}\,. (12) Here, \sigma_{\mathrm{\tilde{T}}} denotes the modified momentum transfer cross-section as given by Eq. ( 6 ). In the case of distinguishable particles (Eq. ( 9 )), it must be replaced by the transfer cross-section (Eq. ( 8 )). The mass of the physical particle represented by particle i is m_{i} . The numerical particle masses of the interacting particles are m_{\mathrm{n},i} and m_{\mathrm{n},j} . Their relative velocity is \Delta\mathbf{v}_{ij} and the kernel overlap \Lambda_{ij} is given by Eq. ( 2 ). We employ the mass ratio r=m_{j}/m_{i} of the physical particles. In the case of a single species SIDM model, we have m_{i}=m_{j} and m_{\mathrm{n},i}=m_{\mathrm{n},j} . Moreover, the Equation implies a momentum change, |\Delta\mathbf{p}_{\mathrm{drag}}|=F_{\mathrm{drag}}\,\Delta t over the time step \Delta t . The momentum diffusion is modelled by adding the energy lost due to the drag force in a direction perpendicular to the motion of the particles in the centre-of-mass frame. The added momentum is given by |\Delta\mathbf{p}_{\mathrm{rand}}|=\sqrt{|\Delta\mathbf{p}_{\mathrm{drag}}|\left(|\Delta\mathbf{p}_{ij}|-|\Delta\mathbf{p}_{\mathrm{drag}}|\right)}\,. (13) Here we use \Delta\mathbf{p}_{ij}=m_{\mathrm{n},i}\,\mathbf{v}_{i}-m_{\mathrm{n},j}\,\mathbf{v}_{j} . The momentum |\Delta\mathbf{p}_{\mathrm{rand}}| is added to each of the two particles but with opposite directions to fulfil momentum conservation. The exact direction of \mathbf{p}_{\mathrm{rand}} is always perpendicular to the motion in the centre-of-mass frame, but determined by a random angle uniformly distributed in the range [0,2\uppi] . Although we have described here the modelling of frequent self-interactions by a drag force and a perpendicular momentum diffusion, it can also be understood as scattering by an effective angle. The underlying physical particles that are represented by the numerical particle would not all have the same deflection angle. Instead, their distribution of deflection angles follows from Molière’s theory ( 84 ) for the limit that is relevant here. 10 10 10 We note that Molière’s theory was formulated for particles scattering off a fixed target, i.e. the recoil of the target particles is neglected. Given that we are interested in the regime where the particle’s velocities change only slightly, Molière’s theory is applicable when taking care of the reference frame. In principle, we could sample deflection angles from this distribution. However, this is unnecessarily complicated for the case of elastic scattering. Using an effective angle as a representation of the whole distribution is fine, as the process of stochastic interactions between the numerical particles converges against the correct distribution of deflection angles ( 35 ; 6 ) . We want to emphasise that every particle interacts with many particles per time step, namely with all particles for which \Lambda_{ij}>0 . Lastly, we want to emphasise that the description above is consistent with the physical model as expressed by Eq. ( 2.2 ). This is true independent of the mean-free path. The drag force derived by 102 is incompatible with the numerical representation that we assume (see Sect. 2.1 ). This applies to DSMC codes as well and in particular concerns the second assumption of the derivation by 102 . However, the collective behaviour of our numerical particles would satisfy their drag force. This is also expected to be the case for the more general drag force by 27 ; 88 derived in the context of DM-baryon scattering when applied to the SIDM case. 2.5 Unequal-mass scattering For the explanations given so far, we already took the general case of particle scattering with unequal masses into account. Here, we elaborate in greater detail on aspects of unequal-mass scattering. In Sects. 2.3 and 2.4 , we have assumed that the mass ratio of the numerical particle masses is the same as that of the physical particle masses. The formulation of the scattering does not directly imply a limit on the physical mass ratio that can be studied. However, for large numerical mass ratios, it can be computationally costly to achieve a sufficiently high resolution, avoiding artificial relaxation from modelling the gravitational interactions. This implicitly limits the parameter space that can be studied. An important characteristic of models with scatterings between particles of unequal masses is that they can give rise to mass segregation, where heavier particles tend to sink towards the centre of DM halos. This can lead to a distinct phenomenology compared to models with equal particle masses ( 96 ; 138 , e.g.) . For very large physical mass ratio, it may be advantageous to choose the ratio of masses of the numerical particles different from the ratio of physical masses for the two species. However, this is non-trivial. In particular, 67 pointed out that there is no reason to symmetrise the scattering probability for a pair-wise interaction with numerically unequal particle masses to simulate equal-mass scattering. Such a symmetrisation would artificially enhance the scattering rate for lighter particles and reduce it for heavier ones. 11 11 11 This, for example, implies that the symmetrisation given by eq. 6 in 107 holds only in the case where the mass ratio of the numerical particles is the same as that of the physical ones. However, in this case, their two probabilities used in the symmetrisation are equal by construction as a consequence of their use of a symmetric geometric factor, as we do too (Eq. ( 2 )). Moreover, obtaining the correct scattering kinematics is not straightforward. When the numerical particles have the same mass ratio as the physical ones, their centre-of-mass velocity is the same. But if the mass ratios do not match, a pair of numerical particles has a different centre of mass velocity than a pair of the physical particles they represent. As a consequence, altering the direction of the momentum vectors of the numerical particles in their centre-of-mass system does not reflect the scattering kinematics of the physical particles they represent. This has a severe consequence: In general, scattering leads to energy equipartition; in the equilibrium state, the temperatures of particles with different masses depend on the mass ratio. If the interactions between the numerical particles do not reflect the scattering kinematics of the physical particles, the simulated system may approach the wrong equilibrium state. Nevertheless, one may modify the equations that we described above to obtain the correct scattering kinematics in the case of non-matching mass ratios, but this may only be possible when compromising on momentum and/or energy conservation. As a consequence, previous studies have only considered the case where the numerical mass ratios matched the physical ones. There are only a few studies with N -body simulations on unequal-mass scattering. These are 138 ; 137 and 96 , where the latter is based on OpenGadget3 . They all have in common that they assume a model with two species that differ in the DM particle mass. Extending the implementation in OpenGadget3 beyond the two-species case is left for future work. Currently, we set the mass ratio of the physical DM particles implicitly with the mass ratio of the numerical particles in the initial conditions (ICs). An additional complication arises for specifying the strength of the interactions. In the two-species case, we assume that the cross-section per mass refers to the lighter species. 2.6 Velocity dependence Many particle physics models that fall into the class of SIDM have a velocity-dependent cross-section ( 33 ; 13 ; 34 ; 75 ; 12 ; 17 ; 122 , e.g.) . In OpenGadget3 we have implemented four different velocity-dependencies that can be used either with isotropic scattering or the forward-dominated limit (Eqs. ( 2.2 ) and ( 9 )). Figure 3: Viscosity cross-section as a function of velocity for different models. We illustrate the velocity dependence of different models implemented in OpenGadget3 . These are a velocity-independent cross-section (black), the velocity-dependent model according to Eq. ( 14 ) (light green), a model with a simple parametrisation for a resonant feature as given by Eq. ( 15 ) (blue), Møller scattering as expressed by Eq. ( 21 ) (orange), and Rutherford scattering following Eq. ( 22 ) (dark green). The cross-section normalisation parameter is set to \sigma_{0,\mathrm{V}}=10\,\mathrm{cm}^{2}\,\mathrm{g}^{-1} and for the velocity-dependent cross-section, w=10\,\mathrm{km}\,\mathrm{s}^{-1} . The first model parametrises the velocity dependence of the angle-averaged cross-section, X , as \sigma_{X}=\sigma_{0,X}\,\left(1+\left(\frac{v}{w}\right)^{\beta}\right)^{\alpha/\beta}\,. (14) Here, the cross-section is constant for v\ll w and decreases as \sigma_{X}\propto v^{\alpha} for v\gg w . The transition between these two regimes is governed by the parameter \beta . The parameters should be chosen such that they fulfil w>0 , \alpha<-1 , and \beta>0 to fall in the range currently supported by the SIDM time step criterion (see Sect. 2.9 ). In addition, we include a model with a resonance, i.e. the cross-section is enhanced for velocities close to w . This is a feature expected from various particle physics models ( 123 ; 17 ; 18 ) . In detail, we use the same formulation as 111 , namely \sigma_{X}=\sigma_{0,X}\,\frac{1+w^{2}/u^{2}}{1+[v-w]^{2}/u^{2}}\,, (15) where u is an additional parameter that determines the width and height of the resonant peak. We leave it for future work to implement more complicated models as done by 121 . Moreover, we have included the velocity dependence of Møller scattering. In detail, the one of the viscosity cross-section (Eq. ( 5 )). \sigma_{\mathrm{V}}=\frac{6\,\sigma_{0,\mathrm{V}}\,w^{8}}{v^{8}+2\,v^{6}\,w^{2}}\left[2\,\left(5+\frac{5\,v^{2}}{w^{2}}+\frac{v^{4}}{w^{4}}\right)\,\ln\left(1+\frac{v^{2}}{w^{2}}\right)\right.\\ \left.-5\,\left(\frac{2\,v^{2}}{w^{2}}+\frac{v^{4}}{w^{4}}\right)\right]\,. (16) It is normalised such that \sigma_{\mathrm{V}}(v=0)=\sigma_{0,\mathrm{V}} . Furthermore, the velocity dependence of Rutherford scattering is available in OpenGadget3 . \sigma_{\mathrm{V}}=\frac{6\,\sigma_{0,\mathrm{V}}\,w^{6}}{v^{6}}\left[\left(2+\frac{v^{2}}{w^{2}}\right)\,\ln\left(1+\frac{v^{2}}{w^{2}}\right)-\frac{2\,v^{2}}{w^{2}}\right] (17) As for Møller scattering, we obtained it by computing the viscosity cross-section and normalised it such that \sigma_{\mathrm{V}}(v=0)=\sigma_{0,\mathrm{V}} . While these velocity dependencies do not capture the angular dependence, we note that we have also implemented the full differential cross-sections for Møller and Rutherford scattering as explained in Sect. 2.7 . The velocity dependencies from above can be used together with isotropic scattering or the forward-dominated limit. The underlying idea is to assume that the angular and velocity dependences are separable. Although this typically does not apply to realistic models, it can be helpful for understanding the role of angle and velocity dependence to define the cross-section as \frac{\mathrm{d}\sigma}{\mathrm{d}\cos\theta_{\mathrm{cms}}}=f_{\mathrm{angle}}(\theta)\,f_{\mathrm{vel}}(v)\,. (18) In the isotropic case, f_{\mathrm{angle}} would be a constant, \sigma_{0} , and in the forward-dominated case it would be given by Eq. ( 2.2 ) or Eq. ( 9 ). Similarly, in the case of a velocity-independent cross-section, f_{\mathrm{vel}}(v) would be unity. Alternatively, it could be given by \sigma_{X}/\sigma_{0,X} using one of the Eqs. ( 14 )–( 17 ). 2.7 Full differential cross-sections Many particle physics models feature interactions for which the velocity and angular dependences cannot be viewed separately, i.e. the differential cross-section cannot be expressed in a form like Eq. ( 18 ). Simulating such SIDM cross-sections was first explored by 103 and later implemented in several codes. OpenGadget3 allows the simulation of full differential cross-sections too. It was initially implemented by 133 , and we will describe it in the following. For generality, we assume to model the differential cross-section in the range \theta_{\mathrm{cms}}\in[\theta_{\mathrm{min}},\theta_{\mathrm{max}}] . This allows us to exclude problematic angular ranges, e.g. for divergent models, as also done by 103 . The full angular range would be \theta_{\mathrm{min}}=0 and \theta_{\mathrm{max}}=\uppi . To compute the interaction probability (Eq. ( 10 )), we need to compute the total cross-section for the corresponding angular range, \sigma_{[\theta_{\mathrm{min}},\,\theta_{\mathrm{max}}]}(v)=\int_{\theta_{\mathrm{min}}}^{\theta_{\mathrm{max}}}\frac{\mathrm{d\sigma}(\theta^{\prime},v)}{\mathrm{d}\cos\theta_{\mathrm{cms}}}\,\sin(\theta^{\prime})\,\mathrm{d}\theta^{\prime}\,. (19) At the beginning of the simulation, we compute \sigma_{[\theta_{\mathrm{min}},\,\theta_{\mathrm{max}}]} for various values of v by numerically integrating the differential cross-section. These values are stored in a lookup table, which is used to interpolate the total cross-section at the requested velocity. To compute the scattering angle, we compute the cumulative distribution function for the scattering angles, \displaystyle\mathrm{CDF}(\theta_{\mathrm{cms}},v)= \displaystyle\frac{1}{\sigma_{[\theta_{\mathrm{min}},\,\theta_{\mathrm{max}}]}(v)} \displaystyle\int_{\theta_{\mathrm{min}}}^{\theta_{\mathrm{cms}}}\frac{\mathrm{d\sigma}(\theta^{\prime},v)}{\mathrm{d}\cos\theta_{\mathrm{cms}}}\,\sin(\theta^{\prime})\,\mathrm{d}\theta^{\prime}\,. (20) The CDF increases monotonically from 0 to 1 and gives the probability that if the particles interact, they scatter with an angle smaller than \theta_{\mathrm{cms}} . To determine the scattering angle \theta_{\mathrm{cms}} , we draw a uniformly distributed random number x\in[0,1] and obtain \theta_{\mathrm{cms}} by inverting x=\mathrm{CDF}(\theta_{\mathrm{cms}},v) . The scattering angle is determined from a lookup table by interpolation. In OpenGadget3 , two different methods are implemented to do so. Here, we briefly describe them; a more detailed description can be found in Appendix A . The first one is the Grid method, which uses a grid in the x – v plane and stores the values for the scattering angle. The grid is linearly spaced in x and allows us to easily infer the scattering angles by interpolating between the grid points. However, computing the lookup table is a little more complicated, as the corresponding values of \theta_{\mathrm{cms}} must be found for the given values of x . The second method implemented in OpenGadget3 is the ZSlice method. Here, a grid in the \theta_{\mathrm{cms}} – v plane is used to store the values of x . The table can be directly computed using Eq. ( 2.7 ). However, inferring the scattering angle from it is computationally more expensive than for the Grid method. But compared to the Grid method, the ZSlice method is able to significantly reduce the numerical error in the most error-prone regions and thus yields overall a higher accuracy when using the same number of grid points. At the same time, no substantial slowdown of the simulation was found due to the increased computational costs in evaluating the scattering angle ( 133 ) . Therefore, we take the ZSlice method as the default method. The tables to store the cumulative distribution function and the angle-averaged cross-sections are logarithmically spaced in velocity, but additionally contain an entry for a velocity of zero. If interacting particles encounter a relative velocity larger than the maximum value stored in the tables, we automatically extend the tables. We have implemented Møller scattering in the non-relativistic limit using the Born approximation. \left.\frac{\mathrm{d}\sigma}{\mathrm{d}\cos\theta_{\mathrm{cms}}}\right|_{\textnormal{M{\o}ller}}=\sigma_{0}\frac{\left(3\cos^{2}\theta_{\mathrm{cms}}+1\right)\frac{v^{4}}{w^{4}}+4\frac{v^{2}}{w^{2}}+4}{\left(\sin^{2}\theta_{\mathrm{cms}}\,\frac{v^{4}}{w^{4}}+4\frac{v^{2}}{w^{2}}+4\right)^{2}}\,. (21) Rutherford scattering is also available in OpenGadget3 . \left.\frac{\mathrm{d}\sigma}{\mathrm{d}\cos\theta_{\mathrm{cms}}}\right|_{\textnormal{Rutherford}}=\sigma_{0}\left[2+\frac{v^{2}}{w^{2}}(1-\cos\theta_{\mathrm{cms}})\right]^{-2}\,. (22) Further differential cross-sections can be easily added, as all derived quantities, i.e. the angle-averaged cross-sections as well as the cumulative distribution function to sample the scattering angles, are computed numerically by the code. The description above has so far assumed that the scattering is only modelled with the scheme for rare interactions (Sect. 2.3 ). However, the differential cross-section could be strongly anisotropic, such that the rare scattering scheme is hardly applicable because of high computational costs. Instead, one may model the interactions with the scheme for frequent scattering, and approximate the differential cross-section with Eqs. ( 2.2 ) or ( 9 ) for the angular dependence and use a separable velocity dependence (Sect. 2.6 ). However, this is only applicable if the differential cross-section does not give rise to significant large-angle scattering. But if this is the case, one can combine the two schemes into a hybrid SIDM scheme as explored by 6 . This hybrid scheme, as implemented in OpenGadget3 , splits the differential cross-section according to a critical angle, \theta_{\mathrm{c}} , into two parts. Scatterings by an angle smaller than \theta_{\mathrm{c}} are described by the frequent scattering scheme, and scatterings above \theta_{\mathrm{c}} are modelled with the help of the rare scattering scheme. Accordingly, the momentum transfer or modified momentum transfer cross-sections relevant for the frequent scheme are integrated over the interval \theta_{\mathrm{cms}}\in[0,\theta_{\mathrm{c}}] . Analogous to the total cross-section, we use a lookup table from which we interpolate the value at the requested velocity. Those are then used to compute the drag force according to Eq. ( 12 ). The integration range for the total cross-section is set to \theta_{\mathrm{cms}}\in(\theta_{\mathrm{min}}=\theta_{c},\theta_{\mathrm{max}}=\uppi] . In the case of models with indistinguishable particles, where the differential cross-section is symmetric (e.g. Møller scattering), it is also favourable to describe scatterings close to \theta_{\mathrm{cms}}=\uppi with the frequent scheme too. Hence, we integrate the modified momentum transfer cross-section over the joint interval \theta_{\mathrm{cms}}\in[0,\theta_{\mathrm{c}}]\cup[\uppi-\theta_{\mathrm{c}},\uppi] and the total cross-section over the interval \theta_{\mathrm{cms}}\in(\theta_{\mathrm{min}}=\theta_{\mathrm{c}},\theta_{\mathrm{max}}=\uppi-\theta_{\mathrm{c}}) . Furthermore, the angular range of the cumulative distribution function to sample the scattering angle, \theta_{\mathrm{cms}} , must be adjusted accordingly. However, in practice, we have two independent critical angels, one for forward scattering and one for backward scattering. The choice of the critical angle, \theta_{\mathrm{c}} , is a trade-off between accuracy and a reduction in computational costs. The larger \theta_{\mathrm{c}} , the less accurate the deflection of particles due to small scattering angles is modelled. But at the same time, the simulation can become much faster. How large the speed-up is depends in detail on the anisotropy of the cross-section. In the case of typical astrophysical set-ups such as merging galaxy clusters, satellite galaxies orbiting their host, or full cosmological simulations, our default value of \theta_{\mathrm{c}}=0.1 may yield sufficiently accurate results. For more details on this hybrid scheme, we refer the reader to 6 . 2.8 Time integration In OpenGadget3 , the time integration is performed using a leapfrog scheme in the kick-drift-kick (KDK) formulation. This integration scheme is commonly used in cosmological N -body codes due to its symplectic nature, and the KDK formulation is particularly helpful for the synchronisation when particles reside on different time steps following a block time step scheme. The computation during one time step is structured as follows: I. First, a half-step kick is executed, which means the velocities of the active particles are evolved by half a time step, given their acceleration, 12 12 12 We are using a notation where the subscript denotes the particle index and the superscript is not an exponent, but refers to the time instance. \mathbf{v}^{n+1/2}_{i}=\mathbf{v}^{n}_{i}+\mathbf{a}^{n}_{i}\,\Delta t_{i}/2\,. (23) II. The drift operation is executed, namely, the particle’s positions are updated by one time step, \mathbf{x}^{n+1}_{i}=\mathbf{x}^{n}_{i}+\mathbf{v}^{n+1/2}_{i}\,\Delta t_{i}\,. (24) III. Now, the forces acting on the particles resulting in their acceleration are computed. The acceleration of the particles results, for example, from gravity or hydrodynamics, \displaystyle\mathbf{a}^{n+1}_{i}=\, \displaystyle\mathbf{a}^{n+1}_{\mathrm{grav},ij}(\mathbf{x}^{n+1}_{j}) \displaystyle+\mathbf{a}^{n+1}_{\mathrm{hydro},ij}(\mathbf{x}^{n+1}_{j},...)+...\,. (25) IV. The pre-scattering velocities 13 13 13 Here, we use \mathbf{u}^{n+1} to denote the velocity before the velocity kicks due to the self-interactions have been applied. This allows us to distinguish it from the velocity \mathbf{v}^{n+1} after the scattering. are calculated by completing the KDK scheme with the second half-step kick, \mathbf{u}^{n+1}_{i}=\mathbf{v}^{n+1/2}_{i}+\mathbf{a}^{n+1}_{i}\,\Delta t_{i}/2\,. (26) V. Depending on the physics that are modelled, some corresponding computations, such as for black holes or cosmic rays, might be executed after the second half-step kick. VI. Last, before we return to I. for the first half-step kick of the next time step, we compute the DM self-interactions, \displaystyle\mathbf{v}^{n+1}_{i}=\mathbf{u}^{n+1}_{i}+ \displaystyle\mathbf{\Delta v}_{\mathrm{scatter}}^{n+1}\bigg(\mathbf{x}^{n+1}_{j}, \displaystyle\mathbf{u}^{n+1}_{j},\Delta t_{i},\frac{\mathrm{d}\sigma}{\mathrm{d}\cos\theta_{\mathrm{cms}}}\bigg)\,. (27) In the last step (VI.), we compute the self-interactions between the two half-step kicks. It is not common to all SIDM implementations to compute the velocity kicks resulting from the scattering at this place. For example, the implementation by 104 ; 103 (in Gadget-3 ) performs the scattering between the first half-step kick (I.) and the drift (II.) operations. The implementation in Gadget-2 by 140 follows the same scheme. For the SIDM implementation in GIZMO ( 107 , based on) , 14 14 14 As employed by 106 . the scattering is computed between the drift (II.) and second kick (III.) operations. Another alternative is the approach implemented in SWIFT , which splits the drift operation (II.) into two half-step drifts and executes the self-interaction in between those drifts ( 20 , actually it is a little more complicated, see sect. 2.3 by) . Possible differences arising from the choice of the position at which the self-interactions are computed have not yet been investigated thoroughly. However, given that the gravitational force computation does only depend on the positions of the particles, whereas the DM self-interactions only alter the velocities, a problematic interference arising from the SIDM computations appears to be less plausible. The adaptive time-stepping in OpenGadget3 is based on a block time-step scheme. It considers a particle to be active or passive depending on the block time step assigned to the particle and the time step that is being computed. The self-interactions are computed between pairs of two active particles and pairs of one active and one passive particle. As the neighbour search is performed for each active particle, we find the pairs consisting of two active particles twice per time step. In practice, we only perform the computations of the SIDM interactions when the identification number of the first particle of the pair is larger than that of the second particle. This way, we consider each pair only once and use the full time step for the scattering computation. 2.9 Time step criterion To ensure accurate simulation results, it is crucial to limit the numerical errors by choosing a sufficiently small time step. This can be achieved by formulating the time-step criterion to limit the interaction probability in the rare-scattering scheme. For the frequent-scattering scheme, the relative velocity change implied by the drag force should be kept small. In the following, we explain the formulation of the SIDM time-step criteria in OpenGadget3 . Starting from Eqs. ( 10 ) and ( 12 ), one can derive the time-step criteria. To control the size of the time step, we introduce the parameters \tau_{\mathrm{rare}} and \tau_{\mathrm{freq}} , which limit the interaction probability and the relative velocity change implied by the drag force, for the two schemes, respectively. For the rare scattering scheme, the time step criterion is \Delta t_{\mathrm{freq}}=\frac{\tau_{\mathrm{freq}}}{m_{\mathrm{n},x}\,\Lambda_{ii}}\left[\max_{v\,\in\,[0,\,v_{\mathrm{max}}]}\left(\frac{\sigma_{\mathrm{\tilde{T}}}(v)}{m_{x}}\,v\right)\right]^{-1}\,. (28) Analogously, for the frequent scattering scheme, it is \Delta t_{\mathrm{rare}}=\frac{\tau_{\mathrm{rare}}}{m_{\mathrm{n},x}\,\Lambda_{ii}}\left[\max_{v\,\in\,[0,\,v_{\mathrm{max}}]}\left(\frac{\sigma(v)}{m_{x}}\,v\right)\right]^{-1}\,. (29) Some quantities in the equations for the interaction probability and the drag force are subject to substantial noise and depend on which neighbours a particle sees. For example, the maximum relative velocity that occurs for all the pairs of which a particle is part in a given time step may fluctuate. The size of this fluctuation also depends on the number of neighbours used to determine the kernel size. We aim for a stringent and robust time step criterion. To cope with these fluctuations, we try to estimate the maximum of the kernel overlap integral and the maximum of the product of cross-section and relative velocity. For a discussion on how to build a SIDM time-step criterion, we refer the reader to appendix B by 39 . For the kernel overlap, we use the self-overlap, i.e. \Lambda_{ii} , in Eqs. ( 28 ) and ( 29 ). This gives an upper bound on the value that \Lambda_{ij} possibly can take. We note that the use of \Lambda_{ii} is based on the assumption that the set of particles employed in determining the kernel size is the same as for the formation of the interaction pairs. In case this is not applicable, a different formulation could be favourable ( 40 , see Sect. 2.4.2 by) . Estimating the maximum value for \sigma_{X}(v)/m_{x}\,v can be more complicated. Some velocity-dependent cross-sections have a well-defined global maximum that could be used. Nevertheless, we keep track of the maximum relative velocity, v_{\mathrm{max}} , that a particle has seen in the last time step. This allows us to limit the relevant domain within which we want to find the maximum of \sigma_{X}(v)/m_{x}\,v , as expressed in Eqs. ( 28 ) and ( 29 ). In the case of the full differential cross-section implementation, we have a lookup table for this maximum as a function of v_{\mathrm{max}} . But for the velocity-dependences expressed in Sect. 2.6 , a different approach is possible. In this case, there is a global maximum for \sigma_{X}(v)/m_{x}\,v at a velocity v_{e} . If v_{\mathrm{max}}>v_{e} , we use the global maximum, \sigma_{X}(v_{e})/m_{x}\,v_{e} , for the time-step criterion. Otherwise we employ \sigma_{X}(v_{\mathrm{max}})/m_{x}\,v_{\mathrm{max}} , as \sigma_{X}(v)/m_{x}\,v is monotonically increasing for v