From sectorial coarse graining to extreme coarse graining of S&P 500 correlation matrices
Manan Vyas 1,2,∗, M. Mijaíl Martínez-Ramos 1,3, Parisa Majari 1,4, Thomas H. Seligman 1,2 · 2025
Read From sectorial coarse graining to extreme coarse graining of S&P 500 correlation matrices by Manan Vyas 1,2,∗, M. Mijaíl Martínez-Ramos 1,3, Parisa Majari 1,4, Thomas H. Seligman 1,2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05463.txt.
Introduction
Manan Vyas and co-authors shrink the correlation matrix of S&P 500 stock returns down to a two-by-two matrix, the most extreme coarse view that still keeps mean correlation as a parameter. A full matrix has far too many values to read as a market gauge.
They split stocks into two sets, average the matrix over the blocks that split creates, and keep that block structure. Sector-based views sit between the full matrix and this bare form. The aim is a tiny set of numbers that still tracks how stocks move together in stress and calm.
Catalog record
| CREATOR | Manan Vyas 1,2,∗, M. Mijaíl Martínez-Ramos 1,3, Parisa Majari 1,4, Thomas H. Seligman 1,2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05463
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From sectorial coarse graining to extreme coarse graining of S&P 500 correlation matrices Manan Vyas 1,2,∗, M. Mijaíl Martínez-Ramos 1,3, Parisa Majari 1,4, Thomas H. Seligman 1,2 Abstract Starting from the Pearson Correlation Matrix of stock returns and from the desire to obtain a reduced number of parameters relevant for the dynamics of a financial market, we propose to take the idea of a sectorial matrix, which would have a large number of parameters, to the reduced picture of a real symmetric 2 × 2 2\times 2 matrix, extreme case, that still conserves the desirable feature that the average correlation can be one of the parameters. This is achieved by averaging the correlation matrix over blocks created by choosing two subsets of stocks for rows and columns and averaging over each of the resulting blocks. Averaging over these blocks, we retain the average of the correlation matrix. We shall use a random selection for two equal block sizes as well as two specific, hopefully relevant, ones that do not produce equal block sizes. The results show that one of the non-random choices has somewhat different properties, whose meaning will have to be analyzed from an economy point of view. keywords Pearson correlation matrices, Coarse graining, S&P 500 market states, k -means clustering, Dimensionality reduction, Multivariate analysis, Econophysics 1 Introduction A central problem of analyzing correlations of a set of time series measured or observed from the same or a related complex system is how to extract the relevant information. The term ‘relevant’ seems to imply two aspects, both important: for econophysics, the primary goal may be defined as understanding the dynamics of the system; for econometrics, the primary goal is making predictions. The two goals are not mutually exclusive, but rather mutually supportive and have a significant overlap. In this paper, we propose to focus on the former option and base our work on the use of states of a financial market by means of clustering of the Pearson correlation matrices of end-of-day returns taken with respect to those of the previous day. We choose a set of stocks, N in number, exchange-traded in the same time zone and specifically those traded in New York and as listed under the S&P 500 index [ 1 , 2 , 3 , 4 , 5 , 6 ] . Other techniques may be spectral analysis [ 7 , 8 ] or principal component analysis [ 9 , 10 ] . All of these methods provide us with information in very high-dimensional spaces, as is quite natural for “big data”. Machine learning and Artificial Intelligence, in general, might be a more modern way to tackle the problem, at least for econometrics, but this may offer less access to understand dynamics, at least with current state of these tools. The tool of our choice is the concept of using discrete states of a financial market [ 1 ] obtained by clustering the correlation matrices into a reduced number of finite and typically fairly large sets. This yields a simple dynamics of jumps between several sets, whose number, in our experience, varies between 4 and up to 12 in one instance [ 1 , 2 , 3 , 4 , 5 , 6 ] . The time evolution of the system in this set of states provides a transition matrix, and its properties take a central role. The time evolution can easily be represented by dots on lines corresponding to each state. We rely on looking for simple images that may show us some outstanding and maybe unexpected features of the data and in turn might give some insight into the underlying dynamics. Two or three-dimensional images are certainly the preferred options. Dimensional scaling [ 11 ] is a powerful tool and was used successfully in the context of correlation matrices [ 2 , 12 , 3 ] and the interpretation of these pictures shows the dominance of the largest eigenvalue or, almost equivalently, the average correlation of the entire matrix. This is not surprising as the “state of the market” [ 1 ] is precisely defined by this parameter. Market states, as introduced in [ 1 ] and expanded in [ 2 , 3 , 4 , 5 , 6 ] , deliver additional insight but the meaning of the remaining coordinates remains elusive. On the other hand, a recent paper [ 6 ] revealed further important properties of market states, especially the COVID state. This state is characterized by strong reduction in correlations, but definitively above the average correlation associated to the lowest market state. Also, comparison with the time horizon until 2019 showed that the COVID state did not exist previously. Indeed, the shrinking to a single market state, which essentially is independent of the label of these states, persists if ordered by the average correlation within each epoch, and the only difference we noted when moving the number of clusters from 5 up to 12 over a time horizon 2006 - 2023 is the label of this state. The meaning of this state is still not well understood. In the present paper, we propose a systematic method to obtain two additional parameters. We start from the idea of forming a coarse grained (CG) matrix, i.e. we use the idea proposed in [ 13 ] to separate the data matrix, and by consequence the correlation matrix, into a number of blocks. An analysis of this type was performed in [ 5 ] for market sectors, which seemed to conserve much of the market state properties. Note that the CG matrix is a real symmetric matrix, i.e., it has the features of a covariance matrix. This indicates that we have N(N+1)/2 potentially independent matrix elements, if we pass from the full correlation matrix to the sectorial one using the concept of CG. Then the correlation matrix is divided in sectorial blocks which are averaged over after eliminating the identities on the diagonal. This seems to be an efficient tool of coarse graining [ 5 ] the correlation matrix. Different kinds of coarse graining techniques have been utilized with great success in a variety of scientific problems [ 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 ] We plan to extend the CG method for correlation matrices to reduce the number of parameters to three due to symmetry, using only two blocks, which is the extreme case. One of the three parameters can be the average correlation, as this will just be the average of the CG correlation matrix. The latter is important as it essentially conserves the “state of the market” concept. We follow the line proposed in [ 5 ] , where the CG was performed according to the sectorial classification of the S&P 500 into 10 blocks, determined by the number of sectors. We will use just two blocks, which we shall choose at random but of equal size, or intuitively depending upon the intra- and inter-sectorial correlations. In the next section, we shall present the data and methodology we use. In the following section, we shall try to divide each data set into two plausible