What is it about?
We provide algorithms to compute the maxima, minima, the number of values in a given interval, frequencies, the mean and variance of the large high-dimensional data set. All these post-processing operations are done in the compressed data format (e.g in a low-rank tensor format, but this representation is not really important). All algorithms are formulated in an abstract setting without reference to a particular compressed format.
Featured Image
Photo by Franki Chamaki on Unsplash
Why is it important?
The amount of data is growing permanently. Very often the data are high-dimensional (e.g., each sample/point is characterized by many features). A new type of algorithms, which require only linear storage and computational cost, are required. We look at some common post-processing tasks which are too time and storage consuming in the uncompressed data format and not obvious in the compressed format, as such huge data sets can not be stored in their entirety, and the value of an element is not readily accessible through simple look-up.
Perspectives
Under certain assumptions, we will be able to solve very large high-dimensional problems, for instance, of size 10^20 or 100^300. Such high-dimensional problems appear in chemistry and physics (Hartree-Fock, Schroedinger, or Master-equations).
Dr. Alexander Litvinenko
Rheinisch Westfalische Technische Hochschule Aachen
Read the Original
This page is a summary of: Iterative algorithms for the post-processing of high-dimensional data, Journal of Computational Physics, March 2020, Elsevier,
DOI: 10.1016/j.jcp.2020.109396.
You can read the full text:
Resources
Contributors
The following have contributed to this page
