Lossy Data Compression

Lossy Data Compression

Post-processing a significant amount of data in STAR-CCM+ for flow visualization in the fluid region is a common practice, but the associated solution history file size quickly becomes prohibitive. To tackle this issue, STAR-CCM+ offers two types of compression mode for solution histories: Lossless and Lossy. The Lossless compression mode utilizes the standard binary compression so that precision is not lost during storing field variables. Conversely, the Lossy achieves greater compression by trading off precision of the field variables. In STAR-CCM+, HDF5 scale offset algorithm is implemented for the lossy compression mode with the D-Scaling option. More details about the algorithm can be found in HDF5 Scale Offset Algorithm. Essentially, user can utilize the “preserved digits” in STAR-CCM+ to control the D-scaling factor and the associated loss of precision, as shown in Figure 1.

Fig. 1Figure 1. Activation of Lossy compression mode.

The “preserved digits” can range from 1 to 8 and this value is referred to as “the number of significant digits of a floating point value that is kept when applying compression” in the STAR-CCM+ User Guide. In practice, the “preserved digits” appears to be preserving the range of orders of magnitude across the dataset of a field variable. Instead of trimming significant digits after the “preserved digits” for every data point within a dataset, Lossy compression mode reduces the data points which is out of the designated range of orders of magnitude to zero and truncates the significant digits for the remaining data points. Moreover, the range counts down from the maximum value of the dataset, and the maximum values within the same order of magnitude of the dataset do not affect the data point reduction process, as shown in Figure 2.

Fig. 2Figure 2. An example of setting “preserved digits” to 4 with different maximum values within the same order of magnitude.


In Figure 3, another example of setting the “preserved digits” to 4 shows that the Lossy compression mode cuts off values below 4 orders of magnitude from the maximum value. If the maximum value is 1.187645e-3, then the minimum cutoff value is 1.0e-6. All values below this number will be cut off, unlike the expected case of trimming down to 4 significant digits for each data point. The rule of thumb for individual field variables: 
​


Fig. 3Figure 3. A comparison between the expected case and in practice when setting "preserved digits" to 4.














    • Related Articles

    • Introduction to Simcenter STAR-CCM+ Webinar

      What's in this webinar? SDA Software Director of Engineering Ted Blowe presents this webinar on Introduction to Simcenter STAR-CCM+. The following topics will be discussed: Understanding Simcenter STAR-CCM+: Simcenter STAR-CCM+ serves as a versatile ...