Accelerating in-transit co-processing for scientific simulations using region-based data-driven analysis

Marcus Walldén, Masao Okita, Fumihiko Ino, Dimitris Drikakis, Ioannis Kokkinakis

Research output: Contribution to journalArticlepeer-review

14 Downloads (Pure)


Increasing processing capabilities and input/output constraints of supercomputers have increased the use of co-processing approaches, i.e., visualizing and analyzing data sets of simulations on the fly. We present a method that evaluates the importance of different regions of simulation data and a data-driven approach that uses the proposed method to accelerate in-transit co-processing of large-scale simulations. We use the importance metrics to simultaneously employ multiple compression methods on different data regions to accelerate the in-transit co-processing. Our approach strives to adaptively compress data on the fly and uses load balancing to counteract memory imbalances. We demonstrate the method’s efficiency through a fluid mechanics application, a Richtmyer–Meshkov instability simulation, showing how to accelerate the in-transit co-processing of simulations. The results show that the proposed method expeditiously can identify regions of interest, even when using multiple metrics. Our approach achieved a speedup of 1.29× in a lossless scenario. The data decompression time was sped up by 2× compared to using a single compression method uniformly.
Original languageEnglish
Article number154
Number of pages22
Issue number5
Publication statusPublished - 12 May 2021


  • visualization
  • parallel computing
  • in-transit
  • co-processing
  • supercomputers
  • data visualization
  • simulation


Dive into the research topics of 'Accelerating in-transit co-processing for scientific simulations using region-based data-driven analysis'. Together they form a unique fingerprint.

Cite this