大型集成數據集的深度學習輔助基於圖像的可視化

Abstract

為了研究不同的物理現象，科學家們經常在超級電腦上運行電腦模擬，以生成不同初始模擬參數的數據集。分析數據集的常見做法是將數據集從超級電腦移動到磁盤，並在後分析機器上分析數據集。隨著數據規模的增長，連接到超級電腦的有限的帶寬和存儲空間成為數據分析管道的瓶頸。為了支持大規模數據分析和可視化，我們提出了一種深度學習輔助的基於圖像的方法。我們的方法產生了一個小型的基於圖像的數據代理，具有較低的圖像分辨率和較低的原位像素射線採樣率，以減少輸入和輸出時間和磁盤存儲空間需求。深度學習模型經過高級訓練，可將小型數據代理恢復到常規採樣率和圖像分辨率，以實現高質量數據可視化和探索。我們評估並表明我們的方法優於多種選擇。To study the physical phenomenon, scientists often run the computer simulation on the supercomputer to generate datasets of different initial simulation parameters. The common practice to analyze the dataset is moving the datasets from the supercomputer to the disk and analyze datasets on the post analysis machine. When the data size grows, the limited bandwidth and the storage that connects to the supercomputer becomes the bottleneck of the data analysis pipeline. To support the large-scale data analysis and visualization, we propose a deep-learning assisted image-based approach. Our approach produces a compact image-based data proxy with a lower image resolution and a lower sampling rate along pixel rays in situ to reduce the I/O time and disk storage requirement. A deep learning model is trained in advanced to recover the compact data proxy to regular sampling rate and image resolution for high quality data visualization and exploration. We evaluate and show that our approach outperforms multiple alternative.