氣象即時資料分析與探勘──以Hadoop分散運算為基

Analysis and Exploration of Real-time Meteorological Data Based on Hadoop Distributed Computing

近年來，氣象資料的更新頻率日益快速，預報系統必須在短時間內不斷儲存並製作圖像資料，且隨著觀測準確度的提升，資料品質及可描述量皆有顯著的提升，氣象預測領域儼然已符合巨量資料的範疇。在巨量資料的時代，如何從龐大且雜亂的資料中，找出有效的資訊來提升組織營運或是應用的效率是相當重要的。因此本研究提出基於Hadoop框架模型的建置，並藉由分散式運算的運作整合HDFS、MapReduce等相關技術，使用資料探勘分析工具建構出氣候資訊的模型架構，最後產生淺顯易懂的資訊、報表，提供給使用者作為災害預防之依據及使用，達到快速處理與分析巨量資料之目的。

In recent years, the update frequency of climatological data become faster than ever. The climate warning system continues storing and producing multimedia data in a short time, and with accuracy enhancement of climate observations. The data quality and quantity about climate have been significant improvement, just like weather forecasting has been an application belong to Big Data development. In an era of Big Data, from large and messy data, identifies effective and hidden information to improve operational efficiency of enterprises is very important. This paper proposes the concept of Hadoop-based framework model with distributed computing capabilities. The Hadoop framework integrates with HDFS, MapReduce and other related approaches, this system also uses data mining tools to construct the analysis model of climatological data. The final results under data mining procedure information and reports are easy to understand. Those data complete the goal of fast processing about Big Data and get valuable data for related uses to decrease the problems of climatological diasters.