Background
Solar Energetic Particles (SEPs) are key drivers of space weather, presenting significant radiation risks to humans and technological systems in space. SwRI has successfully developed MEMPSEP (Multivariate Ensemble of Models for Probabilistic Forecast of Solar Energetic Particles), a hybrid model using physics and neural networks to forecast SEP events – providing key advantages such as uncertainties to the predictions, and the forecast of occurrence and properties of the SEP event. However, MEMPSEP has so far operated only on historic data (2001–2013). To meet operational space weather requirements and make it market-ready for upcoming opportunities, validating MEMPSEP on near real-time (NRT) data was essential, and this was the objective of this IR.
Approach
Figure 1: Top-level chart showing MEMPSEP transition to operational domain.
The project transitioned MEMPSEP from research to operations through two primary objectives: (1) pipelining a full NRT dataset into the model, including on-the-fly data processing; (2) developing end-user infrastructure to ensure operational demonstration and/or deployment at stakeholder websites. An automated NRT data downloader was built to access multiple data sources with latencies between 5-360 minutes. Outlier identification methods were implemented to process NRT data in under one minute. MEMPSEP was retrained exclusively on NRT-qualified inputs, and a graphical user interface (GUI) was developed to enable user interaction, visualization, and forecasting.
Accomplishments
The project exceeded all performance metrics with three key accomplishments:
- NRT Data Infrastructure: An automated downloader accesses 18 instruments every minute, processing observations as available.
- Model Performance: MEMPSEP retrained on NRT data achieved 82% accuracy, 83% detection rate, and True Skill Statistic of 0.65 on test cases.
- Operational Integration: A Python Streamlit GUI handles errors and visualizes forecasts. The GUI seamlessly masks unavailable inputs to maintain functionality.
Publications
Dayeh, M. A., Starkey, M. J., Chatterjee, S., Elliott, H. A., Hart, S., & Moreland, K. (2024). A Machine Learning-Ready Data, Processing Tool for Near Real-Time Forecasting. Proc. of IAC-24; D5,IP,12,x89662. arxiv: 2502.08555