人工智能机器学习Operation of a Data Acquisition,Transfer,and Storage System for the Global Space-Weather Observation Network

DataScienceJournal,Volume13,30October2014

OPERATIONOFADATAACQUISITION,TRANSFER,ANDSTORAGE

SYSTEMFORTHEGLOBALSPACE-WEATHEROBSERVATION

NETWORK

TNagatsuma1*,KTMurata1,KYamamoto1,TTsugawa1,HKitauchi1,TKondo1,

HIshibashi1,MNishioka1,andMOkada2

1NationalInstituteofInformationandCommunicationsTechnology,4-2-1Nukui-kita,Koganei,Tokyo184-8795,Japan

*Email:tnagatsu@nict.go.jp

2NationalInstituteofPolarResearch,10-3Midorichou,Tachikawa,Tokyo190-8518,Japan

ABSTRACT

Asystemtooptimizethemanagementofglobalspace-weatherobservationnetworkshasbeendevelopedbythe

NationalInstituteofInformationandCommunicationsTechnology(NICT).NamedtheWONM(Wide-area

ObservationNetworkMonitoring)system,itenablesdataacquisition,transfer,andstoragethroughconnection

totheNICTScienceCloud,andhasbeensuppliedtoobservatoriesforsupportingspace-weatherforecastand

research.Thissystemprovidesuswitheasiermanagementofdatacollectionthanourpreviouslyemployed

systemsbymeansofautonomoussystemrecovery,periodicalstatemonitoring,anddynamicwarning

procedures.OperationoftheWONMsystemisintroducedinthisreport.

Keywords:WONMsystem,Dataacquisition,Globalobservationnetwork,Spaceweatherforecast,NICTScienceCloud

1INTRODUCTION

TheEarth’smagnetosphereisformedbyinteractionbetweenthesolarwindandtheEarth’smagneticfield.Solar

windconditionschangeaccordingtochangesinsolar-activity.Thus,disturbancesinthespaceenvironment

aroundtheEarth,called‘geospace’,aredrivenbybothtransientandrecurrentsolaractivities.Thegeospacehas

beenrecognizedasakeyareaforhumanendeavorsinspaceandalsoforsocialinfrastructure,whichis

vulnerabletogeospacedisturbancesdrivenbysolaractivities.Tomitigatetheriskscausedbygeospace

disturbances,spaceweatherforecastsareoffundamentalimportance.

TheNationalInstituteofInformationandCommunicationsTechnology(NICT)istheinstituteresponsiblefor

space-weatherforecastinginJapanandisaRegionalWarningCentreoftheInternationalSpaceEnvironment

Service.Toprovidenowcastingandforecastingofspace-weatherinformationasoperationalservices,real-time

monitoringofsolaractivityandthegeospaceenvironmentareessential.Moreover,real-timemonitoringis

usefultocheckthecurrentstatusofobservationalfacilitiesandtheconditionofthedatanetwork.Withthese

considerationsinmind,wehavebeendevelopinganear-realtimedataacquisitionandtransfersystemfor

space-weathermonitoring,followingrecentprogressininformationandcommunicationstechnologies

(Ishibashi&Nozaki,1997;Nagatsuma,Obara,Ishibashi,Hayashi,&McEwen,1999).NICThasbeen

promotingtheNICTSpaceWeatherMonitoringNetwork(NICT-SWM),aprojectwiththeaimofestablishinga

globalnetworkofspace-weatherobservations(Nagatsuma,2009;2013).Thebasicconceptoftheprojectisto

improvethereliabilityofspace-weatherforecastingbyintroducingreal-timedataobtainedbyourglobal

networkofspace-weatherrelatedobservationalfacilities,namely,ionosondes,magnetometers,high-frequency

radars,GlobalPositioningSystem(GPS)receivers,solarradiotelescopes,andasatellitedatareceptionsystem.

Dataanalysesofarchiveddataarealsoimportantforfurtherdevelopmentofspace-weatherforecastingmodels.

Collectioninnear-realtimeofspaceweathermonitoringdatafromalargenumberofobservatoriesdistributed

throughouttheworldandinspaceisanontrivialtask.Especially,theArcticandAntarcticregionsareimportant

gatewaysofsolarwindenergy,whichflowsfromthemagnetospheretotheionosphere.NICToperates

approximately30observatoriesintheNICT-SWMproject.Theseobservatoriescoverawide-areaoftheEarth,

includingtheArcticandAntarctica,andallobservationaldataaretransferredonareal-timebasistoNICTand

PDA51

DataScienceJournal,Volume13,30October2014

depositedinalargestoragesystemintheNICTScienceCloud(Murata,Watari,Nagatsuma,Kunitake,

Watanabe,Yamamoto,etal.,2013;Watanabe,Yamamoto,Tsugawa,Nagatsuma,Watari,Murayama,etal.,

2013).However,managingtheentireoperationhasbecomeincreasinglycomplexusingthelegacysystem

becauseitcontainsavastarrayofobservationalinstruments,eachhavingitsowncharacteristicsandconditions.

Problemsarebeginningtoamplifyasthedatatransfernetworkconnectsadditionalobservatories,andthereisa

shortageofhumanresourcestomaintaintheobservationalsystems.

Toovercometheseissues,wehavedevelopedanewintegratedmanagementsystemofglobalmultipoint

observations.ThedesignedandimplementedsystemisnamedtheWide-areaObservationNetworkMonitoring

(WONM)system;theconceptandcurrentoperationofwhichareshownthroughouttheremainderofthispaper.

2WONMSYSTEMCONCEPT

AschematicofthebasicWONMsystemconceptisshowninFigure1.Thissystemconsistsofa“clientserver”

ateachobservationsite,a“datatransfer”part(viatheInternet),and“acentralsystem”ataterminalsite.Itis

necessaryfortheWONMclienttohavean“automaticrecovery”functionwithhigh-leveltoleranceand

redundancycharacteristicstoassurestableoperationofthesystem.Furthermore,theuseofasmall-size,

low-power,andfan-lesspersonalcomputer(PC)serverisessentialforminimizingtheloadattheobservation

site.ThesoftwarefortheWONMclientcan,incontrast,beinstalledonpre-existingserversattheobservation

sitestoreusethecurrenthardwareresources.

Figure1.AschematicshowingthebasicconceptoftheWONMsystem

Datatransferisacentralissueofthissystem.Preparingahigh-performancenetworkbandisanoptimalsolution

forrapidandcontinuousdatatransferfromtheremotesites.However,realizingsuchanetworkperformanceis

difficultinpracticebecauseourobservationsitesareoftenlocatedinisolatedregionsworldwide.Toavoiddata

gapsduetointerruptionofthenetwork,functionsthatretrydatatransferandthatperformconsistencychecksof

thedatafilesmustbeincludedinthesystem.Moreover,becausenetworkpoliciesaresite-dependent,flexibility

isensuredbypreparingdifferenttypesofprotocolsfordatatransferinadvance.

Thecentralsysteminstalledattheterminalsiteneedsavaststoragecapacitywithanappropriatelevelof

redundancysothatalargenumberofdatafilescanbeheldwithhighreliability.Suchahardwareenvironmentis

availableintheNICTScienceCloud(Murataetal.,2013),anditisthereforeemployedastheWONMcentral

system.Thissystemalsorequiresmonitoringsoftwaretowarnoftheconditionofthenetworkandofdata

transfer,anddataandstatusinformationaretransferredfromeachobservationalfacilitytotheterminalsite

usingthe‘datacrawler’and‘statuscrawler’software,whichoperateonaPCserverattheterminalsiteorona

WONMapplianceserverinstalledattheobservatorysite.Whenbothofthe‘datacrawler’and‘statuscrawler’

softwareareinoperation,anddataandstatusinformationaresuccessfullytransferred,thedetailsarearchived

PDA52

DataScienceJournal,Volume13,30October2014

withintheterminalsitestorage.IfthePCserverattheobservationalfacilityflagsthatconditionsareabnormal,

orifthedataandstatusinformationarenotbeingtransferred,theWONMsystemwillgiveanalertmessageas

warninginformation.

Combiningtheaforementionedthreecomponents,theWONMsystemisexpectedtoacquire,transfer,andstore

dataproducedbyaglobalobservationnetwork.Althoughthissystemconceptisspecifiedasausecaseforspace

weathermonitoring,itisapplicabletoavarietyofresearchfieldsoperatinganetworkofobservational

instrumentsdistributedworldwide.

3CURRENTOPERATIONOFTHEWONMSYSTEM

TheWONMsystemwasdevelopedaccordingtotheconceptualdesignshowninSection2,andatrail

implementationcommencedoperationinFebruary2013.ItstechnicaldetailswillbedescribedinMurata,

Nagatsuma,Yamamoto,Watanabe,Ukawa,Muranaga,etal.(2014).OncewehadconfirmedthattheWONM

systemwasfullyfunctional,westartedtoreplaceourpresentdataacquisitionandmonitoringsystemfor

NICT-SWMwiththisnewsystem.LocationsofobservatoriescurrentlybeingmanagedbytheWONMsystem

areplottedonamapinFigure2,wherethesiteofeachobservatoryisdenotedbyaPCserver.Atthetimeof

writing,onlypartoftheNICT-SWMismanagedbythissystem.

Figure2.(Top)MapoftheobservatorynetworkmanagedbytheWONMsystem;(Bottom)Keyofstatusiconsusedinthemap

Figure3isascreengraboftheworld-mapwindowproducedfromtheWebapplicationinstalledintheWONM

system.ThestatusofeachobservatorycanbemonitoredbythisWebapplication,andthecurrentstatusesofthe

observatorieslocatedinAntarctic,Arctic,andSoutheastAsiaregionsaredisplayedusingdifferenticons,as

showninFigure2.Ifonewishestocheckthedetailsofafacility,simplyselectthatobservatoryfromthelistor

maptoviewthestatustimehistoryandspecificinformation.ThisWebapplicationthusenablesustomonitor

theentirenetworkinasingledisplay.

PDA53

DataScienceJournal,Volume13,30October2014

Figure3.ScreengrabofWebapplicationforbrowsingWONMinformationandthestatusofeachobservatorynode

Table1liststhefrequencyandnumber(size)ofdatatransferredviatheWONMsystem.SinceJuly2013,we

havealreadyarchivedabout2.3TBofdata,andwearereceivingabout8500datafilesperdayonaverage

(equivalenttoaround4GBofdata)fromgloballydistributedobservatories.

Table1.DailyfrequencyandaccumulatednumberofdatatransferredviatheWONMsystem(asofOctober2013)

PDA54

DataScienceJournal,Volume13,30October2014

AtimeseriesofthesizeandnumberofdatafilesrecordedintheNICTScienceCloudisplottedinFigure4.The

discontinuityinMay2013indicatesthepresenceofalongnetworkinterruptionand/orproblemsatalocaldata

transfersite.Aftertheproblemwasdetectedandrepaired,filetransferquicklyincreasedandrecoveredtothe

nominallevelofdatatransfer,suggestingthatdataacquisitionusingtheWONMsystemissmoothandstable.

Figure4.TimeseriesofsizeandnumberofdatafilesrecordedintheNICTScienceCloud:(Left)GPSdatafromChumphon;(Right)Frequency-ModulatedContinuous-WaveIonosondedatafromCebu

4CONCLUSION

WehaveshownthattheWONMsystemprovidesuswithanintegratedandefficientmeanstomanageanumber

ofspaceweatherobservatoriesdistributedworldwide.Wehavealsoshownthataninterruptionindata

acquisitioncanberecoveredautomaticallybythissystem.Futuredevelopmentsincludeincreasingthenumber

ofobservationsitestoimprovethespaceweathermonitoringandforecastingabilitycurrentlyofferedbythe

WONMsystem.Moreover,becausemanyprojectsemployingglobalobservationnetworksexperiencesimilar

difficultieswithdatastewardship,thebasicdesignofoursystemcanbeappliedtootherresearchfields.

5ACKNOWLEDGEMENTS

ThepresentworkisbasedontheNICTScienceCloudproject.IonosphericobservationsatSyowaStationare

basedontheconsignmentstudyfromtheMinistryofInternalAffairsandCommunications.

6REFERENCES

Ishibashi,H.&Nozaki,K.(1997)Developmentofintermagnet/HiraisoGINsystem.ReviewofCommunications

ResearchLaboratory43,pp291–299.

Murata,K.T.,Watari,S.,Nagatsuma,T.,Kunitake