Meteorological Command and Control:

An End-to-end Architecture for a Hazardous Weather Detection Sensor Network

Michael Zink, David Westbrook, Sherief Abdallah, Bryan Horling, Vijay Lakamraju, Eric Lyons, Victoria Manfredi, Jim Kurose

Dept. of Computer Science, and

Center for Collaborative Adaptive Sensing of the Atmosphere

University of Massachusetts Amherst, MA 01003

 

 

Kurt Hondl

National Severe Storms Laboratory

National Oceanic and Atmospheric Administration

Norman, OK 73019, USA

 

 

Abstract— We overview the software architecture for a network of low-powered radars (sensors) that collaboratively and adaptively sense the lowest few kilometers of the earth’s atmosphere. We focus on the system’s main control loop – ingesting data from remote radars, identifying meteorological features in this data, and determining each radar’s future scan strategy based on detected features and end-user requirements. Our initial benchmarks show that that these components generally have sub-second execution times, making them well-suited for our NetRad system.

 

    I.        Introduction

 

Distributed Adaptive Collaborative Sensing (DCAS) of the atmosphere is a new paradigm for detecting and predicting hazardous weather using a dense network of low-powered radars to sense the lowest few kilometers of the earth’s atmosphere [McLaughlin 2005]. Distributed refers to the use of large numbers of small radars, spaced close enough to “see” close to the ground in spite of the Earth’s curvature and avoid resolution degradation caused by radar beam spreading. Collaborative operation refers to the coordination (when advantageous) of the beams from multiple radars to view the same region in space, thus achieving greater sensitivity, precision, and resolution than possible with a single radar. Adaptive refers to the ability of these radars and their associated computing and communications infrastructure to dynamically reconfigure in response to changing weather conditions and end-user needs. The principal components of the DCAS system include sensors (radars); meteorological algorithms that detect, track, and predict hazards; interfaces that enable end-users to access the system; storage; and an underlying substrate of distributed computation that dynamically processes sensed data and manages system resources.  

 

NetRad is a prototype DCAS system whose goal is to detect a tornado within 60 seconds of formation and to track its centroid with a temporal error no greater than 60 seconds. At the heart (or perhaps more appropriately, the “brains”) of NetRad is its Meteorological Command and Control (MC&C) component that performs the system’s main control loop – ingesting data from remote radars, identifying meteorological features in this data, reporting features to end-users, and determining each radar’s future scan strategy based on detected features and end-user requirements. In this sense, NetRad is truly an end-end system, from the sensing radars through the computing and communication infrastructure and algorithms, to the end users.

 

In this paper, we describe the software architecture of NetRad’s MC&C and present initial benchmarks of its computational/communication requirements and performance.  The important components of the MC&C that we study in the paper are (i) the data ingest, field retrieval, and meteorological detection algorithms, (ii) a feature repository that maintains a multi-level grid of feature values with associated user-based utilities, and that generates new sensing tasks for the networked radars, and (iii) a resource allocation/optimization process that determines the radars’ scan strategy for the next system heartbeat. We discuss event notification mechanisms, and the computation of user-based utilities for competing sensing requests. We also discuss how NetRad timing considerations are addressed by structuring the feature repository as a blackboard system that temporally decouples data ingest/processing from the generation/optimization of future sensing activity.  Our initial benchmarks, obtained by evaluating NetRad components using reflectivity and wind velocity data from the NOAA NEXRAD WSR88D system [NOAA 2005] show that that these components generally have sub-second execution times, making them well suited for use in the NetRad system.

 

The remainder of this paper is structured as follows.  In the following section we briefly describe the overall NetRad system, and then dive down into the details of NetRad’s MC&C architecture. In section 3, we present benchmark execution times for various MC&C components.  Section 4 summarizes this paper and discusses our future research efforts.

 

 

 

    II.       NetRad System Overview and the MC&C Architecture.

 

We are currently building a NetRad prototype system to be deployed in southwestern Oklahoma, consisting of four mechanically scanned X-band radars atop small towers, and a central control site (later to be decentralized as the number of radars increases) known as the System Operations and Control Center (SOCC). The SOCC consists of a cluster of commodity processors and storage on which the MC&C components execute.

 

NetRad radars are spaced approximately 30 km apart from each other and together scan an area of 80km x 80km and up to3 km in height. Each radar is tasked to scan an angular sector of up to 360 degrees in 1-degree increments, with a range gate (radial voxel) size of 100 meters out to 30 km. With two elevation scans during each tasking, each radar can thus produce up to 360*300*2 = 216K reflectivity and velocity values each time it is tasked. While existing meteorological radar systems such as NEXRAD generally operate in “sit and spin” mode (taking full 360-degree volume scans independently of location and type of meteorological features present), NetRad radars are tasked by the MC&C to focus on volumes of high interest to end-users, as discussed below.

Each radar consists of three subsystems:

·  The Rotating Tower Top houses the radar unit and an embedded system that monitors the radar’s operational parameters and enables operator actions in the case of anomalies (e.g., mechanical problems).

·  The Non-rotating Tower Top Subsystem is located below the rotating joint and houses a data acquisition board (based on Field Programmable Gate Array (FPGA) technology), a radar controller, and a Gigabit Ethernet switch. The FPGA processes raw digitized data (at a rate of approximately 100 Mbps) into packets that are sent via the switch over a fiber optic cable to the Tower Base Subsystem. The radar controller controls the movement of the radar pedestal.

·  The Tower Base Subsystem consists of a compute cluster (currently just a single node) and an IDE RAID storage system, connected via a Gigabit Ethernet switch to a router. The tower base takes raw radar data, computes so-called moment data (essentially an average of multiple radar-pulse measurements for a given voxel of space), while performing quality control (e.g., attenuation correction and range folding) on this data. The 1 Mbps moment data is sent to the SOCC over OneNet [OneNet 2005], an IP network operated by the Oklahoma state regents, which is configured to provide 4 Mbps connectivity from each radar to the SOCC. Raw data is archived at the tower base storage and can be transferred to SOCC storage in the background.

The SOCC is a centralized compute cluster (later to be decentralized) interconnected via a Gigabit switch, on which the MC&C algorithms execute. As shown in Figures 1 and 2, the SOCC has five main components (i) data ingest and storage, (ii) meteorological feature detection and multi-radar merging, (iii) feature repository, (iv) utility and task generation, and (v) optimization.  We describe each of these functions below in more detail, roughly following the path taken by radar data through the MC&C, and the resulting re-tasking of the radars.

A.     Data Ingest and Storage

One SOCC computer is responsible for data ingest, archiving and distribution. Here, moment data (as well as the higher rate raw data, which is transferred at low priority) is streamed from the sensor nodes to the MC&C detection algorithms, and written to storage. In the future, both moment and raw data will be available to end-users via a query interface.

Data from each elevation scan is sent from a remote radar to the MC&C data ingest routines using LDM [LDM 2005], client/server middleware that reliably transfers radar data over a TCP connection. Given the pre-provisioning of OneNet bandwidth for NetRad use, congestion loss is not a concern in our initial testbed. However, we are currently developing a UDP-based transport protocol that uses application-specific selective dropping for congestion-control in bandwidth-constrained environments [Banka 2005]. As illustrated in Figure 2, the per-radar received reflectivity and wind velocity data for an elevation scan are converted to NetCDF format, and stored in a file. An event is then posted and distributed among the MC&C procedures using the Linear Buffer (LB) pub/sub construct of the WDSSII software [Hondl 2003]. In section 3, we report the time needed to transfer an elevation scan from sender to receiver over a Gigabit LAN switch using LDM, write the associated files and post/distribute an event.

 

B.     Meteorological Feature Detection, Multi-radar Data Merging

Once the notifications of available per-elevation reflectivity and wind velocity data have been posted, various meteorological detection algorithms can then read this data and perform feature detection.  Figure 2 shows two such algorithms: LLSD (for which we provide per-elevation execution times in Section 3) computes wind shear and rotational divergence; TDA performs tornado vortex detection. Other per-radar detection algorithms can be easily “plugged in” using the LB pub/sub mechanism. Once a detection algorithm completes its execution, notification is provided through the LB. The merge procedure converts polar coordinate data to latitude/longitude coordinates, and fuses together spatially overlapping data from multiple radars. We benchmark the performance of the merge routine in Section 3.

C.    The Feature Repository

NetRad system is a "real-time" system in the sense that radars must be re-tasked by the MC&C every 30 seconds – the system “heartbeat” interval.  This heartbeat interval was chosen based on the physical properties of the mechanically-scanned radars, the timescale over which atmospheric conditions change, and the system goal of detecting and tracking tornados within 60 seconds. A notion of heartbeat also allows the radars to easily synchronize their operation (e.g., having overlapping radars scan the same volume in order to perform 3D wind retrieval), and also helps simplify the optimization of radar targeting.  As we evolve from mechanically-scanned radars to rapidly reconfigurable solid-sate radars, we expect to relax the notion of a system heartbeat.

As discussed above, radars are retasked based on detected meteorological features and the projected future evolution of these features. In order to decrease the timing dependencies between the ingest/processing of radar data and the generation of radar commands, the MC&C adopts a blackboard-like architecture [Jaganathan 1989]. At the heart of the MC&C is the feature-repository, a multi-level grid that stores both the underlying per-voxel reflectivity and wind velocity data, as well as higher-level spatially-coherent meteorological “objects” such as storms cells, areas of high wind shear or precipitation, and tornados. Each object also has a position, a spatial extent for non-point objects, and a tag representing the meteorological phenomenon that the object represents (e.g., storm-cell, mesocyclone, and tornado). The multi-level grid-construction procedure writes this information into the feature repository as needed data becomes available via the linear buffer, as shown in Figure 2.

The generation of radar commands (the lower half of the control loop in Figures 1 and 2) proceeds asynchronously from the input processing of data (the upper half of the control loop). In this decoupled architecture, detection algorithms continuously post their results to the feature repository. As shown in Figure 3, at 30 second intervals the task generation component posts a set of tasks based on current state of the feature repository, and the optimization component then processes this task set and generates a scan strategy for the radars for the next 30 second cycle. In this design, we have relieved the time pressure on the detection components and somewhat relieved the time pressure on the MC&C components, task generation and optimization. One consequence of this design is that data that is not processed and posted on the feature repository before the task generation begins will not be acted upon until the next cycle of the system. This allows the system to avoid stalling, while waiting for late-arriving data (e.g., due to unanticipated network and processing delays). We are interested in the effects of this "decision lag" and also its relationship to the selection of the value of the system heartbeat.

Figure 3 illustrates that the data-driven streaming retrieval and detection algorithms write the results of their execution into the feature repository.  Starting D time units (where D is the execution time of the task generation and optimization algorithms) before the radars are to be re-targeted, the task generation process executes, followed by the optimization process. These processes may use all data available (in both the current time step, and previous time steps) in their computations.  Once the optimization process has completed, the radars are then re-targeted for the next 30 second cycle.

We note here that the feature repository is the central system “data structure.” It is from here that meteorological objects can be obtained and subsequently delivered/displayed to end users.  It is here that assimilated exogenous data (e.g., from satellite or from NEXRAD) can be stored and merged with NetRad-generated data.

D.    Utility, Prediction, and Task Generation

Within the feature repository, each voxel and each object has an associated utility that represents the “value” of scanning that voxel/object during in the next heartbeat.  The utility value weights considerations such as the time since the voxel/object was last scanned, the object type (e.g., scanning an area with a tornado vortex will have higher utility than sensing clear air), and user-based considerations such as the distance from a population center (e.g., among two objects with identical features, the one closer to a population center will have higher utility). 

We are currently developing a predicting component (shown with dashed lines in Figure 2) that tracks meteorological phenomena (e.g., a storm’s centroid) and predicts their future locations. New objects, corresponding to the predicted future locations of the phenomena, can then be added into the feature repository – allowing these predicting modules to be easily integrated into the current architecture.