Abstract
We describe our past and continuing work to help geophysicists at the Air Force Phillips Laboratory, Hanscom Air Force Base, view together various sets of optical image and radar data of the Ionosphere, both real and model-generated. The data occupy an altitude regime from 200 to 700 km and are visualized in a 3-D perspective display. The optical and radar renditions of depletion structures can be seen in relation to each other.
We selected Data Explorer in order to develop a fully functional and versatile visualization software package that will provide a backbone not only for the current visualization activity, but which also has the expandability to meet the needs of future projects. Data Explorer permitted us to quickly set up some initial visualizations, support manipulation of these visualizations, and experiment with different techniques.
Data Explorer's use of the single field type to represent all data helped simplify the programming of these custom modules. These new modules when combined with the already useful modules of DX allowed us to create networks that are tailored for specific jobs. Several new modules have been created for importing specific data sets. Other modules help relate different data sets together in time.
One presentation approach uses a pyramid glyph to explore displays of scattered data based on various data values. In another approach, the optical images are converted to latitude and longitude data, and displayed as a movable transparent film that can be placed at various altitudes under, over, or through the radar data. Various other techniques such as a system for multispectral image fusion that provides automatic enumeration of parameter mappings, and stereoscopic displays, are currently being explored to enhance the visualizations. Our future work, which we will involve increasing the number of input data sets up to ten, and will include flat images, scattered points, 3D fields, and numerous data curves all to be incorporated into a single visual network for interactive manipulations. These extensions will provide comprehensive support for integrated displays of additional types of ionospheric data.
1. Introduction
Geophysicists at Phillips Laboratory, Hanscom Air Force Base, Massachusetts, require visualization tools to study structures in the ionosphere. Their most recent work in this area centers around the observation of depletion structures from sites in Chile, the optimal locations for watching the south end of these ionospheric irregularities that run South to North along Earth's magnetic field lines. Needed are the means for accessing various data sets, transforming these sets into meaningful representations, and displaying representations for the geophysicists to explore and manipulate.
1.1. Background Information
The section of the ionosphere we have been observing covers about ten degrees of longitude, ten degrees of latitude, and roughly three hundred kilometers of altitude (150km to 450km). Though we are currently viewing only this small volume of the ionosphere, the project will eventually be extended to cover larger geographical areas and multiple sites. We are using two data sets provided by Phillips Laboratory, all-sky optical images and scattered radar point observations. The all-sky images were obtained with a digital camera that uses a fisheye lens to capture the night sky from horizon to horizon in all directions. These images are taken once a minute in each of three wavelengths, although only one depicts the depletion zones clearly. The radar data sets are obtained at several different base frequencies collected every ten minutes from the same sites as the images, and contain such information as Doppler frequency, line of sight velocities, and error variables. Of great interest are the relationships between the two types of data. Previous methods for seeing these two data sets required the ability to read optical images along side charts showing the scattered points in altitude and longitude for any given time. Our goal was to combine these two sets into one easy to read display.
1.2. Visualization Package
The IBM Data Explorer visualization package was selected to provide the backbone to our visualization application. This decision allowed us to get an initial system up and running quickly, while providing for highly flexible development in the long term. Data Explorer provided a means to access the data, to roughly manipulate the data sets, and then to quickly get usable displays. We needed a system that also supported a large number of capabilities for visualization, yet which could be extended by programming, and which could be used in programming and application environments. Data Explorer is supplied with a standard set of data input, output, manipulation, visualization, and display modules, an initially powerful tool kit for viewing data sets of various formats and sizes. With some basic C programming, one can extend this collection to meet custom demands required for specific tasks. The standard application window of Data Explorer is the network-building interface for the selection, connection, and execution of modules during development. It also includes the tools for creating user interfaces for specific user interactions. Then, once a visualization package has been created, Data Explorer can be run in a custom mode opening only the required display windows and user interface windows. However, from any of the main windows one can view the network, the displays, or edit dialog boxes and interface controls.
2. Visualization Software
Data Explorer was selected as the primary visualization system backbone for its network flexibility, rapid prototyping, and programming simplicity. In comparison to other available commercial systems, Data Explorer showed promising potential for the current project of visualizing the ionosphere, for future projects for Phillips Laboratory, as well as for supporting research activities at the Institute for Visualization and Perception Research. The modular architecture provides great flexibility in the way data can be manipulated and viewed. The drag-and-drop approach to network and interface creation allows for fast prototyping. The use of a unified data field to represent all data types simplifies module programming.
2.1. Network Flexibility
The modular approach used by Data Explorer to create visual networks greatly increases the flexibility of the overall system. Although there are only a small number of supplied modules, this set will handle most user needs. Most of these modules contain several user-defined parameters allowing precise control as to the module's actions. Other modules give explicit control to the user to change field components and data values. The overall functionality of all the modules provides for numerous possibilities of inter-connection and arrangement.
2.2. Rapid Prototyping
The simplicity to create visual networks and user interfaces in Data Explorer enables rapid prototyping and testing. The drag-and-drop approach to the placement of network modules and interface elements provides a quick method for developing visual packages. The entire arrangement around mouse controls simplifies the assembly process and increases productivity. The design of custom interface windows for specific input modules is reduced to defining the visual requirements of the input device, leaving the actual window-specific processes to Data Explorer. The user need only open the desired input window, select the specific module which needs a user interface element, and from the menu select the new element. With a few more mouse clicks, one can resize the interface, change its properties, and set default values. The process is basically mimicked for the creation of networks. The developer selects the desired tool to add from the list of available modules, then one simply drags and drops the modules into the network where the modules are connected. Hence, the overall approach to creating and changing the networks and interfaces can be performed relatively quickly.
2.3. Programming Simplification
With the use of the field data-type, programming custom modules has been greatly simplified. Once a developer understands the field data structure and how each field has several associated components with related data arrays, the creation and manipulation of such fields becomes just another data type to use in Data Explorer. Use of the various provided functions removes most of the complexity of programming modules to access the various data sets that are used in the visualization. After developing the first user-defined module, a basis is then created for other user modules, which can evolve over time with experience. The hardest part to developing new modules is keeping them simple and confined to a single problem in order to be reusable. It is advisable to study carefully the existing modules since many of the initial modules can perform many of the tasks which a user requires. However, special purpose modules, modules created to fulfill a specific task not initially available with the supplied Data Explorer module set, are perfect instances where user-defined modules can fit. One specific instance for creating a module would be the access to particular data files that have unique formats and where there are several files available. Using Data Explorer's data prompter to create a usable field from this data set would require a new file input description for each file name, whereas creating a new defined input module could provide a way to access the data by file name that could even be automated for multiple files.
3. Extended Module Set
In order to customize the resulting package and simplify the data selection process, we created several new modules for the already powerful module collection. Data-importing modules were created to read the unique file formats of the supplied data sets. Other modules were developed to arrange the various data sets in time. Finally, automated collection modules were written that read files and data sets based on their relationships to currently visualized data.
3.1. Data Importing
The first set of additional modules we created were for data importing. These modules simplify the input process of accessing our particular data sets. Two input modules were created, one module to read a binary image file and another to read structured text files containing the scattered radar data values. The all-sky images used contained an ASCII header section that holds the specifics of the images and a comment area that holds the camera settings for the images. At the end of this header is a time stamp that requires decoding from a binary format. Then follows the binary data, which can be represented in either eight- or twelve-bit format arranged by image lines, left to right, and top to bottom. The new module outputs a resulting field containing the image data and a string containing the converted image time stamp. This module also performs additional processing of the image positions, which is discussed below under current techniques. The second input module accesses the scattered radar data file types. However, the module performs more than just reading in a file. It reads in all such files from a given directory. Each file in the directory contains the set of values over generally the same time space but created from different frequencies. The module outputs one field containing all the data from each file. This field then represents the entire data set for all test frequencies for the given time period of the observation. Thus, we can now open the images and radar data and create a field containing these values for creating the visualization.
3.2. Time Sequencing
Other modules written include ones that can run through a time sequence of the data set, and correlate the images to the radar data. Because Data Explorer's built-in sequencer module uses only integer values, and because our times are represented as decimal hours, we needed a way to relate the two time sequences. The easiest approach was to create a time field using the actual times as positions with integer data values representing the relationship to a monotonically increasing integer sequence that the sequencer module provides the index value into. Currently we base the actual times on the ten minute intervals provided by the radar data. This limits us to using a single image per time slot, since ten images are actually created during the ten minute interval used to collect the radar data. The image occurring five minutes into the interval is used to relate the optical data to the radar data. We are now devising ways to use more images, maybe by setting the sequencer module time to the images or by taking an average of several images over the time interval for one radar data set.
3.3. Automated Collection
In order to help simplify the controls of the system, we are performing many data collections automatically, especially in regards to the correlation of the images to the radar data. Currently we first produce the time field from the radar data reader. Then we open the required set of images that relate to each radar time, given some position within the defined time interval, and produce a running list of image file names with related times. Then the user supplies the current integer time to this new module, which in turn outputs the image file name for the current image that is to relate to the radar data of the current time. With these automated tools in place, the user interface for running the system uses only three tools for actually getting the system up and running. First, the user needs to supply the directory and name to one of the scattered radar data sets. Next, the user supplies the directory and name to one of the images. From these two inputs, the package reads and opens all the radar data sets and the related images automatically, allowing the user to proceed to use the "sequencer" to control motion through the data by time.
4. Current Techniques
With Data Explorer providing the foundation, and several new modules written for customization, we needed new techniques to view the data in a 3-dimensional space. We currently perform image transformations that make it easy to relate the data sets, allow for data selection within data sets, and use the built-in visual manipulation tools to alter the resulting displays.
4.1. Image Transformations
We transform the raw image to undo the fisheye lens affect, transforming the image into one that is related to World latitude and longitude coordinates for a given altitude. Thus, the resulting image is an idea of what is seen in the sky at a given altitude from horizon to horizon. By connecting the resulting image points we can smooth the image by the built-in interpolation processes of Data Explorer when rendering the display. After this transformation we reduce the image by setting bounds on latitude and longitude to an area of interest which the radar data encompasses; given that we are only interested currently in the relationships between the two data sets, and the radar data only covers the central portion of the image. Another recent addition to the image transformation includes an option to generate a three-dimensional field from the image by stretching the image in altitude, and using the isosurface module produces a synthetic approximation to the walls of the depletions. The importance of this addition comes from the need to understand the relationship between the radar points and the dark bands visible on the images. The isosurface module accentuates the wall structure helping to visualize the relationship of the walls to the radar data.
4.2. Data Selection
The scattered radar data is supplied with latitude and longitude positions with a group range value that partially represents an altitude for providing the point positions. For each position there are several related values including the particular source frequency, a Doppler frequency, and a line of sight velocity. With a user input selection window, the observer can choose which particular value to view. We map two values to glyphs that control size and color. We are using pyramid glyphs for their simplicity. Our generated pyramids are centered on the radar data positions with solid coloring to make them visually standout. Though Data Explorer is supplied with various automated glyphs, these default glyphs are flattened when scaling the display. We correct this by providing initially distorted glyphs that scale to the desired final state. We color the glyphs based on their positions by unique color maps for each particular data values.
4.3. Visual Manipulations
Through the use of Data Explorer's display tools, the user can visually manipulate the resulting image. Our rendered image exists in a three-dimensional environment which can be rotated and scaled to view various aspects of the relationships between data sets. Our initial attempt combined the flat image at a set altitude with the scattered radar points above and below the image plane, depend on the image settings. To convey the three-dimensional space, we visualized the volume with Data Explorer's perspective rendering capabilities. However, by looking from the front of the data sets, a view facing North in World coordinates, the user could not readily relate the latitude and longitude positions of the radar points to the image. The user was required either to rotate the image from the current position to an overhead position or work with a second image of the radar data seen from overhead. To resolve this conflict, we generated the three-dimensional stretch image developed above, which partially resolves the problem. Other approaches are currently been examined to remedy this situation. For now, the user must rely on the use of Data Explorer's manipulation tools to move around the three-dimensional space to gain an understanding of the interactions.
5. Conclusion and Future Work
We are embarking on new improvements and additions to increase the geophysicists' understanding of the ionosphere. The next steps include the addition of several new data sets, the incorporation of various models, and an extension toward stereoscopic displays.
5.1. Increased Data
We are currently using only two sources of information, the images and the radar data. However there are at least six more data sources eventually to be included. Some of the new data comes from sources similar to those currently being studied, while others, like satellite information, will be completely new. In order to effectively handle all these new data sets we are in the process of developing a multiple data set interface to handle the multiple file formats, required transformations, and other conversions to produce a standard output that Data Explorer can use. Numerous models will be incorporated as fixed data sets or calculated during execution time. These will add many more dimensions to the ever growing number of values available for various positions and times in this visualization package.
5.2. Stereoscopic Display
Another visualization technique to be tried will be the development of stereoscopic displays. We currently use a Cyberscope, a hardware monitor attachment that contains an arrangement of mirrors used to create a fused stereoscopic image from two displaced images. We are also looking into using crystal glasses to produce stereoscopic images with multiple glasses allowing for multiple users. This will provide a way of much more naturally looking around the three-dimensional space we are dealing with. Both techniques would require new modules to Data Explorer to produce the new types of displays.
Acknowledgments
The authors acknowledge the support by the Phillips Laboratory through contract F19628-90-K-0029.
GEOVIS - The initial Data Explorer setup for Phillips Labs' ionospheric visualizations.
All-Sky images: shown here in gray with an adjustablegreen highlight area for general tracking of the depletions.
