14 Matching Results

Search Results

Advanced search parameters have been applied.

Using High-Speed WANs and Network Data Caches to Enable Remote and Distributed Visualization

Description: Visapult is a prototype application and framework for remote visualization of large scientific datasets. We approach the technical challenges of tera-scale visualization with a unique architecture that employs high speed WANs and network data caches for data staging and transmission. This architecture allows for the use of available cache and compute resources at arbitrary locations on the network. High data throughput rates and network utilization are achieved by parallelizing I/O at each stage in the application, and by pipe-lining the visualization process. On the desktop, the graphics interactivity is effectively decoupled from the latency inherent in network applications. We present a detailed performance analysis of the application, and improvements resulting from field-test analysis conducted as part of the DOE Combustion Corridor project.
Date: April 18, 2000
Creator: Bethel, Wes; Lau, Stephen; Tierney, Brian; Lee, Jason & Gunter, Dan
Partner: UNT Libraries Government Documents Department

Visapult: A Prototype Remote and Distributed Visualization Application and Framework

Description: We describe an approach used for implementing a highly efficient and scalable method for direct volume rendering. Our approach uses a pipelined-parallel decomposition composed of parallel computers and commodity desktop hardware. With our approach, desktop interactivity is divorced from the latency inherent in network-based applications.
Date: April 17, 2000
Creator: Bethel, Wes
Partner: UNT Libraries Government Documents Department

HDF5-FastQuery: Accelerating Complex Queries on HDF Datasets UsingFast Bitmap Indices

Description: Large scale scientific data is often stored in scientific data formats such as FITS, netCDF and HDF. These storage formats are of particular interest to the scientific user community since they provide multi-dimensional storage and retrieval. However, one of the drawbacks of these storage formats is that they do not support semantic indexing which is important for interactive data analysis where scientists look for features of interests such as ''Find all supernova explosions where energy >105 and temperature >106''. In this paper we present a novel approach called HDF5-FastQuery to accelerate the data access of large HDF5 files by introducing multi-dimensional semantic indexing. Our implementation leverages an efficient indexing technology called ''bitmapindexing'' that has been widely used in the database community. Bitmapindices are especially well suited for interactive exploration of large-scale read-only data. Storing the bitmap indices into the HDF5 file has the following advantages: (a) Significant performance speedup of accessing subsets of multi-dimensional data and (b) portability of the indices across multiple computer platforms. We will present an API that simplifies the execution of queries on HDF5 files for general scientific applications and data analysis. The design is flexible enough to accommodate the use of arbitrary indexing technology for semantic range queries. We will also provide a detailed performance analysis of HDF5-FastQuery for both synthetic and scientific data. The results demonstrate that our proposed approach for multi-dimensional queries is up to a factor of 2 faster than HDF5.
Date: December 7, 2005
Creator: Gosink, Luke; Shalf, John; Stockinger, Kurt; Wu, Kesheng & Bethel,Wes
Partner: UNT Libraries Government Documents Department

HDF5-FastQuery: Accelerating Complex Queries on HDF Datasets usingFast Bitmap Indices

Description: Large scale scientific data is often stored in scientific data formats such as FITS, netCDF and HDF. These storage formats are of particular interest to the scientific user community since they provide multi-dimensional storage and retrieval. However, one of the drawbacks of these storage formats is that they do not support semantic indexing which is important for interactive data analysis where scientists look for features of interests such as ''Find all supernova explosions where energy > 10{sup 5} and temperature > 10{sup 6}''. In this paper we present a novel approach called HDF5-FastQuery to accelerate the data access of large HDF5 files by introducing multi-dimensional semantic indexing. Our implementation leverages an efficient indexing technology called bitmap indexing that has been widely used in the database community. Bitmap indices are especially well suited for interactive exploration of large-scale read only data. Storing the bitmap indices into the HDF5 file has the following advantages: (a) Significant performance speedup of accessing subsets of multi-dimensional data and (b) portability of the indices across multiple computer platforms. We will present an API that simplifies the execution of queries on HDF5 files for general scientific applications and data analysis. The design is flexible enough to accommodate the use of arbitrary indexing technology for semantic range queries. We will also provide a detailed performance analysis of HDF5-FastQuery for both synthetic and scientific data. The results demonstrate that our proposed approach for multi-dimensional queries is up to a factor of 2 faster than HDF5.
Date: March 30, 2006
Creator: Gosink, Luke; Shalf, John; Stockinger, Kurt; Wu, Kesheng & Bethel,Wes
Partner: UNT Libraries Government Documents Department

DEX: Increasing the Capability of Scientific Data Analysis Pipelines by Using Efficient Bitmap Indices to Accelerate Scientific Visualization

Description: We describe a new approach to scalable data analysis that enables scientists to manage the explosion in size and complexity of scientific data produced by experiments and simulations. Our approach uses a novel combination of efficient query technology and visualization infrastructure. The combination of bit map indexing, which is a data management technology that accelerates queries on large scientific datasets, with a visualization pipeline for generating images of abstract data results in a tool suitable for use by scientists in fields where data size and complexity poses a barrier to efficient analysis. Our architecture and implementation, which we call DEX (short for dexterous data explorer), directly addresses the problem of ''too much data'' by focusing analysis on data deemed to be ''scientifically interesting'' via a user-specified selection criteria. The architectural concepts and implementation are applicable to wide variety of scientific data analysis and visualization applications. This paper presents an architectural overview of the system along with an analysis showing substantial performance over traditional visualization pipelines. While performance gains are a significant result, even more important is the new functionality not present in any visualization analysis software--namely the ability to perform interactive, multi-dimensional queries to refine regions of interest that are later used as input to analysis or visualization.
Date: February 4, 2005
Creator: Stockinger, Kurt; Shalf, John; Bethel, Wes & Wu, Kesheng
Partner: UNT Libraries Government Documents Department

Progress on H5Part: A Portable High Performance Parallel DataInterface for Electromagnetics Simulations

Description: Significant problems facing all experimental andcomputationalsciences arise from growing data size and complexity. Commonto allthese problems is the need to perform efficient data I/O ondiversecomputer architectures. In our scientific application, thelargestparallel particle simulations generate vast quantitiesofsix-dimensional data. Such a simulation run produces data foranaggregate data size up to several TB per run. Motived by the needtoaddress data I/O and access challenges, we have implemented H5Part,anopen source data I/O API that simplifies the use of the HierarchicalDataFormat v5 library (HDF5). HDF5 is an industry standard forhighperformance, cross-platform data storage and retrieval that runsonall contemporary architectures from large parallel supercomputerstolaptops. H5Part, which is oriented to the needs of the particlephysicsand cosmology communities, provides support for parallelstorage andretrieval of particles, structured and in the future unstructuredmeshes.In this paper, we describe recent work focusing on I/O supportforparticles and structured meshes and provide data showing performance onmodernsupercomputer architectures like the IBM POWER 5.
Date: June 22, 2007
Creator: Adelmann, Andreas; Gsell, Achim; Oswald, Benedikt; Schietinger,Thomas; Bethel, Wes; Shalf, John et al.
Partner: UNT Libraries Government Documents Department

VisPortal: Deploying grid-enabled visualization tools through a web-portal interface

Description: The LBNL/NERSC Visportal effort explores ways to deliver advanced Remote/Distributed Visualization (RDV) capabilities through a Grid-enabled web-portal interface. The effort focuses on latency tolerant distributed visualization algorithms, GUI designs that are more appropriate for the capabilities of web interfaces, and refactoring parallel-distributed applications to work in a N-tiered component deployment strategy. Most importantly, our aim is to leverage commercially-supported technology as much as possible in order to create a deployable, supportable, and hence viable platform for delivering grid-based visualization services to collaboratory users.
Date: June 9, 2003
Creator: Bethel, Wes; Siegerist, Cristina; Shalf, John; Shetty, Praveenkumar; Jankun-Kelly, T.J.; Kreylos, Oliver et al.
Partner: UNT Libraries Government Documents Department

NERSC Annual Report 2004

Description: The National Energy Research Scientific Computing Center (NERSC) is the premier computational resource for scientific research funded by the DOE Office of Science. The Annual Report includes summaries of recent significant and representative computational science projects conducted on NERSC systems as well as information about NERSC's current and planned systems and services.
Date: April 15, 2005
Creator: Hules, John; Bashor, Jon; Yarris, Lynn; McCullough, Julie; Preuss, Paul & Bethel, Wes
Partner: UNT Libraries Government Documents Department

Recent Advances in VisIt: AMR Streamlines and Query-Driven Visualization

Description: Adaptive Mesh Refinement (AMR) is a highly effective method for simulations spanning a large range of spatiotemporal scales such as those encountered in astrophysical simulations. Combining research in novel AMR visualization algorithms and basic infrastructure work, the Department of Energy's (DOEs) Science Discovery through Advanced Computing (SciDAC) Visualization and Analytics Center for Enabling Technologies (VACET) has extended VisIt, an open source visualization tool that can handle AMR data without converting it to alternate representations. This paper focuses on two recent advances in the development of VisIt. First, we have developed streamline computation methods that properly handle multi-domain data sets and utilize effectively multiple processors on parallel machines. Furthermore, we are working on streamline calculation methods that consider an AMR hierarchy and detect transitions from a lower resolution patch into a finer patch and improve interpolation at level boundaries. Second, we focus on visualization of large-scale particle data sets. By integrating the DOE Scientific Data Management (SDM) Center's FastBit indexing technology into VisIt, we are able to reduce particle counts effectively by thresholding and by loading only those particles from disk that satisfy the thresholding criteria. Furthermore, using FastBit it becomes possible to compute parallel coordinate views efficiently, thus facilitating interactive data exploration of massive particle data sets.
Date: November 12, 2009
Creator: Weber, Gunther; Ahern, Sean; Bethel, Wes; Borovikov, Sergey; Childs, Hank; Deines, Eduard et al.
Partner: UNT Libraries Government Documents Department

NERSC Strategic Implementation Plan 2002-2006

Description: This strategic proposal presents NERSC's vision for its activities and new directions over the next five years. NERSC's continuing commitment to providing high-end systems and comprehensive scientific support for its users will be enhanced, and these activities will be augmented by two new strategic thrusts: support for Scientific Challenge Teams and deployment of a Unified Science Environment. The proposal is in two volumes, the Strategic Plan and the Implementation Plan.
Date: September 1, 2002
Creator: Kramer, William; Bethel, Wes; Craw, James; Draney, Brent; Fortney, William; Gorda, Brend et al.
Partner: UNT Libraries Government Documents Department

Laser Plasma Particle Accelerators: Large Fields for Smaller Facility Sources

Description: Compared to conventional particle accelerators, plasmas can sustain accelerating fields that are thousands of times higher. To exploit this ability, massively parallel SciDAC particle simulations provide physical insight into the development of next-generation accelerators that use laser-driven plasma waves. These plasma-based accelerators offer a path to more compact, ultra-fast particle and radiation sources for probing the subatomic world, for studying new materials and new technologies, and for medical applications.
Date: March 20, 2009
Creator: Geddes, Cameron G.R.; Cormier-Michel, Estelle; Esarey, Eric H.; Schroeder, Carl B.; Vay, Jean-Luc; Leemans, Wim P. et al.
Partner: UNT Libraries Government Documents Department

Performance Modeling for 3D Visualization in a Heterogeneous Computing Environment

Description: The visualization of large, remotely located data sets necessitates the development of a distributed computing pipeline in order to reduce the data, in stages, to a manageable size. The required baseline infrastructure for launching such a distributed pipeline is becoming available, but few services support even marginally optimal resource selection and partitioning of the data analysis workflow. We explore a methodology for building a model of overall application performance using a composition of the analytic models of individual components that comprise the pipeline. The analytic models are shown to be accurate on a testbed of distributed heterogeneous systems. The prediction methodology will form the foundation of a more robust resource management service for future Grid-based visualization applications.
Date: June 30, 2004
Creator: Bowman, Ian; Shalf, John; Ma, Kwan-Liu & Bethel, Wes
Partner: UNT Libraries Government Documents Department

ProteinShop: A tool for interactive protein manipulation and steering

Description: We describe ProteinShop, a new visualization tool that streamlines and simplifies the process of determining optimal protein folds. ProteinShop may be used at different stages of a protein structure prediction process. First, it can create protein configurations containing secondary structures specified by the user. Second, it can interactively manipulate protein fragments to achieve desired folds by adjusting the dihedral angles of selected coil regions using an Inverse Kinematics method. Last, it serves as a visual framework to monitor and steer a protein structure prediction process that may be running on a remote machine. ProteinShop was used to create initial configurations for a protein structure prediction method developed by a team that competed in CASP5. ProteinShop's use accelerated the process of generating initial configurations, reducing the time required from days to hours. This paper describes the structure of ProteinShop and discusses its main features.
Date: May 25, 2004
Creator: Crivelli, Silvia; Kreylos, Oliver; Max, Nelson; Hamann, Bernd & Bethel, Wes
Partner: UNT Libraries Government Documents Department