source /usr/local/Standards/netcdf
NetCDF (network Common Data Form) is an interface for scientific data access and a freely-distributed software library that provides an implementation of the interface. It was developed by Glenn Davis, Russ Rew, and Steve Emmerson at the Unidata Program Center in Boulder, Colorado. The netCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data.
netCDF data is:
Via anonymous FTP from ftp.unidata.ucar.edu:pub/netcdf/netcdf.tar.Z.
This is version 2.3.2pl2.
Version 2.3.2 of netCDF has been tested on the following platforms:
CRAY Y-MP UNICOS 6.1.6 DEC Alpha OSF/1 1.2 DEC VAX VMS 5.5-2 DEC VAX Ultrix 4.3 DECstation 3000 Ultrix 4.3 HP-9000/7xx HPUX 9.0 (see note below on HPUX) IBM PS/2 MSDOS 5.0 IBM PS/2 OS/2 1.2 IBM RS-6000 AIX 3.2 (see note below on AIX) NeXT NeXTOS 3.0 SGI Iris IRIX 4.0.5F SPARCstation Solaris 2.3 SPARCstation SunOS 4.1.3The experience of outside developers indicates that netCDF is relatively easy to port to any system that has a C compiler and an XDR library (eXternal Data Representation, used for NFS and usually supplied by vendors).
Yes, the latest version of this FAQ document as well as a NetCDF User's Guide hyperdocument are available from http://www.unidata.ucar.edu/packages/netcdf.
In addition, if you have compiled Mosaic with the HDF/netCDF support, you can open a netCDF file from Mosaic and see the structure of the file, including the names of variables and the values of attributes.
NetCDF 2.3, made available in April 1993, contains bug fixes, portability enhancements, performance enhancements, and a few new capabilities.
Some new optimizations for the library result in significant speedups for accessing cross-sections involving non-contiguous data.
New capabilities include some additional interfaces that provide a more general form of hyperslab access. This supports sub-sampling along specified dimensions and a mapping between the points of the hyperslab and the memory locations of the corresponding values. In a generalized hyperslab, an index mapping vector is used to define the mapping between points in the generalized hyperslab and the memory locations of the corresponding values, so data values that are written or read need no longer be contiguous in memory.
There are also some new interfaces that can be used to write, read, and inquire about records, where a record may contain multiple variables of different types and shapes. Where before you had to access a record's worth of data using multiple calls, now you will be able to use a single call.
The ncdump utility supports several new command-line options including the ability to specify for which variables data values will be output, to provide brief annotations in the form of CDL comments to identify data values for large multidimensional variables, or to provide full annotations in the form of trailing CDL comments for every data value.
The current release also includes a prototype implementation of a C++ interface for the netCDF data access library. It provides all the functionality of the previous C interface, improves type safety by eliminating use of void* pointers, and is somewhat simpler to use than the C interface. With the C++ interface, no IDs are needed for netCDF components, there is no need to specify types when creating attributes, and less indirection is required for dealing with dimensions. However, since this is a prototype interface and implementation, it may be changed before a supported version is released.
CDF was developed at the NASA Space Science Data Center at Goddard, and is freely available. It was originally a VMS FORTRAN interface for scientific data access. Unidata reimplemented the library from scratch to use XDR for a machine-independent representation, designed the CDL text representation for netCDF data, wrote a User's Guide and made other additions including aggregate data access, single-file implementation, named dimensions, and variable-specific attributes.
NetCDF and CDF have evolved independently. CDF now supports many of the same features as netCDF (aggregate data access, XDR representation, single-file representation, variable-specific attributes), but some differences remain (netCDF doesn't support native-mode representation, CDF doesn't support named dimensions). There is no compatibility between data in CDF and netCDF form, and as yet no translation software exists to convert data in one form to data in the other form.
The National Center for Supercomputing Applications (NCSA) developed the HDF software and makes it freely available. HDF is an extensible data format for self-describing files that was developed independently of netCDF. Applications and utilities based on HDF are available that support raster-image manipulation and display and browsing through multidimensional scientific data. The HDF software includes a package of routines for accessing each HDF data type, as well as a lower-level interface for building packages to support new types. HDF supports both C and Fortran interfaces, and it has been successfully ported to a wide variety of machine architectures and operating systems. HDF emphasizes a single common format for data, on which many interfaces can be built.
NCSA has implemented software that provides a netCDF interface to HDF. With this software, it is possible to use the netCDF calling interface to place data into an HDF file. The netCDF calling interface has not changed and netCDF files stored in XDR format are readable, so existing programs and data will still be usable (although programs will need to be relinked to the new library). There is currently no support for the mixing of HDF and netCDF structures. For example, a raster image can exist in the same file as a netCDF object, but you have to use the Raster Image interface to read the image and the netCDF interface to read the netCDF object. The other HDF interfaces are currently being modified to allow multi-file access, closer integration with the netCDF interface will probably be delayed until the end of that project.
Eventually, it may be possible to integrate netCDF objects with the rest of the HDF tool suite. Such an integration will then allow tools written for netCDF and tools written for HDF to both interact intelligently with the new data files.
Unidata doesn't test or maintain a version of netCDF for Macintoshes, but see the files in the directory ftp.unidata.ucar.edu:pub/netcdf/contrib/mac/ for some contributed notes and Macintosh MPW makefiles for porting netCDF to an Apple Macintosh.
Unidata doesn't test or maintain a version of netCDF for Borland compilers, but see the files in the directory ftp.unidata.ucar.edu:pub/netcdf/contrib/borland/ for compiled libraries for MSDOS/Borland platforms, including XDR and netcdf libraries for all four memory models.
Hard-copies of some of these are available from the Unidata Program Center, P.O. Box 3000, Boulder, CO 80307-3000:
Rew, R. K., G. P. Davis, and S. Emmerson, NetCDF User's Guide, An Interface for Data Access, Version 2.3, April 1993. (Available from Unidata or in PostScript form by anonymous FTP from here). There is also a new experimental hypertext version of the User's Guide available here .
Rew, R. K. and G. P. Davis, "NetCDF: An Interface for Scientific Data Access," IEEE Computer Graphics and Applications, Vol. 10, No. 4, pp. 76-82, July 1990.
Rew, R. K. and G. P. Davis, "The Unidata netCDF: Software for Scientific Data Access," Sixth International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology, Anaheim, California, American Meteorology Society, February 1990.
Jenter, H. L. and R. P. Signell, 1992. "NetCDF: A Freely-Available Software-Solution to Data-Access Problems for Numerical Modelers". Proceedings of the American Society of Civil Engineers Conference on Estuarine and Coastal Modeling. Tampa, Florida. (Also available as a PostScript file via anonymous FTP from here).
Fulker, D. W., "Unidata Strawman for Storing Earth-Referencing Data," Seventh International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology, New Orleans, La., American Meteorology Society, January 1991.
Brown, S. A, M. Folk, G. Goucher, and R. Rew, "Software for Portable Scientific Data Management," Computers in Physics, American Institute of Physics, Vol. 7, No. 3, May/June 1993, pp. 304-308.
We have observed a problem with the optimizer in the C compiler of AIX 3.2. The default for our configure script is to specify optimization by using the "-O" compiler flag, but that causes the netCDF library to be built incorrectly under AIX 3.2, with the symptom that "make test" will fail even though there are no compile or link errors when building from source. A workaround is to build the netCDF library with CFLAGS="" (or "-g") explicitly, by setting the environment variable CFLAGS to "" or "-g" before invoking configure, or by using one of the other methods suggested in the INSTALL file to prevent the use of the "-O" flag for compilation. The specific file that is incorrectly compiled with "-O" is libsrc/string.c, so another workaround is to just compile that file without "-O". This problem will be fixed in the next release.
The current version of the HPUX operating system provides an incorrect definition of the FILENAME_MAX macro. This results in the symptom that the ncredef() function does not work properly when invoked from a directory other than the one the open netCDF file is in. A workaround is to apply the following patch to the file netcdf/libsrc/local_nc.h before building the library:
diff -c -r1.39 local_nc.h *** /tmp/T0a004uN Tue Mar 8 11:14:12 1994 --- local_nc.h Wed Feb 9 15:56:36 1994 *************** *** 12,17 **** --- 12,21 ---- #include <stddef.h> /* size_t */ #include <stdio.h> /* FILENAME_MAX */ + #if defined(_HPUX_SOURCE) && FILENAME_MAX < 255 + /* Arrgh. HP thinks FILENAME_MAX is _POSIX_NAME_MAX */ + #undef FILENAME_MAX + #endif #ifndef FILENAME_MAX #define FILENAME_MAX 255 #endif
Probably due to a write error, perhaps because of exceeding disk quotas, a full device, or permission problems. The netCDF library is usually built on a vendor-supplied XDR library layer. Sometimes errors occur in the XDR layer rather than the netCDF layer, and in this case the error messages from the XDR layer can be cryptic. For example, if the user is writing data and exceeds a disk space quota, this might be detected first in the XDR layer. Although we try to catch and elaborate on errors that occur in the XDR layer, sometimes there is not enough information passed up to calling routines from the XDR layer to make construction of a meaningful error message possible.
There are many ways to represent the same information in any general-purpose data model. Choices left up to the user in the case of netCDF include which information to represent as variables or as variable attributes; what names to choose for variables, dimensions, and attributes; what order to use for the dimensions of multidimensional variables; what variables to include in the same netCDF file; and how to use variable attributes to capture the structure and meaning of data. We provide some guidelines in the NetCDF User's Guide (e.g Section 2.3.2, ``Differences between Attributes and Variables''), but we've found that a little experience helps. Occasionally we have decided it was useful to change the structure of netCDF files after experience with how the data is used.
We have no plans to add data compression to netCDF (although we do hope to eventually add a form of transparent data packing on write and unpacking on read whenever the reserved attributes "_Nbits", "_Scale", and "_Offset" are defined).
Hyperslab access and direct access to individual array values conflict with most simple compression schemes. With netCDF, the elements of an array variable can be filled in any order or as cross-sections in any direction. NetCDF permits writing elements in one order and reading them later in different orders.
Some compression methods require that all the data to be compressed are known before starting the compression. Techniques like run-length encoding or anything that depends on exploiting similarities in nearby values can't be used if nearby values aren't all known at the time some of the data are to be written.
An alternative that can be implemented above the netCDF library is to adopt a convention for compressed data that uses a "compression" attribute to encode the method of compression, e.g.
x:compression = "rle" ;for run-length encoding of the data in a variable x. Then when you write the data, compress them into a bland array of bytes and write all the bytes. Note that it would be difficult to define the size of such a variable in advance, since its compressed size depends on its values. You would also have to give up on hyperslab access for such variables, but instead read the compressed array in all at once and uncompress it before using it.
Utilities available in the current netCDF distribution from Unidata are ncdump, for converting netCDF files to an ASCII human-readable form, and ncgen for converting from the ASCII human-readable form back to a binary netCDF file or a C or FORTRAN program for generating the netCDF file. In addition, the first release of the netcdf operators package, containing three general-purpose netCDF operators, is available via anonymous FTP from pub/netcdf/ncopers.tar.Z on ftp.unidata.ucar.edu/.
A list of software that can be used to manipulate or display netCDF data is available in two forms:
Work on additional netCDF operators has been suspended until we can get more programmer resources, due to a higher priority recently placed on development of an event-driven network data distribution system. Volunteers to implement one or more of the netCDF operators are hereby solicited. We will coordinate community efforts to avoid duplication of effort, so before you volunteer to work on one of the planned netCDF operators, please contact support@unidata.ucar.edu to find out if someone else is already working on it.
Yes - there are two. The mailing list netcdfgroup@unidata.ucar.edu has over three hundred readers and gateways; you can subscribe or unsubscribe to the mailing list by sending mail to netcdfgroup-adm@unidata.ucar.edu
If you would prefer to get only a single daily digest of the postings to the netcdfgroup mailing list, subscribe instead to the netcdfdigest mailing list by sending a request to netcdfdigest-adm@unidata.ucar.edu
All the postings to the netcdfgroup mailing list are archived in the file mail-archives/netcdfgroup available via anonymous FTP from ftp.unidata.ucar.edu.
If you find a bug, send a description to support@unidata.ucar.edu. This is also the address to use for questions or discussions about netCDF that are not appropriate for the entire netcdfgroup mailing list.
The WAIS index unidata-support-netcdf.src provides a full-text search of the support questions and answers about netCDF provided by Unidata support staff.
This index and other information about Unidata can be accessed through the Unidata gopher server at gopher.unidata.ucar.edu
The netcdfgroup and netcdfdigest mailing lists have over 300 addresses (some of which are aliases to more addresses) in fifteen countries. Several groups have adopted netCDF as a standard way to represent some forms of scientific data.
A list ofsome of the projects and groups that are using netCDF is available in two forms: