1. The OPeNDAP servers tested served data from one and only one data file, database table or view at a time."[p23]

Note that the HDF storage format is a NASA standard but has only one implementation (and would qualify as pseudo-proprietary).

1. Variable names, attributes, and a form of output from the OPeNDAP servers are closely tied to those elements in the underlying data repository [I'm not sure I understand the 'form of output' part, but this lack of abstraction keeps coming back as an important issue.]
2. In order to modify the output from an OPeNDAP server, the underlying data must be altered
3. The OPeNDAP servers tested served data from one and only one data file, database table or view at a time."[p23]

1. Simple point or vector data,
2. Grid data such as remotely sensed imagery based, and
3. Model output or other hierarchically structured data

Point, vector, raster and model data all contain geographic elements and lend themselves to be managed and utilized in either a raw record (time series) centric or a geographic centric data model."[p3]

The authors used JMeter to test data server performance.[p8]

[R]"The format files are stored in the same directories as the data files they map. If a data holder wants to serve data from more than one flat file format, then all the files for each format must be segregated into a directory with their own format file. This is problematic as it necessarily impacts the directory structure and file naming conventions used by the data holder and thus violates the need for the IOOS DMAC DT to be minimally invasive. "[p8] This is not actually true. Look at our data served from test.opendap.org. There are a number of different format flat files all served from the same directory. However, the important requirement here is that IOOS DMAC DT be 'minimally invasive'.

"The DODS server required the creation of DODS Data Attribute Service[sic] (DAS) and Data Descriptor Service[sic] (DDS) files ..."[p8-9] Not correct but an understandable editorial error. The Freeform server requires neither be created but the DRDS (the relational database server) Does. As the authors point out later on, this is actually a useful feature since it provides a degree of abstraction from the storage form of the data.

The only graphical interface used by the authors was the HTML query interface. We see that this is the first point of contact for many users and it points to the importance of continually improving it.[p9]

the authors hand-built SOAP servers for data and found those to be significantly slower than the OPeNDAP CGI-based server (a factor of roughly 2 to 3).[p10]

[R?]"The DODS DAS and DDS could be exploited to facilitate parsing the different formats, but, the lack of standardization in the DODS server output makes dependable parsing the results problematic."[p10] It's hard to know if there's a requirement here or not. I think the lack of standardization refers to the variable names.

[R]"Further, as the IOOS community identifies appropriate data transport formats, a DODS server will require that the underlying data files be modified to deliver the new format. [I don't think this follows, but that's not what's important here...] Since the DMAC asserts that its data transport infrastructure should be minimally invasive, this is unacceptable. The DODS development effort should implement a mechanism to abstract the structure of the underlying data and allow the transformation of the data content into a standardized output format."[p10]

I'm not sure how to interpret the preceding two paragraphs, but figuring out just what the authors mean by 'standardized' is clearly important.

More tests using NetCDF files and CGI DAP server are compared to SOAP servers accessing the same files. The DAP server is faster, this time by a slightly greater margin, but only for the light load tests. For the heavy load tests, the SOAP servers were much faster. No reason given, just a note that the results were double checked. This is sort of interesting, but the real news for us comes later on when you can compare the CGI servers to servlet servers. It's not a simple or clear-cut comparison, but it is interesting just the same ...

[R]"As was found with the FreeForm server, the structure and format of the NetCDF server results, including variable names and data types were directly tied to the underlying data files."[p11] Clearly not a plus in the authors eyes I think their point is well taken. This is a very strong argument for the planned extension to the AIS so that variable names can be aliased so as to produce alternate views of data sets that conform to some standard.

[R]The importance of variable name aliasing is again raised on page 13 (at the top) and the authors note that there currently is no standard for names (they use the word 'format' but I take it to mean variable names given the context). This is key because it points out a hidden requirement that server be able to adapt a given data set to different (and probably changing over time) 'standards' for variable names. Just what the AIS can do for attributes now and we plan to do for variables in the near future.

[R]A second requirement that's also a little hard to see is that the AIS functionality needs to be clear and present in the server. It is OK for it to also be available as a standalone software package, but it must be present in the server if data providers are to know about, and use it.

[R]The SOAP and DRDS servlet based servers were compared. The values are close but the key thing is that: First, JMeter can compare both the CGI and servlet servers, and; Second, the servlet servers were much faster, by close to an order of magnitude. Also noted was that the servlet based servers responded well to the 'stress test' runs as well as the light load tests (which was also true of the CGI servers, but there was a bigger spread in times and that odd result with NetCDF).[p14] The requirement here is that our server switch to servlets (which we are doing, but it's nice to see independent confirmation).

"The addition of the HDF server to the base DODS CGI application was quick and straight forward"[p15] Conclusion: People will add new handlers to an existing server so making that easy is important.

[R]"The DMAC DT strategy advocates sound data management at a local level, low implementation barriers, non-invasive technologies and an architecture that includes data aggregators and archive centers."[p21]

The authors argue that having all data in a RDBMS is a benefit in the long term. This does not fit well with our experience with users who have tried using RDBMS technology. I think files are messy, but so is an undocumented database designed by someone with little or no IT experience and who has just taken a job in a lb on the other coast. I think the best we can hope for is to hide the complexities of actual storage (files, databases, et c.) behind some form of a network interface that supports zero of more data presentation standards. Those standards (conventions?) are mostly non-existent at present (COARDS --> CF being a notable exception). See page 21, the section titled "Data Storage."

[R]"OPeNDAP server architecture is unnecessarily complex for simple data and since the OPeNDAP server output characteristics are rooted in the underlying data repository, it is inappropriately invasive to the data holders IT infrastructure. Modification of the server output requires modification of the base data."[p22]

[R]"*Recommendation*: DMAC partners publish data holdings using SOAP and XML-based implementations. Schemata for each data service should be published and a collection of mutually agreed upon transport, method and content standards be promoted."[p22] My initial comment was XML yes, SOAP no. I still that's the way to go, but after looking at how the work on Server4 has progressed, I think we can support a HTTP/GET interface and a SOAP interface without much extra effort.

[R]"OPeNDAP servers violate NOAA IT Security standards, and likely other agency standards due to the preferred CGI installation process that involves software in the Web servers cgi-bin directory."[p22-23] I think that the changes in Server3.5 address this, except that the server's config file is in the CGI dir. However, that file does not have exploitable info in it.

[R]"Other deficient OPeNDAP server characteristics include:

1. Variable names, attributes, and a form of output from the OPeNDAP servers are closely tied to those elements in the underlying data repository [I'm not sure I understand the 'form of output' part, but this lack of abstraction keeps coming back as an important issue.]
2. In order to modify the output from an OPeNDAP server, the underlying data must be altered
3. The OPeNDAP servers tested served data from one and only one data file, database table or view at a time."[p23]

Review of the requirements contained in "An Exploration of Candidate Architectures for IOOS Data Sharing."

Editorial comments inside quotes are mine and are indicated using square brackets. Page references to the original document are also in square brackets. Things that I think are clearly requirements we should be aware of are denoted using '[R]'.

Definition of Web Services for the paper: Reusable software component that provide a standardized means for computer systems to request data and data processing from on another, typically using messages expressed in the eXtensible Markup Language (XML) and conveyed using the ubiquitous communications protocol of the World Wide Web, HTTP (HyperText Transfer Protocol). Also: "... a Web Service is a software system identified by a Universal Resource Identifier (URI), whose public interfaces and bindings are defined and described using XML."[p2]

These definitions point toward HTTP/SOAP/WSDL systems, but leave flexibility with regard to the actual messaging protocol. For example, systems that use HTTP/GET, XML and include a WSDL interface description would clearly fit the bill. The utility of SOAP in such systems has been called into question by a number of other sources, and it's good to see that the basic definitions of a Web Service for IOOS does not require it. Furthermore, many systems that employ HTTP/GET and XML can also support SOAP interfaces without significant extra effort.

[R] Regarding interoperability: "These services [Web pages, FTP sites and pseudo-proprietary standards like DAP] are insufficient in the IOOS context as they lack the standardization necessary to facilitate machine to machine interoperability." [p3]

I would argue that the DAP is not like a proprietary standard (itself an oxymoron) but it's important to note this perception and see what characteristics lead to it. There's a clear perception that the only implementations of DAP software, specifically servers, comes from our group. That's not correct, but understandable. This no doubt leads to the concerns about the proprietary nature of DAP software. We should make other servers (e.g., GDS) more prominent and seriously consider if the DAP will ever have the installed base large enough to have many different implementations. This clearly relates to increasing the complexity of DAP with its next revision. If we make the protocol more complex, we reduce the likelihood that other groups will implement it. Note, however, that the HDF storage format is a NASA standard but has only one implementation (and would qualify as pseudo-proprietary).

-- JamesGallagher - 06 Nov 2005