November QARTOD-III
Wednesday, 2, 2005, 8:30 AM NOON
John Orcutt
Scripps Welcome/ORION SCCOOS Welcome Ocean.US status Eric Terrill
Good time to discuss QAQC 38% of buoy data to NDBC now is from IOOS partners QAQC is an issue for NDBC SCCOOS Overview (Built off of CalCOFI program, glider program, automated shore stations, COCMP HF RADAR Coastal Current Monitoring Program) What is the relationship between QARTOD, IOOS & DMAC? Julie Bosch DMAC Steering Team (ch Lee Dantzler) 20-25 team members with broad backgrounds Expert Teams and Caucaus's (metadata, transport archive, standards process, private sector, international, modeling systems eng and enterprise architecture) Review the draft work plans Breakout sessions focused on developing plans emphasizing Relevance to QARTOD NOAA/NAVY Interop Demo (2 projects with Boeing and Northrup-Gruman) End-to-end demos (RT data discover, access, transport and integration) In NOAA/NAVY demos (key findings were (1) and (2) lack of community-based QA standards) IOOS must foster adoption of community standards that encompass QA so looking to QARTOD and other activities. DMAC recognizes and supports the efforts of QARTOD During QARTOD consider the following elements metadata, data discovery, transport, online browse and visualization, and archival. Workshop Goal Bill Burnett
NDBC is receiving more data. Lucille Ball and Vivian Vance more chocolate on the conveyor belt New types of data (gliders, drifters, ADCP) In FY05, 400 all stations processed, 38% from IOOS partners, 20% NWLON, 49% NDBC Mission Overview IOOS (QARTOD and DMAC) QARTOD spans observing system, data management, data analysis and modeling in all aspects of DMAC spiral. I. Define QC standards, quality control flags, highlight metadata issues II. Refine QC flags, metadata, and data dissemination of insitu currents, waves, and HF Radar III. Develop IOOS approved QC flags metadata standards and data dissemination for temperature and currents
Salinity Workshop -- Develop IOOS approved QC flags, metadata standards and data dissemination Waves Workshop -- Develop IOOS approved QC flags, metadata standards and data dissemination Today's goals: Address challenges related to the distribution and description of RT ocean data Fast and accurate assessment of the quality of the data streaming from the IOOS partner systems Operational data aggregation and assembly from distributed data sources Trustworthy and consistent quality descriptions for every observation distributed Develop quality descriptions for parameters such as waves, currents, wind, sea surface temperature and salinity. Hurricane Katrina Aftermath NDBC buoy Sliver Spring, MD GTS Data never stopped flowing from NDBC buoys At Stennis, no communication so no internet to data in Silver Spring, so no website Also data providers not getting data in because FTP was out since at Stennis No QA with no humans looking at the data during and in aftermath Need to talk about webform duplication to keep website duplicate Need to talk about having duplicate FTP site for partners Single point of data flow from buoy to Silver Spring Mark Brushnell mentioned that there is a similar problem at NOS with only one satellite communication downlink through Wallops Island. What metadata has been developed within the IOOS community and how does this affect the QC? Julie Bosch Definition of metadata: information is needed to identify, assess, use, access, exchange, transport, archive data. MMI www.marinemetadata.org (resources of content and standards, vocabs and ontologies, transport protocol, guides, examples August 1005 Workshop for Advancing Domain OWL files generated for 40 existing vocabs Break out groups (waves, CHL, CTD, Units, Sensors and Instruments and Benthic Habitat Classification) Demostration of interoperable data access using metadata and a vocab COTS/ONR Project Metadata Working Group (develop a work plan to develop metadata to assist in enabling the IOOS Interop Demo 2, plan focused on determining controlled vocabs and metadata guidance for winds, SST, waves, surface currents and CHL, metadata work plan segment. Past workshops and metadata QARTOD-I (metadata requirement, understanding that we need well defined flags, specific test used for QA/QC be defined, pros and cons of existing standards discussed)
QARTOD-II Discipline specific metadata defined for waves, insitu current, remote currents Post QARTOD-II progress on waves, NDBC and CDIP FGDC wave metadata generation. Additional work to be done on incorporating the How and What of QC into metadata. Salinity Workshop August 2005 QC for RT salinity measurements, drafted metadata record example of salinity data attributes Waves Workshop Nov 2005 significant advancements on QC requirements and recommendations, identification of details that need to be captured in metadata.
NOAA/NAVY IOOS interop demo data discovery/access lessons learned determined the best reasonable sources of data for the demo was a time consuming process and required extensive domain expertise and contacts with individual Recommendations: Standards must be established that will define a minimum set of common info FGDC compliance is required but present standards do no address most IOOS data to allow functional discovery and access. IOOS DMAC Metadata Expert Team (status, plans, ) GOALS: Coordinate with related organizations and activities in order to determine: Content requirement, appropriate standard(s), implementation of the standards(s), metadata exchange, data discovery tools. Make recommendations to Steering Committee. Provide forum with info including metadata examples, templates, and tools for IOOS participation. ACTIVITIES: research metadata standards being used for coastal and ocean obs, engage data collectors, users, and modelers to identify the information needed to discover, use, understand, access, transport, exchange, and integrate obs data, develop templates and profiles. Examples of basic metadata for CTD, HF Radar, wave data are available During QARTOD Identify and note specific QC details that need to be captured as part of the metadata. Why are ontologies important? Luis Bermudez
Ontology Philosophy (wikipedia) Specific purposes and practical differences Example, Google/Yahoo Categories are practical differences In computer sciences specification of conceptualizations (lake, river) identify properties, define similarities like has water, create relations, uses logic to make inferences. Why ontologies? Share common understanding Reuse domain knowledge Make domain assumptions explicit In QARTOD want to share common info on quality Levels, flags, sensors, instrument methodology, calib procedures, QC software, verification .... MMI and Ontologies
BODC TEMP GCMD Sea Surface Temperature GCMD Ocean Temperature CF sea_water_temperature MMI is solving semantic issues with directly created relations or inferred relations Harmonization of vocabularies OWL (Web Ontology Language) good for expressing classes and subclasses W3C recommendation 01/2004 Based on RDF Inference capabilities Restriction of inherit properties Can be used to express specifications and vocabs Domain Ontologies Repository (XML) on MMI web page (including SEACOOS) VOC2OWL (ascii terms) VINE (vocab integration environment) sameAs:owl Tethys Web Services MMI has a community role Data Source, Data Provider, Data User need community guides (MMI provides this community. With 200 members as of Oct 2005. Please join and contribute. It is your community.) Tethys: Implement two methods and make them available using SOAP web services, convert the parameters, sources, and units used in their system to an ontology (tool VOC2OWL ascii to OWL), map the terms used in the system to the MMI preferred ontology: standard vocab for discovery (GCMD) and for usage (CF). Best Practices Workshop on Salinity Jim Boyd
Concept grew from QARTOD-II, focused specifically on one IOOS core variable, data quality, metadata, and transport. Small group with dissemination back to wider audience, process as important as immediate results, should be a "portable" process for other to implement Process: small group, 3 topics, 1 room. Resulted in some confusion and too much "educating" across topical areas. Suggest: stick with small group, specifically defined outcome, breakout rooms for each topical area, reconvene in plenary to share. Document is being put together. Results: Data quality recommended criteria, metadata recommended elements, transport agreed to develop a light SOAP XML webservice Next Steps: Share results with IOOS community, "submit" to DMAC, ;push to DMAC to develop a plan for comment, feedback, distribution, recommendation, etc. and improve the workshop process, choose another variable and to it again!
IOOS Interoperability Demo Status
Jim Boyd
Demostrates community cooperation, advances the goal of data interoperability, provides demo products to showcase the value of IOOS. 1st demo sst and wind 2nd demo winds, waves, water, river stage satellite imagery for hurricane season Interop Demo II national, regional uses with theme for hazards Future of OpenIOOS--add more regional demos, add more data and data providers to the national map, link testbed activities to standards develop, external review.
QARTOD-III
Wednesday, November 2, 2005, 1:30-5:00 PM,
Norman Hall
GTSPP Global Temperature Salinity Profile Program
Global ocean T-S data of the highest quality as possible. GTSPP makes these data quickly and accessible to users. Lots of info on flags used by GTSPP http://www.nodc.noaa.gov/GTSPP/gtspp-home.html http://www.nodc.noaa.gov/GTSPP/document/codetbls/gtsppcode.htm What is the difference between Quality Assurance and Quality Control? Bushnell QA is what you do before the instrument goes out QC is what is done after data are received. Focus on previous QARTOD workshops has been to steer away from QA and focus on QC and QC flages Review of QARTOD I and II flags Ports Uniform Flat File Format (PUFFF) NOAA/NOS PORTS data QA/QC flags http://140.90.121.76/publications/pufff4.pdf What existing IOOS QC practices and methods exist already for: Remote Currents Jack Harlan NOAA IOOS NOS in partnership with NWS and NDBC National HF Radar Data Server Developing QA/QC Methods Developing Algorithmic Standards Monty Python's "The Holy Graile" HFR holy graile is to help verify and test ocean circulation model nowcasts and forecasts If no QA/QC, congress will catapult the bunny (IOOS HF RADAR) off the tower FY05-06 IOOS Level 1 Errors: Influence radial velocity (spectral freq, SNR, Geophysical contribution, eg spatial variability) Mark
Level 2 Errors: Influence total velocity (geometry of site, number and variability of radials w/in total velocity) Goal is to have standard algo's for NOAA operations. Collaborate with academic HFR users Implement results of QARTOD and MMI Collaborate with vendors Fund new studies More regimes, more RADAR types (e.g. long range) QA/QC for HFR derived wave heights Product development (trajectory analysis, others) What existing IOOS QC practices and methods exist already for: In-situ Currents Bill Burnett None. QARTOD-II Review Current meters, ADCPs, and drifters measure insitu currents, but QII concentrated on just ADCPs. See QII report. Flags were agreed upon. List of metadata descriptors to be included but not fleshed out. Tomorrow we will work on coming up with exact standards for metadata descriptors. Define thresholds for specific tests. What existing IOOS QC practices and methods exist already for: CTDs Mary Johnson Richard Bouchard: Build on Salinity Workshop Sea temperature was discussed in Q-I but not as specific as other parameters Mary Johnson CTD Data Quality Control at STS/ODF (Shipboard Technical Support/) Sea Bird (SBE) WOCE Hydrographic Program prescribed what standards to apply when reporting CTD measurements. Documented problems that have occurred in many deployments. One example is mounting CTD horizontally if calibration was done in vertical mount configuration. There will be a pressure offset. Look for shifts in bottle salinities as well as CTD salinities. What existing IOOS QC practices and methods exist already for: Waves Chung-Chu Teng NDBC Wave Program 96 buoys reporting directional waves and 4 C-MAN stations QA/QC problems with waves. See problem with dominant wave period as seen on one NDBC buoy during Hurricane Katrina. Plot wave period with wind data. After sensor evaluations, individual sensor tests, payload tests, blue tests, burn-in, deployment tests, after quality assurance.
Users don't see flags, withhold data. Flags and data confuse some users 100%. Have storm limits which have wider range or shifted range. Yesterday's Wave Workshop Timeseries Tests Data gaps Spikes Range test (min/max) Mean shift (segments) Acceleration test a<1/2 gravity Percent points good Recommended (flat episodes, equal peaks, mean crossing ... things to revisit) Spectral Tests Freq range Incident swell direction Check factor or ratio Recommended (freq distributions, range check, white noise check) Breakout tomorrow can review these tests and recommendations What existing IOOS QC practices and methods exist already for: Chlorophyll Melissa Carter RT -- Satellite imagery Non-RT -- in situ fluorometry, discrete samples How is fluorescence data converted into Chl-A? How is satellite data ground truthed? What is needed for RT chlorophyll data? Not enough time and expertise (?) here to address this parameter QA/QC at this workshop. Need to start discussion and group.
QARTOD-III
Thursday, November 3, 2005, 8:30 AM NOON
Jack Harlan Kenneth (Kip) Laws
Remote Currents (HF RADAR) Breakout Error Characterization in HF Radar Ocean Surface Currents
Project goals Provide users w/ pt-by-pt error estimates Evaluate error indicators (CODAR error flag, spectral width, indicating current availability, s2n ratio)
Characterize the effects of error sources (current availability, point targets, electronic noise and interference, antenna pattern distortions.
Methods Evaluation of meas. Errors (radial currents) using existing radar sites and large volume of existing data sets (baseline comparison, insitu comparison) Examine correlation with error indicators and sources (CODAR error flag, avg. and individual spectral point SNR, spectral width and spectral position) Evaluation of current vectors due to geometric dilution of precision (DOP) including analysis of various radial component combination schems Assess the performance of least squares vector mapping techniques that use variable radii within area of coverage Baseline vs ADCP Comparison Compare radials same effective depth (~1m), and area (~10 km^2), both error contributions unknown Compare radials from HFR to || component of ADCP vectors, depth is10m vs 1m and different areas point vs 10 km^2, single unknown error contribution Preliminary results Analysis of CODAR's statistically-based error flag Standard CODAR processing error flag values sum error flags in quadrature compare error estimates to evaluate skill of error indicator Standard processing CODAR radial current measurements estimate errors (baseline comparison) measured error estimate compare error estimates to evaluate skill of error indicator What is the error flag? How is it derived? RMS of boxcar avg. ?? Discussion of what goes into CODAR error flag Establish CODAR best practice setup for error flags, How configuration of processing (MUSIC) affects CODAR output flags Highest correlation near midpt found at small angle from baseline Monterey Bay SCRZ and NPGS Compare spatial and temporal matchup of set of selected cell (r^2 = 0.75) radial currents (cm/s) from SCRZ and NPGS 1st approach: correlate CODAR error flag with baseline current comparison (shotgun pattern!) current diff vs error flag bin data according to error flag value RMS ~ STD std dev of baseline diff correlates to error flag bin low mean low error flag larger bias larger error flag values
can we separate CODAR flag from radial velocity? If CODAR flag is larger, then speeds will higher CODAR flag is more like an uncertainty number Conclusions Baseline comparisons eliminate problems of spataial extent and depth mismatches inherent in ADCP/HFR comparisons Prelim results using baseline comparsions demonstrate high skill of CODAR error flag in predicting uncertainty [which should be investigated more based on discussion here] Josh has knowledge of what the CODAR statistical practices NOAA funded QA/QC (from IOOS) Brian Emery Review literature Assess SNR vs rms differences (codar output and other measures of SNR, rms diff;s with in situ data) Use rms diffs to eval radial combining techniques (least squares) and variable radii for deciding which radials to combine GPS-Cellular Drifter Technology of Coastal Carter ??
Designed drifters in upper 1m (follow water, not the wind) known slip <2 cm/s) What is communication Japanese system 50 km range near high-population centers (good for short burst digital data) Will try a new Iridium devise coming out soon (I didn't know this) Onshore and alongshore surface currents within HFR cell. Many drifters can see these whereas single point measurements (ADCP or CM) might miss. Comparison of drifter and HFR velocities For most of days and times (Jul 10, Jul 19, Aug 12, Aug 24) being compared HFR and drifters compare nicely. However, on Jan 11 bias in HFR relative to drifters. Suspicious that antennae pattern different in six months so HFR is not looking at same "patch", need to see when pattern was measured. Also environmental changes, dry ground and wet ground can change calibration. HFR 1 hour, 1-2 km grid, 10 cm/s accuracy More drifters more accurate More drifters improve the variance Need to think about how currents move on small scales Need to compare sub-grid measurements within HFR cell Future Try comparison with radial vectors. This was done with totals. Correlation coeff of currents (U,V) to neighboring points Eric Terrill Spatial variability of the cov implies we can not apply a universal model. Comparison with chl and sst satellite maps make good comparison if small scale features are real. Not a quanitative but visual check.
Time series of radial currents magnitude using measured beam patterns How well do we measure currents close to coast? Pier-mounted ADCP comparison with HFR with known issues of Simulation of MUSIC with model currents SBRI Report and MATLAB version of MUSIC program Tony de Paolo Worst cov where currents are small MUSIC algo simulation gives currents over land CODAR erases overland (thank god) radials Large error (>80 cm/s) popped up Large range set in CODAR default is 150 cm/s We can tweek this by changing this range input under different conditions SNR CODAR does a wedge shaped boxcar avg and sweeps around radial pattern MUSIC still gets low RMS values or holes after smoothing Normalized error is bad where currents are low. Need better control of what MUSIC is doing to data SNR computed Skill metric and sample SNR by cell Skill falls off by range cell Time variability to this and affect of sea-state Averaging over time improves skill (if currents not changing) 10 min worst, but 1 hour sufficient, 4 hours best Eigenvalue ratio Signal power ratio Off diagonal ratio Need to be careful of noise and tracking noise in system (processing?) Group Discussion Need to get a handle on distinguishing various sources of error Environmental conditions affecting signal propagation and scattering Beam patterns Wave field Rainfall Soil moisture Instrument configuration and processing
MUSIC Frequency dependence? Electrical noise Statistical error from actual spatial and temporal variability in ocean surface velocities Sub-grid scale horizontal shear Documentation of radial error flag in white papers on CODAR website CODAR to support a new data format of MUSIC sesttings, SNR and eigenvalues to be provided from MUSIC Review of QARTOD-II QUESTION 1: What real-time quality control tests should be applied? Discussion and notes inline from QII report Level 1 Establish signal to noise ratio Develop criteria to define 1st order peak (for direction finding systems) Keep these. Identify single and dual angle solutions (Applies to direction finding radars only; is not applicable to phased array systems)
What are eigenvalues from MUSIC? . This would be nice to know. An area for more research. MUSIC decides whether to use single or dual angle solutions. Can set a parameter used by MUSIC that would force MUSIC to only choose one or the other Establish thresholds for sector-to-sector changes in radial speed (A sector is an area over which radials are determined within a range cell.)
Agree to use some maximum threshold to be determined by getting stats from available sites. This would require an analysis of available sites. RDT file (Radial Diagnostic) from SNR channels (3 of them) reported in RDT file are from a specific range cell (Diag Range Cell). What criteria is used by MUSIC/CODAR for choosing this range cell? But the issue from earlier discussion is that SNR chosen and used MUSIC is the only cutoff for whole radial but we saw that SNR changes with range and need to be able to use SNR from each range. Useful to have from CODAR, SNR channels for each sector Provide standard deviation and number of vectors in each sector Provide percent coverage for each time interval over which radials are computed
Evaluate eigenvalues for each radial o This is a quality parameter used by the MUSIC algorithm. o Establish other QA/QC parameters for new algorithms Under research and needs more as seen today. But it would be useful to have from CODAR Establish parameters and processing methods for new equipment (e.g. new currentmeasuring radars) Evaluate vendor-supplied software outputs for providing additional QA/QC information (e.g. CODAR Ocean Sensors, Ltd. diag.save file)
Level 2 Evaluate radial misfit error root-mean-square difference from back-calculating the radials based on total vectors Incorporate geometric dilution of precision (GDOP) into processing o GDOP is easily defined if only two sites are contributing radials o Need to establish new method if three or more sites are used Site specific but Josh uses 1.5 and finds it to be a good filter for "cleaning" totals each vector, each time step. Others use slightly different values but also found it to be good yet sensitive. This is a good research topic. What GDOP works for various sights and how well does it do. Develop current speed Thresholds for total vectors o These values will vary by region o They should be defined to provide additional quality checks Keep this. Identify number of radials and sites contributing to total vectors -- it is important to know which (and how many) sites contributed radials to quantify the geometric error). For realtime error assignments this must be a dynamic filter applied to each total vector.
Discussed above somewhere Provide percent coverage at grid points over time interval for which total vectors are computed.
This is a metric and useful tool for system check, but probably not for RT. QUESTION 2: What categories of real-time quality descriptor flags should be applied? -9 = missing value 0 = quality not evaluated 1 = bad 2 = questionable/suspect 3 = good 4 = interpolated data
Issue with flag = 4. Is this a process? This is done on a per-radial basis and is a decision made by MUSIC (?). QUESTION 3: What real-time metadata descriptors should be applied? Keep all flags determined in QII Add method and software version of how totals were processed Need info on which sites used for totals Grid version (for national version or local hi-res grid) Will need to create fields and attributes for metadata Should we try this now for L2 totals? Has anyone created an FGDC compliant HF Radar record? (**** Something I can try perhaps) QUESTION 4: What real-time calibration flags should be applied? The things outlined in QII under this question are procedures that should throw soft or hard flags but these procedures Is anyone doing these checks in RT? Seems that there are too many questions and too much needed research. There is a lot of info in these procedures but how do you act upon it. Provide quidance to determine system health and guide operators to flag data as suspect on a whole but this is tricky to automate at this time. How do know if some guy has parked a trailer by the antenna? The beam pattern would be changed. What metric could be used to know this besides visiting site or video camera? What action taken? Assurance would be to flag data as suspect or bad until have time to run a new antenna measurement. More discussion on last questions from QII remote currents. I stopped taking notes ... since mostly everything was either accepted or noted as needing more work. I would like to implement qc_flags in our latest HF Radar And try our hand refining the QARTOD metadata record example.
QARTOD-III
In situ Currents
Friday, November 4, 2005, 8:30 AM NOON
Bill Burnett
Created and refined lists of parameters, metadata, and checks (pass, caution, fail) that are required and then recommended. These will be posted on qartod.org. SEACOOS data dictionary and MMI used extensively in metadata controlled vocabulary that was generated in this breakout. Remote Currents Jack Harlan
Informal talks about HFR error uncertainty analysis. From these talks/discussion, it is very apparent that we need to fund more analyses.
Mainly, worked off the QII results Interactive session on fundamental sources of uncertainty Need documentation from vender for error estimate and SNR. Consensus (nearly) on radial processing software Interpolated data flag from QII needs to be revisited (not sure what to do so TBD) some concern to have it or not have it FGDC this group needs clarification on this topic what is needed and how to present it (any examples would help) Julie Bosch has an example but she is not sure it is correct and needs reality check from HFR Group. Modified results from QII More research needed on diagnostic data from vendor (several dozen) Abandoned GDOP in terms of getting any QC check OpenDAP speed issue Meeting with ROWG and many its members at this workshop Another ROWG meeting later this year or next spring (??) CTD Jim Boyd
Methods of collection (profiling from ship, moored, profiling floats, fixed platforms, gliders, expendables) Directly measured (cond, temp, press) Derived parameters (salinity, depth, density) Other potential parameters other sensors that can be on CTDs(dissolved oxygen, optical sensors, position). No brainer test (range, climatology, gradient, spiking routines, comparison Other tests Required and recommended tests for each measured parameter Definitions (climatology, range tests, gradient tests, nearest neighbor, etc) References (woce, etc) Next Steps (need follow up meeting, identify other participants for follow up, review and cross reference salinity workshop report) Waves Julie Thomas
Make up of IOOS waves community on CDIP website here?? http://cdip.ucsd.edu/? nav=documents&sub=index&units=metric&tz=UTC&pub=public&map_stati=1,2,3&xitem=pro c&xdoc=qc_table Definitions of tests and examples from group and manufactures Timeseries values
Spectral values
Next Steps
Bill Burnett
Mark Bushnell: Another QARTOD focused on quality assurance David Castel: 2nd this notion and UC can contribute with expertise setting up buoy cal facility Julie Bosch: What is critical information for CTDs and waves? Keep it focused. What are parameters, units? Don't worry about FGDC or specific metadata formats but capture the relevant information. Stephan Howden: recommends having members of this workshop take workshop/training offered by CSC or NCDDC. It helped him focus on what was important metadata and not on specific formats. Sara: While metadata and QC flag efforts and discussions here and at other QARTOD workshops have been extremely helpful, I would like to see or challenge QARTOD community to address working algorithms and QC checks into code or pseudo-code and libraries. Jeremy: 2nd this. Focus on definitions and algos into scripts or programs Lisa ??: Experts (i.e. instrument users) know the most about QC checks need their input and some have algos coded already Bill Burnett: Vendors too and they can leverage input at the instrument level and have been very forthcoming of errors and accuracy and calibration procedures. Julie Thomas: envision adding code algos on CDIP website http://cdip.ucsd.edu/? nav=documents&sub=index&units=metric&tz=UTC&pub=public&map_stati=1,2,3&xitem=pro c&xdoc=qc_table Bill Burnette: Future QARTOD focus on: Address quality assurance practices Sharing libraries of tests and procedures (cookbooks) How to disseminate metadata (SOAP/ XML used in salinity exercise) Lee Danztler's write-in questions (DMAC) [missed most of these since read quickly] How can DMAC serve QARTOD? What existing standards (Q has reviewed these for parameters addressed) Identify gaps in standards (Q currently addressing this) What structure Accept what QARTOD recommends and make an arm of DMAC Ocean.US Sponsor these workshops QARTOD website on Ocean.US website?? Janet Fredericks from WHOI will see about hosting Q4 @ WHOI but needs to confirm possibly next May or Sept?? Since you mention metadata ... Steve Diggs
Interesting (philosophical?) conversation about building or creating metadata Top-down (Ontologies) and bottom-up (folksonomies) Folks help bring associated keywords
Experts unify and naildown the language used and solidify structural relationships so that metadata can be used for automated data discovery so there is no ambiguity Good metadata will be based on a mixture of both approaches (top-down and bottom-up). The example of the iPod is a good one. The idea that databases of music can be searched based on keywords such as R&B or jazz or artist for discovery is important, but also the ability to pull all the information about an album (artists, songs, who did the artwork) based on its CD number is based on well-established structured metadata. It is a mixture of both functions. Searchable and accessible.