Community-developed processor helps reduce optical sensor uncertainty

A team of ocean optics experts, led by Plymouth Marine Laboratory, has evaluated the accuracy of above-water optical sensors, using a community-developed processor, to produce the highest-quality data for satellite ocean color validation and facilitate monitoring the health of our coastal seas and global ocean.

The study is published in the journal Optics Express.

Much can be learned about marine environmental conditions from the color of the ocean. Sunlight interacts with different materials in the water, such as phytoplankton, sediments and dissolved organic matter, which absorbs or scatters light photons at different wavelengths.

This absorption and scattering results in spectral variations in “color,” that can be investigated by scientists to determine what is present in the water, without necessarily having to undertake direct in-water measurements.

Earth observation scientists need to ensure that the instruments measuring the light that is reflected out of the ocean give consistent results, especially since these are used to verify that the satellite data are accurate.

To ensure that environmental measurements and the resulting processed data are of high quality and standards, many laboratories worldwide use the same recognized Quality Management Systems (ISO) to collect them.

Optical and remote sensing scientists are taking this data quality to a higher level by collecting “Fiducial Reference Measurements,” a standard developed and adopted by the scientific community and space agencies, such as the National Aeronautics and Space Administration (NASA), European Space Agency (ESA) and European Agency for the Exploitation of Meteorological Satellites (EUMETSAT).

There have been a number of studies over the last 25 years comparing the accuracy of optical sensors, and although the differences were within reasonable ranges, the discrepancies have often been larger than desired.

An important finding from these studies was that differences in how the data is processed can lead to errors that mask the true performance of the sensors, as well as highlighting the need for detailed uncertainty estimates of the measurements and a consistent data processing method. The need for a community-wide, open-source community processor was also identified.

Method and findings

The international study team of world experts in ocean optics from the EU and U.S. performed a 10-day test in the Adriatic Sea of some of the most popular scientific optical sensors (radiometers) that are commonly used in the US and Europe: Sea-Bird HyperSAS and TriOS-RAMSES. The data collected was then processed using the open-source community processor HyperCP.

The team used two different approaches for processing the data and establishing associated uncertainties:

1. Sensor-specific processing: When each instrument’s unique characteristics were used, the differences between the systems were small; about 2% variation for most optical measurements and 2.5% for the specific measurement related to water-leaving light. The error or uncertainty in these measurements stayed low at around 1.5–5%.
2. Class-based processing: When generic calibration characteristics were used for each instrument type (either Sea-Bird or TriOS), the differences between instruments were noticeably larger.

The study suggests that using HyperCP with sensor-specific characteristics is highly recommended to reduce the associated uncertainties and to produce the highest quality data for satellite ocean color validation. Tailoring the data processing in this way makes the overall measurements more precise and trustworthy for further studies and application in environmental science.

Dr. Gavin Tilstone, lead author and Bio-Optical Oceanographer at Plymouth Marine Laboratory, commented, “We are excited to share this step-change approach to optical sensor data processing for satellite ocean color validation. Before this study, we did not have confident uncertainty estimates for some of the most popular radiometers used by the ocean color optics community. The development of this community processor will significantly increase confidence in the data intended for ocean color research, crucial for climate change and water quality studies.

“Since the community processor is open-source, this means that it is widely available to countries, smaller organizations and young scientists who may not have previously had the knowledge, technical expertise and processing capability to process the optical data to these high standards.”

Dr. Tom Jordan, co-author and Earth observation scientist in ocean optics at Plymouth Marine Laboratory, comments, “An important aspect of the study was that it demonstrated consistency between uncertainties computed by HyperCP and variability of measurements by different sensors in the field. Going forward, this gives improved confidence that we can combine multiple sensors and field deployments to provide a trusted reference for satellite ocean color validation.”

A large range of user-groups and individuals are already using the enhanced community processor and early career researchers in numerous countries are being taught how to use it through specific training events.

Standardizing data around the world

For data to be accepted for use by agencies such as NASA, ESA and EUMETSAT, they need to be of “Fiducial Reference Measurement” quality, which is the highest quality, accepted standard for principal radiometric data used to validate and support satellite ocean color missions. To reach this level of quality, four criteria need to be met:

• Recognized protocols for satellite validation are followed.
• Traceable International System of Units (SI) standards are applied.
• Uncertainty estimates of the measurements are included.
• All sensors are evaluated through international field intercomparisons.

Dr. Dirk Aurin, co-author and Senior Research Scientist at the NASA Goddard Space Flight Center, summarizes the history of satellite ocean color processing standardization and how this study helps advance solutions to the issue of standardization: “The standardization of satellite ocean color processing at NASA began with SeaWiFS in the 1990s, culminating in the development of freely distributed software SeaDAS, which today is applicable to most US and international ocean color missions.

“However, the processing of in situ radiometry used to validate these orbital missions was not similarly standardized until the release of the community processor, HyperCP. HyperCP has evolved from its roots at NASA in 2019, during the latest overhaul of the protocols for satellite validation by the International Ocean Colour Coordinating Group (IOCCG), to an international collaboration including space agencies in the US and Europe, academic institutes, and the private sector.

“The field campaign highlighted in this article represents the first published application of HyperCP for the purpose of minimizing uncertainties and discrepancies across multiple, collocated in situ radiometric platforms from different manufacturers run by different labs, and demonstrates the capability for collecting field radiometry with portable, off-the-shelf commercial instruments of a quality sufficient for ocean color satellite validation following the IOCCG protocols and Fiducial Reference Measurement metrology.”

Dr. Riho Vendt, co-author and Associate Professor at the University of Tartu, continued, “Routine comparison measurements are essential for validating Earth observation data. This latest comparison introduces harmonized data processing using the open-source HyperCP community processor, allowing for alignment with widely accepted processing standards and in-depth uncertainty analysis. It is a significant step forward in the highest quality for validating satellite-based Earth observation data products.”

The next steps are to incorporate new and additional sensors with uncertainty estimates into the community processor to continue to improve confidence in ocean color satellite data.

The study was part of the Fiducial Reference Measurements for Satellite Ocean Color (FRM4SOC) Phase-2 project. The development of HyperCP was initiated by NASA’s HyperInSPACE and the University of Victoria’s PySciDON software processing packages, and is currently being conducted by an international team of expert scientists from institutions such as NASA, NPL, NOAA, EUMETSAT, PML and others.