Nov 16, 2018

EO Quality Assurance Workshop

Introduction:
Can more rigorous Quality Assurance (QA) and uncertainty evaluation of the EO end-to-end supply chain play a role in increasing the uptake and overall value/benefit of EO based products and services? And could this be further enhanced by the creation of some form of ‘certification’. This workshop should explore existing insufficiencies, roles and responsibilities of various actors along the chain, opportunities for improvement and a road map for implementation.

The workshop will be organized in ESRIN, April 9 & 10 2019

Estimated Article Reading Time: 4 min.

Context and Background:
The EO services sector is changing very rapidly. New data sources, cloud storage and processing, artificial intelligence, blockchain technologies are all dramatically altering the businesses working in the sector. New entrants are challenging the established players and many new services and business models are emerging. There is a clear trend to on-line services which was confirmed by the EARSC “MAEOS” survey of stakeholders and during a recent workshop held by EARSC.

The digital societal trend is to throw new products and services out there and promote them for users to pick up and exploit. Whilst this works for consumers who prefer to pay less but with uncertain quality, it is less clear that it will work in the business-to-business or business to government markets where quality has a higher value and may in the former case, if left unchecked, ultimately lead to damage for the market as a whole.

In this context, we need to consider if a QA system facilitating more rigorous assessment and more comprehensive/consistent reporting of quality metrics at all stages of the data/information production process is in practise an enabler of market growth? Not only does it remove the need for costly, often duplicated QA, at the various stages of product development it also removes a potential barrier to free and open competition by enabling informed customer choice. A European led QA system, potentially underpinned by a formal certification schema could provide competitive advantage for early adopters as they seek to address needs of discerning (risk/high value/public sector) customers and their intermediaries.

Quality assurance should demonstrate that the data are consistent with their claimed performance and contain enough information to allow them to readily assessed for their ‘fitness for purpose’. Data integrity will play a key role: at the highest level, data integrity is taken to mean “a measure of confidence in the data, arrived at by characterising and monitoring quality at specific points along the production and dissemination chain”. Data Integrity (DI) includes the analysis of:

- Quantitative ‘traceable’ error/uncertainty information contained within the data and its basis;
- Data quality – based on the documented and accessible results of routine Quality Control checks and calibration and validation activities;
- Data accuracy – based on specialised data evaluation and comparison exercises;
- Data availability – considering the length, coverage and operational scenario of each mission and the performance of data production within the ground segment;
- Data accessibility – considering data policy issues and the practical implications of accessing data;
- Data provenance – considering the origin, evolution and status (e.g. uncertainty evaluation) of the algorithms and processes applied to the data.

All of the above will contribute towards an overall view of the data’s fitness for purpose for a given application.

On data provider side, in the context of QA4EO (Quality assurance framework for earth observation, http://qa4eo.org/ developed by international consensus in the context of CEOS), previous work has been carried out by ESA with the support of National Metrology Institute (NMI like National Physical laboratory in UK). In particular, the process related to quantitative error/uncertainty estimates for core EO products was discussed in a workshop at ESA on 24/25 October 2017: Workshop on Uncertainties in Remote Sensing. Click here ESA has also initiated support activities through its FRM series of projects and is looking to build on QA initiatives developed in the context of the ESA CCI program and also pioneered in EU projects such as QA4ECV, GAIA-CLIM and FIDUCEO.

On the service side, previous work on this carried out by Hollidge Consulting Limited (IE) on behalf of ESA and EARSC has firstly focused on:

1. Management process where a dedicated scheme was introduced to streamline or prepare for full ISO9000 certification and the concept of the ‘Product Specification’ introduced for EOP Products

2. Some initial thinking around product certification including consultation with user communities on their interests. A product certification scheme has been investigated as a result.
Two workshops were held to test and consult on the proposed scheme; in April 2013 which initiated the work and a second one in November 2014. The latter was held in co-operation with the International Oil & Gas Producers Association; one of the key user sectors with an expressed interest in certification and this exchange will be further expanded in this workshop.

During these discussions, a clear need to provide “certified” quality information for the complete EO end-to-end chain has been expressed.

In a parallel activity NPL is also leading a program of work to establish a QA framework for the Copernicus climate change service.

The workshop will look to build upon this current basis, as outlined above, in conjunction with key stakeholders from the risk sensitive and high commercial value sectors, but will also encourage input from other public and private sectors as well as service providers and developers , . The main goal will be to gather the views of the various stakeholders and to establish what next steps in QA, Uncertainty evaluation and reporting and/or certification would be perceived advantageous for them (if any).

Stakeholders:

Three communities of stakeholders should be involved:
• EO service companies
• Suppliers to the EO service companies i.e. data suppliers, satellite operators
• Clients of the EO service companies i.e. users.*

Each element of the supply chain should be explored. The workshop should look at:
• What is wanted by the EO Service Companies from the suppliers of satellite data and/or lower level products
• What is expected by end users/customers
• What requirements do EO Service Companies place on themselves with respect to quality (Approaches to product output validation and the proof of product/service performance vs. claims)
• What level and nature of governance if any should be implemented
Size and Location