EU EIP sub-activity 4.1 is dedicated to frameworks and guidelines to describe and assess the quality for ITS data and ITS services in Europe. Besides several Quality Packages, being reported frequently on this website, a dedicated task was to explore optimisation aspects in terms of quality, from the perspective of European road authorities.
The goals of this co-called “Optimum Quality” task were:
- to analyse and assess the optimum quality levels for selected ITS services,
- to collect experiences of different quality improvement actions,
- to give recommendations to road authorities on options to improve quality of the different quality criteria,
- to analyse the whole value chain in the wider context, and
- to recommend action and best practices how to maximise societal benefits of the services.
An introduction to these topics was already published in 2019:
https://eip.its-platform.eu/highlights/introduction-optimum-quality-its
In the meantime, the working group has elaborated the items above via an iterative process, looking at theoretical and practical approaches on optimizing quality.
The elaborations are compiled in a recent report, which can be downloaded here.
To crystallize the main take-aways for European road operators, the working group defined the following logical steps to work towards optimum information quality and maximised societal benefits. The recommendations were created in an internal workshop on 22nd October 2020.
1. Know your data quality
Use the EUEIP Quality Package to design the local quality framework and identify the most feasible quality assessment methodologies. Reach for the European community on advice on best practises, since some quality criteria may be harder to measure.
2. Set quality targets
Consider the current quality level and the recommended optimum/target quality levels and consider the different needs in various operating environments when setting the quality targets. The EU EIP Quality Framework provides a validated and agreed baseline for organisations for target setting. Remember, that the EU EIP quality criteria and requirements may serve as a basis for service agreements regarding different companies developing systems for the road operators. Also, knowing the quality requirements and targets on the service side is a good starting point for defining the quality targets in the first part of the value chain, i.e. the content side.
3. Assess the expected quality improvement from the ongoing development projects
Analyse how the technological development (monitoring systems, vehicle data, development of IT systems and automation) will affect the data quality of different information services. This may prove a difficult task since there is a lack of evidence and knowledge to what extent different actions will affect quality locally. Nonetheless, an assessment should be made about which quality criteria will be affected (improved) by the ongoing projects and roughly how much, and which criteria will most likely remain untouched, and need further attention. Many organisations have ongoing investment projects that will lead to “jumps” in the quality levels. From the viewpoint of quality management, it would be beneficial to build in a practise of pre-assessment of potential quality effects at least to the major investment programs and plans.
4. Define and implement the needed complementary actions to reach targets
Identify the complementary actions needed to reach the targeted quality levels using the recommendations in this report. Target especially those of the critical quality criteria that seem to be lacking most behind the targets. Prepare a pre-assessment of the expected quality improvement to overcome the internal hindrances for funding and human resources. Utilise the published evaluation reports from the European community in the pre-assessment.
5. Engage in the collaboration and regulation work with the local and global value chain stakeholders
Facilitate active and continuous collaboration with the global and local service providers and agree about actions that streamline the information flow in the whole value chain. The wide-spread use of European NAPs, based on the ITS Directive, will cater for cross-border collaboration between parties. Participate in the TISA collaboration forums. As authorities have limited resources, regulation is seen as a secondary means whereas ecosystem building and voluntary collaboration that benefit all parties as the first priority. The revision of the ITS Directive may bring new regulations that will further institutionalise the EU EIP Quality framework and also bring more funding for quality assessment and related research.
6. Evaluate quality improvements
Build a local quality monitoring system for continuous quality measurement and use a wide range of incremental quality improvement actions. Evaluate systematically the quality improvements achieved by the projects and actions. Evaluate both the content side (data quality) and service side (may be covered by user satisfaction survey) and compare the findings to the users’ needs. Document the used evaluation methods and datasets. Share knowledge to the European community to strengthen learning effect on the effectiveness of different actions.
The Quality Frameworks developed by the EU EIP project are validated and tested, and therefore mature enough to support all quality improvement activities within the European road operators organisations. A slightly discouraging observations is, however, that during the many years of developing the framework, rather few organisations have built their own systematical quality management schemes and many are still struggling to fill in the action no 1 (Know your data quality). Some quality criteria have been seen as too hard to measure, but the main reason for the non-action may be more of institutional in nature. Quality of information services may be lacking from the strategic KPIs of the organisations, and therefore quality management receives insufficient funding independent of other development projects.
Institutional challenges are rarely easy to overcome. If the quality effect of investments or other actions is not evaluated systematically, the European community cannot develop new knowledge of the effects and benefits, and the situation remains. How to incorporate the quality aspects to the strategic management and investment processes? Based on the analyses and discussions in the working group, the European Commission should either consider clear incentives for road operators to implement the Quality Framework on a policy level, or consider stronger regulation to support deployment in the revision of the ITS Directive.