The objective of this second webinar was to transfer knowledge about the FESTA methodology for designing and conducting Field Operational Tests. The FESTA handbook can be downloaded from: http://fot-net.eu/Documents/festa-handbook-version-5-2014/.
Yvonne Barnard, of the Institute for Transport Studies of the University of Leeds and seconded to ERTICO, presented the FESTA “V”. The FESTA handbook establishes the process to guarantee the best quality possible during each step when designing and conducting a FOT or NDS.). The handbook went through a series of thorough revisions; the most recent one, revision 5, also includes NDS, cooperative systems etc.
Yvonne Barnard talked about performance indicators (PI), experimental procedures and sensors, and defining and arranging things to get the vehicles out on the road and collect data.. Yvonne highlighted that PIs can be both objective (e.g. messages transmitted, speed, lateral position) and subjective (e.g. acceptance, workload, willingness to pay). PIs are usually designed for comparison because often the hypotheses are also defined in a qualitative way. From definition to measures, Yvonne explained five different types of measures, namely direct measures (raw data, e.g. distance to lead vehicle), indirect measures (pre-processed data, e.g. time-to-collision), events (e.g. overtaking manoeuvre), self-reported measures (e.g. questionnaire), and situational variables (e.g. weather data).
Yvonne highlighted that identifying crash relevant events is difficult in practice as no specific recommendations or commonly agreed trigger values are available. She presented four commonly used methods for the identification, which are driver response, safety function response, driving context and driving history, recommended that a combined approach could be the most appropriate.
Yvonne presented a matrix in which PIs, measures and sensors are all related to each other and informed that there is a link in the FESTA handbook where readers can find the matrix excel sheet. PIs can also related to impact areas such as driving performance and safety (e.g. driver’s glance, speed), system performance and influence on driver behavior (e.g. false alarms, interaction system-driver), environmental aspects (e.g. exhaust emissions, fuel consumption), traffic efficiency (e.g. travel time, mean speed), acceptance and trust (e.g. perceived usefulness, rate of use).
When talking about experimental procedures, Yvonne emphasized that a number of issues need to be covered for example participants, study design, experimental environment, and that conducting a pilot study to test the evaluation process is absolutelty necessary. Sometimes controlled testing is the best method, for example for testing cooperative systems. Experimental rigour and scientific quality always need to be ensured.
Sensors are the instruments to help find out what we want to know and how to measure it. Yvonne explained different types of sensors to be used for different purposes and mentioned some specific sensors (e.g. video, vehicle-bus data, head/eye trackers, radar, map matching).
Yvonne finished her presentation by reiterating that the FESTA steps are not always that easy and need to consider iteration, resources, practicalities, ethical & legal objections, and data analysis issues. She reassures the audience that it is complicated but they can get help from the FOT implementation plan which is detailed in the FESTA handbook.
The webinar was concluded by a series of questions as listed below:
Finally, webinar participants were invited to join the next webinar that will be dedicated to ethical and legal issues and data gathering, 26 November 2014, by Helena Helena Gellerman at SAFER.
This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 610453.