Review and mapping of methodologies for effective evaluation of future security risks to society

Share This:

The main goal of the FORCE project is to develop a decision support system enabling effective evaluation and execution of security foresight efforts. It includes three main stages – mapping of recent EU Security Foresight Projects, developing a Foresight Model and developing an Intelligent Decision Support System (IDSS).

This policy brief includes the results of reviewing and mapping of 24 recent EU Security Foresight Projects as well as some insights based on other publications related to security foresight. Efforts were made to enrich the information sources through scanning national and international foresight networks. It is intended to provide a metadata[1] that is necessary to the subsequent development of the foresight model and the decision support tool.


This Policy Brief is based on the research conducted under FORCE WP3 – Review of recent and current research related to security-oriented Foresight and Horizon-scanning activities. A full version of the results can be found in the deliverable report “D3.1 – Review and mapping of methodologies for effective evaluation of future security risks to society” – and may be downloaded from the FORCE website (www.force-europe.eu) as soon he is approved by EC.


Foresight methods

Security foresight projects generally follow a process encompassing typical elements including the application of various foresight methods. Each foresight method has its unique strengths and weaknesses. Delphi, for example, is a well-established technique that is intended to elicit experts’ judgement while avoiding a dominating influence of high-status advocates. It is a powerful technique for exploring issues objectively but it is also difficult sometimes to recruit participants for Delphi surveys. Foresight methods selection is a complex process that includes several (around 10) selection criteria.

The research done so far on how to combine foresight methods is still very basic and does not concern ‘optimal’ choice of methods. Foresight methods are still selected based on human intuition and experience. Based on the EFMN database[2] the most widely used foresight methods were literature review (54%), expert panels (50%) and scenarios (42%). The database enables the analysis of methods that are often employed together. Brainstorming, for example, is performed together mostly with future workshops, SWOT, key technologies, Delphi, environmental scanning and interviews.

Evaluating foresight processes and projects

There are several approaches for evaluating foresight processes and projects. Such evaluations usually include the criteria such as Relevance, Effectiveness, Efficiency, Appropriateness, Utility and Impact. The main indicator in foresight evaluation is the impact of foresight activities, although impact is generally difficult to assess. Evaluation of foresight projects is still a scarce phenomenon and scientific work on foresight evaluation models is still in progress.

The type of outcomes generated by security foresight projects include (among others) trends, threats and risks, scenarios, wild cards and weak signals, methods, and technologies. In the “D3.1. Review and mapping of methodologies for effective evaluation of future security risks to society” report we provide a sample of these outcomes rated by relevance. The criterion measures how well retrieved information meets the information needs of a potential user of the FORCE IDSS.

Mapping Results

Some of the security foresight projects are focused on developing new methods for the specific needs of the security sector. In the ETCETERA[3] project, for example, the main objective was to develop a methodology for identification of technology dependency risks and to recommend research agenda to deal with these risks.

Some foresight projects use a larger number of foresight methods relative to others. Projects iKNOW[4] and SANDERA[5] lead in this respect with 11 methods and 10 methods, followed by ETCETERA (8) and ETTIS[6] (7). The number of foresight methods used can be a measure of triangulation, which is the use of various methods to study the same topic in order to increase the quality of the findings.

Some projects were quite effective in the use of foresight methods. Project iKNOW, for example, was effective in generating a varied list of wild cards and weak signals using a combination of literature review, environmental scanning, expert panels, futures workshops, brainstorming, interviews, Delphi and web-based crowdsourcing.

Conclusions

Some of the security foresight projects mapped in FORCE suffer from weaknesses that can be overcome by the use of modern quantitative foresight methods as we learn from recent research. It is recommended to combine qualitative and quantitative foresight methods. It is also recommended to apply statistical methods for long term projections or early warning systems, such as time series analysis and indicator-based models. Other examples of relatively new foresight methods include agent based models[7] and models based on machine learning and big data.

The security domain is being impacted by growing complexity and uncertainties emanating from hyper connectivity and posing serious threats to society. Such situations mandate the use of novel foresight methods, such as wild cards, gaming, crowdsourcing, agent based models and the global participatory platform advocated by “Future ICT”[8].

Several examples of using novel foresight methods are mentioned within D3.1, including the Worldwide Integrated Crisis Early Warning System (W-ICEWS)[9], The Good Judgement Project[10], Risk Assessment and Horizon Scanning (RAHS)[11] and Online Foresight Platforms such as the Institute for the Future’s (IFTF) Foresight Engine and Wikistrat[12].

24 projects mapped in FORCE
Project acronym Full name Main objective Web site
ETCETERA Evaluation of critical and emerging technologies for the elaboration of a security research agenda Develop methodology to identify technology dependency risks http://www.etcetera-project.eu/
ACRIMAS Aftermath Crisis Management System-of-systems Demonstration

 

Develop aftermath crisis management demonstration roadmap http://www.acrimas.eu/
VALUESEC Cost-benefit analysis of current and future security measures in Europe Develop tool-set for cost-benefit analysis of security measures http://www.valuesec.eu/
STRAW Security technology active watch Provide tool for detecting new threats and technologies http://cordis.europa.eu/result/rcn/46098_en.html
ESPAS European strategy and policy analysis system Map major existing trends that are likely to shape the future http://europa.eu/espas/index_en.htm
Security Jam 2012 Security Jam 2012 – Brainstorming global security Identify and discuss global pressing challenges in defence and security policy http://tinyurl.com/oox28ko
iKNOW Interconnecting knowledge for the early identification of issues, events

and developments (…) shaping and shaking the future of STI in the ERA

Elucidate wild cards and weak signals likely to impact the ERA http://wiwe.iknowfutures.eu/
DESSI Decision support on security investment Provide participatory assessment process that accounts for complex societal dimensions in security investments decisions http://securitydecisions.org/
SANDERA Security and defence in the European research area Foresight future relationships between ERA and EU’s security policies to promote stronger coordination https://sandera.portals.mbs.ac.uk
STAR-TRANS Strategic Risk Assessment and Contingency Planning in Interconnected Transport Networks To strengthen the overall security management of interconnected and interdependent transport networks http://cordis.europa.eu/result/rcn/55235_en.html
EU-GRASP Changing multilateralism: The EU as a global regional

actor in security and peace

Studying the role of the EU as a global-regional actor in security and peace www.eugrasp.eu
GST5 Global Strategic Trends – Out to 2045 To describe possible futures to provide a strategic context for policy- and decision-makers across the UK Government; To inform policy-makers as they grapple with the opportunities and threats that the future could bring. www.gov.uk/government/publications/global-strategic-trends-out-to-2045
FESTOS Foresight of Evolving Security Threats Posed by Emerging Technologies To identify and assess security threats posed by the potential abuse of selected emerging technologies http://cordis.europa.eu/result/rcn/53833_en.html
Global Europe 2050 Global Europe 2050 To construct a number of scenarios on the condition of the EU in 2050 https://ec.europa.eu/research/social-sciences/pdf/global-europe-2050-report_en.pdf
SERON Security of Road Transport Network to investigate the impacts of possible manmade attacks on the transport network (tunnels and bridges) www.seron-project.eu
EURACOM EUropean Risk Assessment and COntingency planning Methodologies for interconnected energy networks Elaborate a common European methodology for risk management and contingency planning; promote a dialogue between energy and security stakeholders; support European policies for the protection of critical energy infrastructures www.eos-eu.com/?Page=euracom
PREDICT PREparing for the Domino effect in Crisis siTuations

 

To provide a comprehensive solution for dealing with cascading effects in multi-sectoral crisis situations covering aspects of critical infrastructures. www.predict-project.eu/
SESTI Scanning for Emerging Science and Technology Issues

 

To contribute to the development of an effective trans-national system for early identification of weak signals and emerging issues. http://eravisions.eu/stocktaking/36

http://cordis.europa.eu/result/rcn/54730_en.html

FOCUS Foresight Security Scenarios: Mapping Research to a Comprehensive Approach to Exogenous EU Roles To develop an effective long-term foresight and assessment tool at the EU level, that will help shape European security research to enable the EU to effectively respond to tomorrow’s challenges stemming from the globalization of risks, threats and vulnerabilities.
ETTIS European Security Trends and Threats In Society to identify, and assess in a scenarios framework future threats, needs and opportunities for societal security, to develop and test a methodological approach and model for security research priority setting, to derive research priorities, to help increase awareness of security research results, and contribute to overcoming barriers by advancing and testing a range of intelligence tools and techniques http://ettis-project.eu/
FORESEC Cooperation in the Context of Complexity:

European Security in Light of Evolving Trends, Drivers, and Threats

To facilitate, through a participatory foresight process, the emergence of a shared vision and coherent and holistic approach to current and future threats and challenges for European security. To assess whether a shared European concept of security could be identified. http://tinyurl.com/kcxqyya

[1] Metadata summarizes basic information about data, which can make finding and working with particular instances of data easier – http://whatis.techtarget.com/definition/metadata

[2] The EFMN was launched in 2004 to keep track of science and technology (S&T) foresight activities being carried out in Europe and elsewhere, and to promote exchanges of information and greater interaction among foresight experts and policy researchers. Central to this initiative is the mapping and analysis of the European foresight landscape, used to inform and guide EU policy with regard to S&T research and innovation.

[3] http://www.etcetera-project.eu/

[4] http://wiwe.iknowfutures.eu/

[5] https://sandera.portals.mbs.ac.uk

[6] http://ettis-project.eu/

[7] http://www2.econ.iastate.edu/tesfatsi/abmread.htm

[8] http://www.futurict.eu

[9] http://www.lockheedmartin.com/us/products/W-ICEWS.html

[10] http://www.goodjudgment.com/

[11] Durst, C. et al. “A Holistic Approach to Strategic Foresight: A Foresight Support System for the German Federal Armed Forces”, Technological Forecasting & Social Change (2014)

[12] http://www.iftf.org/foresightengine/

Share This:

Leave a Reply

Your email address will not be published. Required fields are marked *