Validating cross-national surveys

November 2012

"The tools are relevant wherever we need to analyse multiple measurements of concepts such as attitudes, opinions and abilities - especially when we want to make comparisons between groups." (Dr Jouni Kuha, London School of Economics and Political Science)New tools developed by social scientists enable findings from different countries to be compared more reliably.

Cross-national surveys compare knowledge of and attitudes towards certain issues, such as the average levels of support for immigration, between different countries. In an increasingly open world, they are often used to inform policy and practice. However, one of the challenges of using this sort of survey data is ensuring that researchers have valid and reliable methods of comparing results from different countries.

Statistical analysis seeking to measure a person's underlying views on things such as the economy, the government and criminal justice is too complex to be captured by a single survey question; therefore a range of different questions are used to measure them. There is a need in cross-national surveys to determine if such 'measurement' of concepts is the same, given cultural and translation differences.

The ESRC-funded 'Latent Variable Modelling of Categorical Data' (LCAT) project solved this problem by developing computing tools helping researchers to test survey questions. The new tools enable faster and more convenient processing of data and comparison of such measurement across countries.

Impact

  • The project's work has influenced the development of the 'Trust in Justice' module of the European Social Survey (ESS), which examined people's attitudes to the police across Europe. LCAT improved both the testing of reliability of results, and the robustness of the survey. This led to the removal of certain questions shown not to work the same way across different countries.
  • The tools developed by LCAT were also used in two other high profile cross national surveys, the EU-funded Eurobarometer and the US National Science Foundation’s Science and Engineering Indicators study, to assess the public’s knowledge about science and technology across countries during late 2009 to 2010.
  • These tools are already planned to be used on the upcoming FIDUCIA project, which investigates 'new European' criminal behaviours that have emerged in the last decade.

Further information