Non-responses un quantitative study surveys

This year the Fundació TIC Salut Social has decided to try something new. We will present the results of our annual Mapa de Tendències [Trends Map] study during the month of March – at the beginning of the first quarter of the new financial year – instead of holding it during the month of December –at the close of the year.

In light of the report, the results of which will soon be available, I would like to invite everyone to reflect on the survey-based research process with reference to an essay by Díaz de Rada (2013), which highlights the importance of such studies. Specifically, the article asks us to consider how to evaluate and manage non-responses to surveys, the implications in terms of the final results of the research itself and some contingency strategies to minimize the impact of said non-responses. Díaz de Rada presents his argument based on studies that quantify the phenomenon of “Non-responses in face-to-face surveys conducted in Spain” Díaz de Rada (2013).

A premise to take into account in ensuring a sample is valid is paying attention to concepts such as the sample size and sampling errors during the selection. In other words, the decisive factor in any approximation to reality is the trustworthiness of the data, which can be understood as the level of reliability of a sample in that it conforms to the reality under investigation, since sample error will depend upon the accuracy of our estimates. Logically, reducing possible sample error to zero is virtually impossible, however, trust in the data can help us determine the acceptable margin of error in order that the final results of the study are valid. According to Rodriguez Osuna (1991), the objective must always be to obtain the highest level of accuracy based on a balance between the sample size and the maximum acceptable margin of error. For this purpose one can consult a precision indicator understood as a mathematical formula in which we divide the sample error by the percentage of categories of a variable. The result is the Coefficient of Variation or Percent Relative Error. It is precisely this relative error regarding the data that we must never lose sight of in order to see as valid the volume of information lost in surveys carried out in relation to non-responses.

Taking these ideas as a starting point, Díaz de Rada (2013) proposes beginning with a phenomenon encountered by various authors which is “a decline in the population that responds to surveys, a situation which has been steadily on the rise, specifically since the beginning of the 21^st century” (Dillman, Smyth and Christian, 2009, as cited in Díaz de Rada, 2013, p.358). This brings us to a third concept that encompasses the previous two. It is vitally important in terms of the quality of the final results that the investigator carries out meticulous planning and design of the entire research process. Especially in terms of fieldwork, particularly when third parties collect data and the information is subsequently analysed. In this way we will have the certainty that the data obtained is valid –that it truly examines the variables we are trying to measure, which is any human or social characteristic that can be quantitatively measured (Sánchez Carrión, 1999, p.28)– and reliable according to the criteria of consistency, homogeneity and unidimensionality proposed by Morales (1988).

Although it is true that we will never fully be able to eliminate the quantity of lost information in relation to non-responses, we can put mechanisms in place to ensure quality control of the work carried out by third parties during field work. This means nothing should be left to chance. We shouldn’t find ourselves in the unfortunate position of having to improvise during a study. Which is why the initial design is so important: decide on the objective of the study, operationalize it and operate it. In other words, define the hypothesis and subject the variables to study, define the instruments and the limits of the study, and build indexed indicators to measure and transform the resulting concepts into knowledge.

Once issues with the design of the study have been resolved, we are ready to focus on the fourth key idea which coincides with the next stage of the research process: fieldwork. The phases of this part begin with the briefing and the detection of critical incidents. This is the time to work with, and if necessary train, the individuals who are to conduct the survey. It is important to pay special attention to this process as it is a chance to meet the field team and identify any strengths and weaknesses. It is also an opportunity to resolve any doubts that may arise when the survey is being conducted and to clarify how to proceed during a situation that may result in a non-response. Basically, this is where we undertake a review of all the possible scenarios which might arise, especially adverse ones, and consider how to reduce as much as possible the effects of situations such as non-response to reduce the risk of error or bias during face-to-face surveys. In this regard, it is worth noting that errors due to the bias introduced by a non-response are critical. For example, if we fail to locate certain observation units, how will we know how many times to try again? Or when to replace these sampling units to minimize the negative impact or validity of the final result? This brings us to one of the main concerns, which is how to increase the respondents’ willingness to cooperate and how to minimize the effects of non-response.

To summarise the main ideas covered in this document. Improvisation must be avoided since they reduce the credibility and quality of the final results.

According to Singleton and Stratis (2005) cited in Díaz de Rada (2013, p.376) “a non-response rate lower than 70% has a serious risk of producing biased estimates”, meaning in the preliminary design and planning stage we will have to conduct a fair and meticulous sampling, accompanied by a calculation of the maximum percentage of relative error which is acceptable. Thus minimizing any risks which might invalidate the research. Another consideration for reducing this risk is to systematically conduct a “thorough study of the sample framework (either by updating or the use of revisits” (Díaz de Rada, 2013, p.376).

For the reasons given above, the foundation would like to thank everyone for their active participation in answering the surveys related to this annual study. Without your participation the results obtained would obviously not be possible. Here at TIC Salut Social we continue to work to improve the quality of person-centred services through ICT. The Trends Map survey is one example of this work, which is why we make the results avaialbe to everyone every year.

Bibliography

• Díaz de Rada, V. (2013). La no respuesta en encuestas presenciales realizadas en España. Revista Internacional de Sociología. Volumen 71, nº 2. Pp. 357-381.

• Morales, P. (1988). Medición de actitudes en sicología y educación. Donostia: Universidad Pontificia Comillas. Pp. 456-462.

• Rodríguez Osuna, J. (1991a). Inferencia Estadística, Niveles de Precisión y Diseño Muestral. Revista Española de Investigaciones Sociológicas, nº54. Pp. 139-162.

• Sánchez Carrión, J.J. (1999). Manual de análisis estadístico de datos. Madrid: Alianza Editorial.