with course grades?
Within a particular study, questions such as those in the left-hand column of Table 2.1 will have a definite answer. Using statistical tests, we can say that the correlation between anxiety and course grades is −0.36 (Teimouri et al., 2019), extraversion correlates with willingness to communicate (WTC) at r = 0.39 (MacIntyre & Charos, 1996) and the correlation between the Ideal L2 Self (motivation) and intended effort is 0.61 (Al-Hoorie, 2018). Across quantitative studies, different values will emerge. It is understood that correlations will vary from one study to another, as will group means and so on, but often the statistical results are similar and discrepancies accounted for by differences in sampling or methods used from one study to the next. The answers produced most often are phrased in concrete terms, using statistical values to create a sense of confidence, and when theory is added to the mix, a sense of genuine research progress can arise. Many researchers and research consumers find this approach satisfying.
The CDST toolbox produces a different approach by starting with a process-oriented account of the phenomena. Instead of finding the value of a correlation between motivation and intended effort or language course grades for example, the CDST account might describe what learners who feel motivated are thinking and feeling, what is changing in their cognitive and emotion systems, what they are doing behaviourally, what is happening with their language production and nonverbal communication, and so on. In this case, the motivation system is under scrutiny as an emergent, self-organizing state that is soft assembled from a learner’s cognition, emotions, behaviours, social context, interactions with peers, teachers, culture, other interlocutors and more. The specific details of the motivation process will differ from one person to the next – perhaps one person clearly imagines a desired future of smooth L2 communication getting closer and closer, but another ties their motivation directly to enjoying the present classroom context and the relationships therein. The research focus shifts from discovering generalizable patterns and estimating group averages to a focus on the specific perspectives of individual experiences and processes.
An additional implication of changing the phrasing of research questions is that it alters the types of analytic techniques that can be applied. Questions such as those in Table 2.1 often are answered with statistical analysis such as correlation, regression, structural equation modelling and analysis of variance, among others. Broadly speaking, there are two ways of constructing research questions - group comparisons and correlation. In group comparisons, we might ask whether differences between the means (averages) for two or more groups have arisen by chance or whether a difference between group means is statistically significant and meaningful. In studies based on correlation the objective is to describe the strength of a tendency for scores to rise and fall together; stronger correlation leads to better prediction. In both cases, variability or deviations (from the mean or from the predicted values in the case of correlations) usually cannot be explained and are used for statistical purposes as estimates of random error. Large sample sizes are valued because they provide more stable estimates of means, correlations and random error, allowing for more confidence in the results.
A dynamic focus follows different rules. Methods to answer CDST questions are not yet commonplace and there is need to further develop and disseminate methodology. In writing CDST projects for publication, we are learning that a focus on dynamics upends many of the accepted criteria for the evaluation of research. Quantitative research projects in the psychology of language learning are most often done with a cross sectional design where a researcher uses a pre-determined set of instruments to measure the concepts being studied while collecting data in as large a sample as possible. In a cross-sectional research design, each person is tested once. The kind of data relevant to a dynamic approach are different. van Dijk et al. (2011: 62) proposed three key criteria for research methods to address CDST questions:
…if we really want to know how an individual (or group) develops over time we need data that is dense (i.e. collected at many regular measurement points), longitudinal (i.e. collected over a longer period of time), and individual (i.e. for one person at a time and not averaged out).
The differences between approaches can upset the expectations of readers. One of the implications of shifting the focus so radically is that the gatekeepers in the field – the reviewers and editors of grant applications, journal articles, dissertation committees and so on – may be accustomed to applying a set of principles drawn from one methodological approach that are inappropriate in the other. For example, in a typical statistical approach, a large sample size is highly valued but in a dynamic approach, a large sample size may be overwhelming because of the density of data at the individual level.
The comparison of quantitative research methods with dynamic ones shows that the perspectives are quite different – not irreconcilable – but different. To provide a more complete picture, the next section contrasts qualitative and dynamic methods.
Qualitative compared with CDST approaches
Qualitative studies are often designed to describe the phenomena under study and to explain what is happening from the participants’ perspective, and sometimes in their voice. Compared to quantitative studies, qualitative studies usually make fewer a priori assumptions about the data and instead carefully interpret the information from interviews, focus groups, textual analysis and other multimodal sources. Qualitative investigations typically allow for the openness of the systems discussed above, are often narrative in style, tend to be closely tied to context, and are often concerned with specificity and uniqueness by examining smaller samples in depth. In terms of describing the dynamics and openness of a given situation, qualitative methods have distinct advantages and require generally less substantial adjustments than a quantitative approach from a CDST perspective.
However, qualitative data are not inherently complex or dynamic. Unless a researcher is careful to include it in the design of the study, identifying the timescale under discussion can sometimes be difficult. The type of data utilized will depend on the specific research projects and questions. Many tools can be designed to focus on stability or variability such as narratives, journals or interviews. Typically, qualitative data is based on self-report data and this implies a host of methodological concerns such as impression management, memory problems, degree of awareness about the issues under investigation and ability to articulate responses. To capture the dynamics created by interacting systems, data can be collected in an ongoing fashion or retrospectively. However, in both cases, the processes that contribute to the dynamics of change may or may not be known by the respondents who typically are asked to account for events as they understand them. Ongoing approaches to data collection are possibly the most promising for investigating dynamics, enabling a research perspective on before, during and after the process under investigation. Capturing the ongoing dynamics as close to the moment they happen allows for a high density of data collection points as well as a reduction in the problems associated with memory and distance to the actual events in action. In contrast, retrospective research risks gaps in relevant data as well as a lower degree of data density.
One of the advantages of qualitative data is that it enables a holistic perspective on phenomena and processes and can potentially reveal more of the complexity of the system than an approach which reduces or fragments variables and processes. However, there is a danger that researchers might use the conceptual framework of CDST as a meta-theory to seek to explain data without the appropriate design to support it. As CDST becomes more widely known as a research approach, there is a risk that it becomes merely a nod to the methodological fashion of the moment. We should caution against dressing a traditional study in CDST clothing because it is something new or different. A gratuitous mention of CDST is not appropriate or even relevant unless a CDST perspective has been applied throughout the design of the study, data collection and analysis process.
Research Examples of CDST