Effect of Formative Feedback via Interactive Concept Maps on Informal Inferential Reasoning and Conceptual Understanding of ANOVA

Abstract

This study assessed the knowledge structure of undergraduate participants related to previously determined critical concepts of Analysis of Variance (ANOVA) by using Pathfinder networks. Three domain experts’ knowledge structures regarding the same concepts were also elicited and averaged to create a referent knowledge structure. The referent knowledge structure served as a basis for formative feedback. Then, each participant’s knowledge structure was compared with the referent structure to identify common, missing, and extraneous links between the two networks. Each participant was provided with individualized written and visual, and multi-media feedback through an online Concept Mapping tool based on the principals of formative assessment and feedback in an attempt to increase their conceptual knowledge of ANOVA. The study was conducted with 67 undergraduate participants from a mid-size university in the United States. Participants completed two data collection tools related to the critical concepts of ANOVA. Later, three different types of feedback around the critical concepts were given to participants in three stages. First, each participant was given visual feedback as a result of the comparison between their own knowledge structures and the referent knowledge structure to highlight similarities and differences between the two. Then, participants were provided with individualized written and multi-media feedback to emphasize conceptual understanding behind ANOVA procedures. This procedure was followed by the re-assessment of participants’ reasoning ability related to ANOVA and knowledge structures related to critical concepts to measure the effect of the intervention. Results suggest that participants both in control and intervention groups had the same level of statistics experience and anxiety before this study indicating that randomization of participants into two different groups was successful. Moreover, women participants reported a statistically significant higher level of statistics anxiety than men, however, it seems that this small difference did not limit their ability to perform required statistical tasks. Further, findings revealed that participants’ conceptual knowledge related to critical concepts of ANOVA increased significantly after the individualized feedback. However, the increase in the conceptual understanding did not help participants to transform this knowledge into more formal understanding related to procedures underlying ANOVA. Moreover, even though, previous similar studies suggest that participants are consistent in using a single strategy for making inferential reasoning across datasets, in the present study, qualitative data analysis revealed that statistics learners demonstrate diverse patterns of inferential reasoning strategies when they were provided with different size of datasets each with varying amount of variability. As a result, findings support the use of an extended framework for describing and measuring the development of participants’ reasoning ability regarding consideration of variation in statistics education.