In this project, our goal is to understand how children create stories and express them through various media, as well as how they shift narrative components across media as they transform stories from one format into another. We specifically focus on written expression as the final outcome, and aim to design tools that support children in writing stories they have created or consumed via visual media such as images or videos. This project is inherently interdisciplinary, integrating knowledge from Human-computer interaction and Multimedia studies, as well as education and psychology disciplines for a meaningful understanding of human cognition and usable and appropriate technology design.
Phase 1: Enactment-based Story Creation to Support Story Writing
In this phase of the project, we conducted studies where children used a full-body motion capture system to create, enact and record imaginative stories. They were then asked to write their stories while viewing these videos. For more details on the interface, see [link to story creation interface]. Our results demonstrated that story enactment could be a pathway to encourage children to create more complex and cohesive narratives. However, transforming the narratives from enacted into written form seemed to be too demanding for them, to the point that a significant part of the details were sometimes left out of the written narrative. The next phase of the project builds upon these observations. The figure below shows the interface and a few screenshots of children’s enactment scenes during recording and playback.
Phase 2: The Role of Stimulus Type and Prompting on Transmedia Story Writing
In phase one of the project, we used self-produced enactment videos as the writing stimulus. However, in this approach, the quality of the writing stimulus is directly affected by the interest, engagement and ability to enact a story in a pretend-play setup. For example, one child might have felt shy during the enactment, therefore not being able to take full advantage of their creative expressive capacity. In this phase of the project, we aim to eliminate this effect, and observe children’s performance with the same/comparable stimulus instead. We are comparing dynamic (video) vs. static (image/storyboard) stimuli with similar content. A secondary goal of this phase is to investigate dynamic writing prompts as additional support throughout the writing process.
PROJECT TEAM MEMBERS
- Dr. Sharon Lynn Chu (ELX Lab Director, University of Florida)
- Dr. Francis Quek (TEILab Director, Texas A&M University)
- Niloofar Zarei (Ph.D. Student, Texas A&M University)
- Nanjie ‘Jimmy’ Rao (Ph.D. Student, University of Florida)
- Zarei, N., Quek, F., Chu, S. L., & Brown, S. A.. (2020). A Comparison of Children’s Narrative Expressions In Enactment and Writing. In International Conference on Interactive Digital Storytelling. Springer, Cham. Bournemouth, UK.