Contributors:
Deval Panchal, Christopher Collins, Mariana Shimabukuro
What is LingoComics?
LingoComics, a web application that embeds AI-empowered stories with narrative and comic-style illustrations to enhance contextualization and personalization in language learning. At the core of LingoComics is the Story Designer module, which allows learners to co-author short narratives using a structured set of parameters within a simple user interface. Leveraging OpenAI’s GPT-4-turbo for text completion and DALLE-3 for image generation, the Story Designer generates contextually relatable stories and comic-style images based on user input.
Try LingoComics Yourself: DEMO
You can try out the demo at by visiting the LingoComics application: lingocomics.apps.science.ontariotechu.ca
Video Demonstration
Check out the pictures from during the poster presentation at UIST 2024
Publications
-
Panchal, D., Collins, C., & Shimabukuro, M. (2024). LingoComics: Co-Authoring Comic Style AI-Empowered Stories for Language Learning Immersion with Story Designer. In Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery.@inproceedings{10.1145/3672539.3686352,
author = {Panchal, Deval and Collins, Christopher and Shimabukuro, Mariana},
title = {LingoComics: Co-Authoring Comic Style AI-Empowered Stories for Language Learning Immersion with Story Designer},
year = {2024},
isbn = {9798400707186},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3672539.3686352},
doi = {10.1145/3672539.3686352},
abstract = {In language learning applications, contextualization and immersion rely on real-life communication and remain challenging despite the recent advancements in artificial intelligence (AI) that have significantly impacted educational and language learning experiences. This paper introduces LingoComics, a web application that embeds AI-empowered stories with narrative and comic-style illustrations to enhance contextualization and personalization in language learning. At the core of LingoComics is the Story Designer module, which allows learners to co-author short narratives using a structured set of parameters within a simple user interface. Leveraging OpenAI’s GPT-4-turbo for text completion and DALLE-3 for image generation, the Story Designer generates contextually relatable stories and comic-style images based on user input. Future work includes user evaluations, activity designs, and additional language learning support features. LingoComics aims to increase learners’ confidence and motivation by enabling personalized, situational language practice, preparing them for real-life communication.},
booktitle = {Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology},
articleno = {100},
numpages = {3},
keywords = {LLM application, language learning},
location = {Pittsburgh, PA, USA},
series = {UIST Adjunct ’24}
}