Non-Verbal Elements as Primary Attentional Anchors Across Generational Cohorts: Eye-Tracking Evidence from Multimodal Communication Context
Main Article Content
Keywords
eye-tracking, multimodal communication, neuroscience, non-verbal, visual attention
Abstract
Contemporary multimodal communication environments integrate verbal with non-verbal elements within unified communicative artefacts spanning health, commercial, digital, and persuasive discourse. Despite theoretical consensus in multimodality and semiotics that non-verbal and verbal modes carry distinct processing affordances, objective neurophysiological evidence for the attentional priority of non-verbal elements, and its consistency across generational cohorts, remains limited in the literature. This study aims to investigate whether non-verbal visual elements consistently capture first visual fixation before verbal elements across multiple multimodal communication contexts, and to examine whether this attentional pattern is consistent across two generational cohorts, Generation Z and Millennials. A cross-contextual multi-study synthesis design was employed, integrating eye-tracking data from four previously published empirical studies conducted at Universitas Prima Indonesia and Universitas Muhammadiyah Prof. Dr. HAMKA. Across all four studies, non-verbal elements captured first fixation in almost all of participants, although no statistically significant inter-cohort difference was observed within one of studies. These findings provide neurophysiological evidence that non-verbal visual elements constitute universal primary attentional anchors in multimodal discourse, functionally independent of communicative context and generational cohort. The results carry direct implications for language-sensitive communication design, plurilingual health messaging, multimodal pedagogy, and the theoretical understanding of verbal–non-verbal semiotic hierarchies in multilingual and multicultural communication environments.
Downloads
Download data is not yet available.
References
Adha Akbari, D., Sumarwan, U., Hasanah, N., & Edison, R. (2023). Exploring just noticeable difference between AI endorsers and human endorsers in advertising campaigns. The Seybold Report, 18(8). https://doi.org/10.5281/zenodo.8241117
Avia, V. A., S, W. C., Zendrato, C. D. K. N., Nasution, M., & Edison, R. E. (2025). Perbedaan exposure konten pornografi terhadap atensi visual remaja. Jurnal Pendidikan Indonesia, 6(11), 4938–4951. https://doi.org/10.59141/japendi.v6i11.8782
Balaskas, S., Yfantidou, I., & Skandali, D. (2025). Eyes on prevention: An eye-tracking analysis of visual attention patterns in breast cancer screening ads. Journal of Eye Movement Research, 18(6), Article 75. https://doi.org/10.3390/jemr18060075
Blazhenkova, O., & Kozhevnikov, M. (2010). Visual-object ability: A new dimension of non-verbal intelligence. Cognition, 117(3), 276–301. https://doi.org/10.1016/j.cognition.2010.08.021
Castaldi, J. (2024). Refining concepts for empirical multimodal research: Defining semiotic modes and semiotic resources. Frontiers in Communication, 9, Article 1336325. https://doi.org/10.3389/fcomm.2024.1336325
Castro-Calvo, J., Cervigón-Carrasco, V., Ballester-Arnal, R., & Giménez-García, C. (2021). Cognitive processes related to problematic pornography use (PPU): A systematic review of experimental studies. Addictive Behaviors Reports, 13, Article 100345. https://doi.org/10.1016/j.abrep.2021.100345
Cope, B., & Kalantzis, M. (2009). “Multiliteracies”: New literacies, new learning. Pedagogies: An International Journal, 4(3), 164–195. https://doi.org/10.1080/15544800903076044
Dimock, M. (2019). Defining generational cutoffs: The end of Millennials and the rise of Gen Z. Pew Research Center.
Djafarova, E., & Bowes, T. (2021). ‘Instagram made me buy it’: Generation Z impulse purchases in fashion industry. Journal of Retailing and Consumer Services, 59, Article 102345. https://doi.org/10.1016/j.jretconser.2020.102345
Edison, R. E., Anisa, Y. H., & Fauzy, F. R. (2021). Analysis of visual attention among smoker and non-smoker by using human eye tracker of health graphic warning on tobacco package. Jurnal Kesehatan Masyarakat Andalas, 15(2), 3–7. https://doi.org/10.24893/jkma.v15i2.656
Eisenlauer, V., & Karatza, S. (2020). Multimodal literacies: Media affordances, semiotic resources and discourse communities. Journal of Visual Literacy, 39(3–4), 125–131. https://doi.org/10.1080/1051144X.2020.1826224
Gatcho, A. R. G., Manuel, J. P. G., & Sarasua, R. J. G. (2024). Eye tracking research on readers’ interactions with multimodal texts: A mini-review. Frontiers in Communication, 9, Article 1482105. https://doi.org/10.3389/fcomm.2024.1482105
Holmberg, N., Holmqvist, K., & Sandberg, H. (2015). Children’s attention to online adverts is related to low-level saliency factors and individual level of gaze control. Journal of Eye Movement Research, 8(2), Article 2. https://doi.org/10.16910/jemr.8.2.2
Kaufman, A., Klein, E. G., Koblitz, A., & Price, S. (2018). A vision for eye-tracking research in tobacco regulatory science. Tobacco Regulatory Science, 4(6), 3–7. https://doi.org/10.18001/TRS.4.6.1
Kozhevnikov, M., Kosslyn, S., & Shephard, J. (2005). Spatial versus object visualizers: A new characterization of visual cognitive style. Memory & Cognition, 33(4), 710–726. https://doi.org/10.3758/BF03195337
Lieberman, M. D. (2007). Social cognitive neuroscience: A review of core processes. Annual Review of Psychology, 58(1), 259–289. https://doi.org/10.1146/annurev.psych.58.110405.085654
Lim, F. V. (2025). Multimodality in multilingual classrooms. In The encyclopedia of applied linguistics (pp. 1–7). Wiley. https://doi.org/10.1002/9781405198431.wbeal20403
Lochbuehler, K., Tang, K. Z., Souprountchouk, V., Campetti, D., Cappella, J. N., Kozlowski, L. T., & Strasser, A. A. (2016). Using eye-tracking to examine how embedding risk corrective statements improves cigarette risk beliefs: Implications for tobacco regulatory policy. Drug and Alcohol Dependence, 164, 97–105. https://doi.org/10.1016/j.drugalcdep.2016.04.031
Machin, D. (2016). The need for a social and affordance-driven multimodal critical discourse studies. Discourse & Society, 27(3), 322–334. https://doi.org/10.1177/0957926516630903
Marcellina, T., Indra, E., & Edison, R. E. (2024). Analisis perbedaan atensi visual customer Shopee terhadap produk flash sale dengan produk normal berbasis eye-tracking. Jurnal Teknik Informasi dan Komputer (TEKINKOM), 7(1).
Maynard, O. M., Attwood, A., O’Brien, L., Brooks, S., Hedge, C., Leonards, U., & Munafò, M. R. (2014). Avoidance of cigarette pack health warnings among regular cigarette smokers. Drug and Alcohol Dependence, 136, 170–174. https://doi.org/10.1016/j.drugalcdep.2014.01.001
Pang, B., Saleme, P., Seydel, T., Kim, J., Knox, K., & Rundle-Thiele, S. (2021). The effectiveness of graphic health warnings on tobacco products: A systematic review on perceived harm and quit intentions. BMC Public Health, 21(1), Article 884. https://doi.org/10.1186/s12889-021-10810-z
Parry, E., & Urwin, P. (2011). Generational differences in work values: A review of theory and evidence. International Journal of Management Reviews, 13(1), 79–96. https://doi.org/10.1111/j.1468-2370.2010.00285.x
Peng, J.-E., Zhang, L., & Chen, Y. (2017). The mediation of multimodal affordances on willingness to communicate in the English as a foreign language classroom. TESOL Quarterly, 51(2), 302–331. https://doi.org/10.1002/tesq.298
Pieters, R., & Wedel, M. (2004). Attention capture and transfer in advertising: Brand, pictorial, and text-size effects. Journal of Marketing, 68(2), 36–50. https://doi.org/10.1509/jmkg.68.2.36.27794
Priporas, C.-V., Stylos, N., & Fotiadis, A. K. (2017). Generation Z consumers’ expectations of interactions in smart retailing: A future agenda. Computers in Human Behavior, 77, 374–381. https://doi.org/10.1016/j.chb.2017.01.058
Quintiliani, C. (2016). Reading the visual: An introduction to teaching multimodal literacy. Brock Education Journal, 25(2). https://doi.org/10.26522/brocked.v25i2.503
Rucci, M., Ahissar, E., & Burr, D. (2018). Temporal coding of visual space. Trends in Cognitive Sciences, 22(10), 883–895. https://doi.org/10.1016/j.tics.2018.07.009
Sandelowski, M., Voils, C. I., & Barroso, J. (2006). Defining and designing mixed research synthesis studies. Research in the Schools, 13(1), 29–40.
Soleh, M. B., Anisa, Y. H., Absor, N. F., Suswandari, & Edison, R. E. (2021). Differences of visual attention to memes: An eye tracking study. Advances in Social Science, Education and Humanities Research, 536, 8–12. https://doi.org/10.2991/assehr.k.210430.022
Sousa, M. J., & Rocha, Á. (2019). Skills for disruptive digital business. Journal of Business Research, 94, 257–263. https://doi.org/10.1016/j.jbusres.2017.12.051
Suhendi, E. T. (2025). Revitalizing the Indonesian language as a national identity in the globalization era: Challenges and strategies. Journal of English Language and Education, 10(1), 188–195. https://doi.org/10.31004/jele.v10i1.635
Theeuwes, J. (2010). Top–down and bottom–up control of visual selection. Acta Psychologica, 135(2), 77–99. https://doi.org/10.1016/j.actpsy.2010.02.006
Turner, A. (2015). Generation Z: Technology and social interest. The Journal of Individual Psychology, 71(2), 103–113. https://doi.org/10.1353/jip.2015.0021
Van der Cruyssen, I., Ben-Shakhar, G., Pertzov, Y., Guy, N., Cabooter, Q., Gunschera, L. J., & Verschuere, B. (2023). The validation of online webcam-based eye-tracking: The replication of the cascade effect, the novelty preference, and the visual world paradigm. Behavior Research Methods, 56(5), 4836–4849. https://doi.org/10.3758/s13428-023-02221-2
Veale, R., Hafed, Z. M., & Yoshida, M. (2017). How is visual salience computed in the brain? Insights from behaviour, neurobiology and modelling. Philosophical Transactions of the Royal Society B: Biological Sciences, 372(1714), Article 20160113. https://doi.org/10.1098/rstb.2016.0113
Wedel, M., & Pieters, R. (2008). Eye tracking for visual marketing. Foundations and Trends in Marketing, 1(4), 231–320. https://doi.org/10.1561/1700000011
Xinna, L., Hannah, S., & Lai, M. (2025). Algorithm awareness and apathy: An exploratory study of Generation Z’s media literacy. Media Education Research Journal.
Ziemba, C. M., Goris, R. L. T., Stine, G. M., Perez, R. K., Simoncelli, E. P., & Movshon, J. A. (2024). Neuronal and behavioral responses to naturalistic texture images in macaque monkeys. The Journal of Neuroscience, 44(42), Article e0349242024. https://doi.org/10.1523/JNEUROSCI.0349-24.2024
Avia, V. A., S, W. C., Zendrato, C. D. K. N., Nasution, M., & Edison, R. E. (2025). Perbedaan exposure konten pornografi terhadap atensi visual remaja. Jurnal Pendidikan Indonesia, 6(11), 4938–4951. https://doi.org/10.59141/japendi.v6i11.8782
Balaskas, S., Yfantidou, I., & Skandali, D. (2025). Eyes on prevention: An eye-tracking analysis of visual attention patterns in breast cancer screening ads. Journal of Eye Movement Research, 18(6), Article 75. https://doi.org/10.3390/jemr18060075
Blazhenkova, O., & Kozhevnikov, M. (2010). Visual-object ability: A new dimension of non-verbal intelligence. Cognition, 117(3), 276–301. https://doi.org/10.1016/j.cognition.2010.08.021
Castaldi, J. (2024). Refining concepts for empirical multimodal research: Defining semiotic modes and semiotic resources. Frontiers in Communication, 9, Article 1336325. https://doi.org/10.3389/fcomm.2024.1336325
Castro-Calvo, J., Cervigón-Carrasco, V., Ballester-Arnal, R., & Giménez-García, C. (2021). Cognitive processes related to problematic pornography use (PPU): A systematic review of experimental studies. Addictive Behaviors Reports, 13, Article 100345. https://doi.org/10.1016/j.abrep.2021.100345
Cope, B., & Kalantzis, M. (2009). “Multiliteracies”: New literacies, new learning. Pedagogies: An International Journal, 4(3), 164–195. https://doi.org/10.1080/15544800903076044
Dimock, M. (2019). Defining generational cutoffs: The end of Millennials and the rise of Gen Z. Pew Research Center.
Djafarova, E., & Bowes, T. (2021). ‘Instagram made me buy it’: Generation Z impulse purchases in fashion industry. Journal of Retailing and Consumer Services, 59, Article 102345. https://doi.org/10.1016/j.jretconser.2020.102345
Edison, R. E., Anisa, Y. H., & Fauzy, F. R. (2021). Analysis of visual attention among smoker and non-smoker by using human eye tracker of health graphic warning on tobacco package. Jurnal Kesehatan Masyarakat Andalas, 15(2), 3–7. https://doi.org/10.24893/jkma.v15i2.656
Eisenlauer, V., & Karatza, S. (2020). Multimodal literacies: Media affordances, semiotic resources and discourse communities. Journal of Visual Literacy, 39(3–4), 125–131. https://doi.org/10.1080/1051144X.2020.1826224
Gatcho, A. R. G., Manuel, J. P. G., & Sarasua, R. J. G. (2024). Eye tracking research on readers’ interactions with multimodal texts: A mini-review. Frontiers in Communication, 9, Article 1482105. https://doi.org/10.3389/fcomm.2024.1482105
Holmberg, N., Holmqvist, K., & Sandberg, H. (2015). Children’s attention to online adverts is related to low-level saliency factors and individual level of gaze control. Journal of Eye Movement Research, 8(2), Article 2. https://doi.org/10.16910/jemr.8.2.2
Kaufman, A., Klein, E. G., Koblitz, A., & Price, S. (2018). A vision for eye-tracking research in tobacco regulatory science. Tobacco Regulatory Science, 4(6), 3–7. https://doi.org/10.18001/TRS.4.6.1
Kozhevnikov, M., Kosslyn, S., & Shephard, J. (2005). Spatial versus object visualizers: A new characterization of visual cognitive style. Memory & Cognition, 33(4), 710–726. https://doi.org/10.3758/BF03195337
Lieberman, M. D. (2007). Social cognitive neuroscience: A review of core processes. Annual Review of Psychology, 58(1), 259–289. https://doi.org/10.1146/annurev.psych.58.110405.085654
Lim, F. V. (2025). Multimodality in multilingual classrooms. In The encyclopedia of applied linguistics (pp. 1–7). Wiley. https://doi.org/10.1002/9781405198431.wbeal20403
Lochbuehler, K., Tang, K. Z., Souprountchouk, V., Campetti, D., Cappella, J. N., Kozlowski, L. T., & Strasser, A. A. (2016). Using eye-tracking to examine how embedding risk corrective statements improves cigarette risk beliefs: Implications for tobacco regulatory policy. Drug and Alcohol Dependence, 164, 97–105. https://doi.org/10.1016/j.drugalcdep.2016.04.031
Machin, D. (2016). The need for a social and affordance-driven multimodal critical discourse studies. Discourse & Society, 27(3), 322–334. https://doi.org/10.1177/0957926516630903
Marcellina, T., Indra, E., & Edison, R. E. (2024). Analisis perbedaan atensi visual customer Shopee terhadap produk flash sale dengan produk normal berbasis eye-tracking. Jurnal Teknik Informasi dan Komputer (TEKINKOM), 7(1).
Maynard, O. M., Attwood, A., O’Brien, L., Brooks, S., Hedge, C., Leonards, U., & Munafò, M. R. (2014). Avoidance of cigarette pack health warnings among regular cigarette smokers. Drug and Alcohol Dependence, 136, 170–174. https://doi.org/10.1016/j.drugalcdep.2014.01.001
Pang, B., Saleme, P., Seydel, T., Kim, J., Knox, K., & Rundle-Thiele, S. (2021). The effectiveness of graphic health warnings on tobacco products: A systematic review on perceived harm and quit intentions. BMC Public Health, 21(1), Article 884. https://doi.org/10.1186/s12889-021-10810-z
Parry, E., & Urwin, P. (2011). Generational differences in work values: A review of theory and evidence. International Journal of Management Reviews, 13(1), 79–96. https://doi.org/10.1111/j.1468-2370.2010.00285.x
Peng, J.-E., Zhang, L., & Chen, Y. (2017). The mediation of multimodal affordances on willingness to communicate in the English as a foreign language classroom. TESOL Quarterly, 51(2), 302–331. https://doi.org/10.1002/tesq.298
Pieters, R., & Wedel, M. (2004). Attention capture and transfer in advertising: Brand, pictorial, and text-size effects. Journal of Marketing, 68(2), 36–50. https://doi.org/10.1509/jmkg.68.2.36.27794
Priporas, C.-V., Stylos, N., & Fotiadis, A. K. (2017). Generation Z consumers’ expectations of interactions in smart retailing: A future agenda. Computers in Human Behavior, 77, 374–381. https://doi.org/10.1016/j.chb.2017.01.058
Quintiliani, C. (2016). Reading the visual: An introduction to teaching multimodal literacy. Brock Education Journal, 25(2). https://doi.org/10.26522/brocked.v25i2.503
Rucci, M., Ahissar, E., & Burr, D. (2018). Temporal coding of visual space. Trends in Cognitive Sciences, 22(10), 883–895. https://doi.org/10.1016/j.tics.2018.07.009
Sandelowski, M., Voils, C. I., & Barroso, J. (2006). Defining and designing mixed research synthesis studies. Research in the Schools, 13(1), 29–40.
Soleh, M. B., Anisa, Y. H., Absor, N. F., Suswandari, & Edison, R. E. (2021). Differences of visual attention to memes: An eye tracking study. Advances in Social Science, Education and Humanities Research, 536, 8–12. https://doi.org/10.2991/assehr.k.210430.022
Sousa, M. J., & Rocha, Á. (2019). Skills for disruptive digital business. Journal of Business Research, 94, 257–263. https://doi.org/10.1016/j.jbusres.2017.12.051
Suhendi, E. T. (2025). Revitalizing the Indonesian language as a national identity in the globalization era: Challenges and strategies. Journal of English Language and Education, 10(1), 188–195. https://doi.org/10.31004/jele.v10i1.635
Theeuwes, J. (2010). Top–down and bottom–up control of visual selection. Acta Psychologica, 135(2), 77–99. https://doi.org/10.1016/j.actpsy.2010.02.006
Turner, A. (2015). Generation Z: Technology and social interest. The Journal of Individual Psychology, 71(2), 103–113. https://doi.org/10.1353/jip.2015.0021
Van der Cruyssen, I., Ben-Shakhar, G., Pertzov, Y., Guy, N., Cabooter, Q., Gunschera, L. J., & Verschuere, B. (2023). The validation of online webcam-based eye-tracking: The replication of the cascade effect, the novelty preference, and the visual world paradigm. Behavior Research Methods, 56(5), 4836–4849. https://doi.org/10.3758/s13428-023-02221-2
Veale, R., Hafed, Z. M., & Yoshida, M. (2017). How is visual salience computed in the brain? Insights from behaviour, neurobiology and modelling. Philosophical Transactions of the Royal Society B: Biological Sciences, 372(1714), Article 20160113. https://doi.org/10.1098/rstb.2016.0113
Wedel, M., & Pieters, R. (2008). Eye tracking for visual marketing. Foundations and Trends in Marketing, 1(4), 231–320. https://doi.org/10.1561/1700000011
Xinna, L., Hannah, S., & Lai, M. (2025). Algorithm awareness and apathy: An exploratory study of Generation Z’s media literacy. Media Education Research Journal.
Ziemba, C. M., Goris, R. L. T., Stine, G. M., Perez, R. K., Simoncelli, E. P., & Movshon, J. A. (2024). Neuronal and behavioral responses to naturalistic texture images in macaque monkeys. The Journal of Neuroscience, 44(42), Article e0349242024. https://doi.org/10.1523/JNEUROSCI.0349-24.2024
Article Sidebar
Published
Jul 29, 2026
Article Details
How to Cite
Edison, R. E., Akbari, D. A., Harnita, P. C., & Sihotang, W. Y. (2026). Non-Verbal Elements as Primary Attentional Anchors Across Generational Cohorts: Eye-Tracking Evidence from Multimodal Communication Context. Jurnal Arbitrer, 13(2), 161–174. https://doi.org/10.25077/ar.13.2.161-174.2026
Issue
Section
Articles
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.