Institute of Philology of the Siberian Branch of Russian Academy of Sciences

ISSN 2307–1753 [16+]

Founder — Institute of Philology, SB RAS

Critique and Semiotics

Digital network scientific journal
for specialists in philology and semiotics


DOI: 10.25205/2307-1737 Roskomnadzor certificate number Эл № ФС 77-84784
Kritika i Semiotika (Critique and Semiotics)

По-русски

Article

Name: Multimodal Studies of Poetic Discourse: Visual, Auditory, and Synthetic Transcoding

Authors: Olga V. Sokolova

Institute of Linguistics of the Russian Academy of Sciences, Moscow, Russian Federation

Issue 2, 2024	Pages 211-229
UDK: 81'42	DOI: 10.25205/2307-1753-2024-2-211-229

Abstract:

The paper explores contemporary multimodal artistic, and especially poetic, texts from the perspectives of linguistic poetics, linguistic pragmatics, and discourse analysis. The article provides an overview of the main multimodal studies of artistic discourse. “Transcoding,” i.e. the transfer of a message from one format to another, is the way to convert the message from one format to another (mainly from analogue to digital). A classification of different types of multimodal poetic texts includes visual, auditory and auditory-visual or synthetic types of transcoding. It is based on the aesthetic-semiotic typology of art forms on the one hand, and on the transcoding mechanism on the other. The paper examines the third, synthetic, type of transcoding in detail through the case study of the Italian poet Vittorio V. Zollo’s multimodal poem ’A Via Crucis (2020).

Keywords: multimodality, transcoding, pragmatic markers, online communication, Italian language, contemporary poetry

Bibliography:

The paper explores contemporary multimodal artistic, and especially poetic, texts from the perspectives of linguistic poetics, linguistic pragmatics, and discourse analysis. The article provides an overview of the main multimodal studies Bateman J., Wildfeuer J., Hiippala . Multimodality. Foundations, Research and Analyses. A Problem-Oriented Introduction. Berlin, Boston, De Gruyter, 2017, 415 p. Benthien C., Lau J., Marxsen M. M. The Literariness of Media Art. New York, London, Routledge, 2019, 332 p. Crystal D. Language and the Internet. Cambridge, Cambridge Uni. Press, 2004, 304 p. Feshchenko V. V. Grammaturgiya Karly Kharriman: interfeisy dialogicheskoi poezii [Carla Harryman’s Grammaturgy: Interfaces of Dialogical Poetry]. New literary review, 2023, no. 3, pp. 163–172. (in Russ.) Gibbons A. Multimodality, Cognition, and Experimental Literature. London, Routledge, 2012, 288 p. Iriskhanova O. K. Polimodal'nyi diskurs kak ob"ekt issledovaniya [Multimodal discourse as an object of research]. In: Iriskhanova O. K. (ed.). Polimodal'nye izmereniya diskursa [Multimodal dimensions of discourse]. Moscow, YaSK Publ., 2021, pp. 15–32. (in Russ.) Jakobson R. O. Lingvistika i poetika [Linguistics and poetics]. In: Ba- sin E. Ya., Polyakov M. Ya. (eds.). Strukturalizm: “za” i “protiv” [Structuralism: “for” and “against”]. Moscow, Progress, 1975, pp. 193–231. (in Russ.) Jefferson G. On the Poetics of Ordinary Talk. Text and Performance Quar-terly, 1996, no. 16, pp. 11–61. Jewitt C., Bezemer J., O’Halloran K. Introducing Multimodality. New York, an London, Routledge, 2016, 232 p. Karpenko E. I. Poetika khudozhestvennogo teksta v polimodal'nom izmerenii [Poetics of literary text in a multimodal dimension]. In: Iriskhanova O. K. (ed.). Polimodal'nye izmereniya diskursa [Multimodal dimensions of discourse] Moscow, YaSK Publ, 2021, pp. 239–263. (in Russ.) Kress G., Van Leeuwen Th. Multimodal Discourse: The Modes and Media of Contemporary Communication. London, Arnold Publ., 2001, 142 p. Logan R. K. The Sixth Language: Learning a Living in the Internet Age. To-ronto, Stoddart, 2000, 318 p. Loginova E. G. Semioticheskii rezonans v estestvennoi kommunikatsii i khudozhestvennom diskurse [Semiotic resonance in natural communication and artistic discourse]. Ryazan, 2021, 284 p. (in Russ.) Machin D. Introduction to Multimodal Analysis. London, Hodder Arnold, 2007, 207 p. Manovich L. Yazyk novykh media [Language of new media]. Moscow, 2018, 400 p. (in Russ.) O’Halloran K. L. Multimodal Discourse Analysis. Systemic-Functional Per-spectives. London, New York, Continuum, 2004, 252 p. Person R. F., Wooffitt R., Rae J. P. (eds.). Bridging the Gap Between Con-versation Analysis and Poetics. Studies in Talk-In-Interaction and Literature Twenty-Five Years after Jefferson. New York, London, Routledge, 2022, 262 p. Sacks H., Schegloff E. A., Jefferson G. A Simplest Systematics for the Organization of Turn-Taking in Conversation. Language, 1974, no. 50, pp. 696–735. Scollon R. Multimodality and the Language of Politics. In: Brown K. (ed.). Encyclopedia of Language and Linguistics. Oxford, Elsevier, 2006, vol. 9, pp. 386–387. Sokolova O. V. Novye tekhnologii i pragmaticheskie tekhniki v sovremennoi poezii [New technologies and pragmatic techniques in modern poetry]. Word.Ru, 2024, no. 2, рр. 81–97. (in Russ.) Sokolova O. V. Transkodirovanie v sovremennoi poezii: vizual'naya i audial'naya perevodimost' [Transcoding in modern poetry: visual and auditory translatability]. New literary review, 2023, no. 181, pp. 173–187. (in Russ.) Toffler A. The Third Wave: The Classic Study of Tomorrow. New York, Random House Publishing Group, 1981, 537 p. Tsvigun T. V., Chernyakov A. N. Neiropoeziya grammatiki i neirogrammatika poezii [Neuropoetry of grammar and neurogrammar of poetry]. New liter-ary review, 2023, no. 181, pp. 188–199. (in Russ.) Zykova I. V. Poetika kinodiskursa v polimodal'nom izmerenii [Poetics of film discourse in a multimodal dimension]. In: Iriskhanova O. K. (ed.). Polimodal'nye izmereniya diskursa [Multimodal dimensions of discourse] Moscow, YaSK Publ., 2021, pp. 153–238. (in Russ.) of artistic discourse. “Transcoding,” i.e. the transfer of a message from one format to another, is the way to convert the message from one format to another (mainly from analogue to digital). A classification of different types of multimodal poetic texts includes visual, auditory and auditory-visual or synthetic types of transcoding. It is based on the aesthetic-semiotic typology of art forms on the one hand, and on the transcoding mechanism on the other. The paper examines the third, synthetic, type of transcoding in detail through the case study of the Italian poet Vittorio V. Zollo’s multimodal poem ’A Via Crucis (2020).

Institute of Philology
Nikolaeva st., 8, Novosibirsk, 630090, Russian Federation
+7-383-330-15-18, ifl@philology.nsc.ru