Post-desktop publishing: future books in an era of ubiquitous digitality. The Implications of intelligent and pervasive book forms.


Abstract

If digital literature is going to emerge as anything other than a facsimile of traditional texts it must deploy the significant strengths that contemporary computation has to offer. This paper will outline my reasons for stating this while also outlining and contextualising the software and book I have developed for my doctoral practice. This software generates bespoke literary content that works in collaboration with an analogue book called South:a psychometric text adventure. I shall explain how philosophies of Real World Interaction and situated programming have informed the development of this work, and why it represents a critique of previous digital forms and the underlying logic of traditional Artificial Intelligence, which I shall argue, has failed to deliver any significant digitally mediated literature.

Author Key words: Digital Technology, The Future of Books, Artificial Intelligence, Computing, Dissemination, Programming, Materiality, Pervasive Computing, Post-Desktop
Stream: Information Society, and Print and Electronic Texts

Introduction

This paper will describe the innovative practice based research I have taken over the last three years as a doctoral researcher and writer at Goldsmiths, Department of Computing, University of London. My research has centred upon the development of interactive software and a book entitled South: a Psychometric Text Adventure (available here in both electronic and paper forms http://www.lulu.com/product/paperback/south/12924476).

The project re-conceptualises the book form, drawing formally upon elements of self-help and text adventures while technically deploying real world interaction, embodiment and situated computing. I will explain what this means and the implications of using technology in alternative paradigms for developing relevant new book forms.

I will evidence the outcomes of my practice and explain how this work has involved deepening conceptions of the relationship between book design, literary content and the complex capabilities of digital-technology. The paper will discuss insights into computational structures, and the materiality of programming practices. I will argue that my status as a writer-programmer, working within a supportive interdisciplinary academy has been conducive to enhancing the development of skills, knowledge and operational effectiveness that can preserve the centrality of the book form while accommodating the meaningful benefits of digital technologies. These benefits are not restricted to dissemination alone, but also have important implications for the structure, form and content of future books


Context and core questions

The South project asks three central questions:

Have digital book forms been successful or have they failed to make the impact initially heralded by for example, Landow (1992) and Bolter (1991)?
If so what are the causes of this failure?
What alternative technological structures and systems might be investigated?

These questions need to be framed in light of the polarised views of digital text that emerged from the 1990s onwards. Writers such as Birkerts (1994) and Updike (2006) characterised hypertext fiction as a threat to traditional reading while Landow (1992) and Bolter (1991) viewed it as representative of a positive shift towards empowering readers, a position that was supported by drawing heavily upon the theories of Roland Barthes (1970), upon notions of ‘writerly’ and ‘readerly’ texts (so called writerly texts require readers to actively construct or produce meaning, whereas readerly texts, according to Barthes, require static, pre-determined, forms of ‘passive’ reading), and upon Michel Foucault’s ideas about the decentring of authorial agency. But despite these two views there is little evidence to support the idea that traditional reading forms are dead or, on the other hand, that hypertext and other digital fictions have engendered a radical shift in traditional modes of reading and writing (though there are other valid debates about changes in attention span, or ‘deep focus reading’ which I will not enter into here). In 1992 Landow wrote;

Electronic text processing marks the next major shift in information technology after the development of the printed book. It promises (or threatens) to produce effects on our culture, particularly on our literature, education, criticism and scholarship, just as radical as those produced by Gutenberg's movable type.
(Landow, 18: 1992).

But has electronic text really had a significant effect upon our reading and writing of fiction, and is the much heralded ‘non-linearity’ of digital text a reality? Espen Aarseth (1997) challenges the notion that digital texts are non-linear while analogue texts are linear, he points out that a book can be randomly accessed as easily as a hypertext, indeed, it is possible to code hypertext in such a way as to control the order of reading above and beyond that of analogue texts. This is also true of other forms of computational text. In 1994 Sven Birkerts wrote of hypertext that it is:

delivering a mighty blow to the long-static writer-reader relationship. It changes the entire system of power upon which the literary experience has been predicated; it rewrites the contract from start to finish.
(Birkerts, 1994: 163).

My research has lead me to question both premises, to ask if it has really been the case that hypertext literature has either ‘revolutionised’ or ‘destroyed’ traditional forms of fiction. Between these two logically and philosophically opposed, but mutually reinforcing positions, there is a third view, that despite the rise of ubiquitous digitality, hypertext and wider forms of digital fiction have, by and large, failed to extend analogue book forms, let alone challenge or threaten them. Far from offering a radical break from traditional books, digital forms of literature are almost invariably blank re-mediations of them, and in the case of e-books, facsimiles of analogue books. Mc Neil (2008) takes the view that the legacy of hypertext has been less than impressive:

ten years later most people don’t even know what Hypertext fiction is. As computer graphics advanced and modems were replaced with Ethernet hookups, users looked for connectivity, not the locked in world of CD-ROM entertainment. Plus, that interactive “Choose Your Own Adventure” spirit already exists in literature like Julio Cortazar’s “Hopscotch” and Mark Z. Danielewski’s “House of Leaves” — and you can take those with you on a plane.
(Mac Neil, 2008 available at http://tomorrowmuseum.com/tag/fiction/page/3/ accessed 10/09/10)


The failure of digital fiction and Good Old Fashioned Artificial Intelligence

It is therefore my primary argument in approaching this research that digital literature has failed to make a significant impact on literary fiction. The second motivating foundation for my research has stemmed from correlating the failure of digital fiction to the wider failure of Artificial Intelligence, specifically the form of Artificial Intelligence known as GOFAI – or good old fashioned artificial intelligence.

GOFAI represents one particular view of human intelligence that has been comprehensively critiqued by, among others, Hubert Dreyfuss (1972) and Alison Adam (1998). Dreyfuss and Adam note the limitations of propositional logic (a form of logic also known as sentential logic, based on combinations of statements, such as dogs have four legs, Elizabeth Windsor is the Queen of England, which are often abstracted into truth tables). Writers such as Lucy Suchman (1987, 2007) offer viable alternatives to GOFAI and therefore to purely propositional, a priori structures, in the form of situated programming. Situated programming has played a significant role in the development of the South book and software. Situated programming relates to the notion of software agents (modules of code that can make things happen in a program) existing in a dynamic environment. Situatedness is also related to the notion of embodiment, as, in order to understand their relationship to an environment, software agents rely on sensors which emulate human senses, such as touch, sight and sonic awareness. Situated programs therefore have a deep and dynamic relationship to the environments in which they operate. They are based on the more contemporary notion that intelligent behaviour emerges from the dynamic interplay between brains, bodies and situations (tantamount to changeable environments). More recently writers and practitioners such as Justine Cassell (2007), Cassell and Vilhjalmsson (2001) and Dourish (2001) have provided further insights into paradigms of digital interactivity that draw upon the body and the real world as opposed to hard coded a priori models of a disembodied and static world.


Real World Interaction

According to Jacob et al, practitioners of Real World Interaction:

employ themes of reality such as users’ understanding of naive physics, their own bodies, the surrounding environment and other people. They thereby attempt to make computer interaction more like interacting with the real, non-digital world
(Jacob et al, 2008).

It is important to acknowledge as Jacob et al do, the difficulty of terms such as ‘real world’, which I would qualify here as meaning social awareness, including inter-subjective interactions, environmental situatedness, (the sense of being in an environment and being able to navigate and manipulate it), and embodiment, particularly proprioceptive awareness of ourselves as bodies within an environment bound by gravity. As I have stated, earlier a priori models have been the legacy of first generation computing which have deployed command line arguments as opposed to direct manipulation, many narrative systems have been built on complex knowledge bases which, as Dreyfus (1972) Adams (1998), and Suchman (1987, 2007) argue necessitate an infinite regress of rules. As a counter argument practitioners (programmer story tellers) such as Chris Crawford have argued that they can see no reason why a computer program should not be able to generate its own complete literary works that are ‘equal’ to those of writers

There can be no doubt that C++ has the expressive potential to write a program capable of, say, writing profound novels
(Crawford in Sengers, Mateas, 2003: 195).


But there is little evidence for the accuracy of this statement, and also, tellingly, no ‘feel’ for the materiality of computing and the fundamental asymmetries of understanding that exist between embodied human beings and disembodied computers. Jacob et al have predicted instead that RWI will become the next major paradigm of Human Computer Interaction. Their analysis highlights a growing move towards embodied interfaces that is only supported by the mass market of iPhones, Androids and iPads that now use cameras and motion to control programs. RWI itself has arguably evolved from a complex interdisciplinary backdrop in which embodied and situated computing has become more prominent, especially in light of the failure of good old fashioned artificial intelligence to deliver the type of insights and understandings it had once promised us.


Another significant critique of abstract cognitive models has come from the philosophers Elizabeth Grosz (1994) and Judith Butler (1990) who have argued for a re-conceptualisation of the body in all systems of human knowledge production. It is in the context of these re-conceptualisations of Human Computer Interaction and the urgent case for methodologies and processes that can preserve the centrality of the book form while embracing the meaningful benefits of digital technologies, that I have developed the South project.


South: a psychometric text adventure

Above, the South book and egg. The egg is a via media between digital and analogue book forms.

Every good book needs a protagonist and luckily you’ve come along just in time to fulfil that role. According to How to Write a Blockbuster ‘characters provide readers with the emotional key, the route into your story’. This principle seems equally applicable to any decent guidebook or gazetteer, but the authors of this book also warn: ‘if your characters are flat and lifeless, no one will want to read it’.
Are you feeling flat and lifeless? You’re in for a pretty boring read if you are. Your job is ‘come alive’ so that I can just sit back and type. You should expertly discover who you ‘really’ are and help the rest of us to get to know you.

Above, an excerpt from South: a psychometric text adventure, Dare, 2010:8.


The South project consists of a book, a software interface and an ‘egg’ which uploads tailor-made content for individual readers. The project exists within the context of my ongoing practice as a writer and fine artist concerned with investigating the meaningful strengths and characteristics that computation can bring to the experience of reading and writing. The software works dynamically with the book in order to provide readers with a tailor made experience of the book, which is in part a travel guide, self-help manual and novel, centred upon the South Bank area of South London.

The book features fictional narratives about and ‘by’ a reclusive computer scientist called Ivan Dâr, the fictional creator of the South software. In order to tell the story of Ivan Dâr the book places location, sensation and tactile response at the forefront of the site-oriented experience it offers to readers. Readers are invited to construct their own narrative interpretations through touch, sound and movement, to locate themselves within and through the environment in which the book is set.


The non-linear South book and its readers, like the volatile content of the Thames itself are inherently unstable and localised, at least in my conception of them. The interdependency of each algorithm, and its direct relationship to the subjective state of readers, enables the book to also at times deliberately limit the mobility of users, emphasising the idealisations as much as the realities at play in this work.
In conjunction with the South software and egg, the book takes on an even greater degree of contingency, issuing instructions that have been generated in response to economic and meteorological events as well as my own subjective changes. It is significant that in this largely mutable configuration it is the overtly fictional content, the stories I have written about Ivan Dâr and his subjective dis-integration that retain the greatest degree of stability. Ambiguity, tension and mystery, if they exist at all, emanate from the experiences readers generate for themselves in the materiality of their research into the site and their own subjectivity and embodiment.


The South project assumes therefore that its readers are intricate, intelligent and often inconsistent. The current form of the South book assumes a broad readership, ranging from people who are interested in London, walking and psycho-geography to those who are curious about what a location can tell them about themselves. Other readers may come to the book through their interest in game forms such as ‘Choose Your Own Adventure’ books and text adventures. All readers will find themselves required to engage in a series of psychometric (or psychologically evaluative) processes woven in and around the South Bank. Depending on specific interactions and situations the egg will also tell readers to look for hidden content in the book, and to navigate new pathways through both the paper-based text and the physical location. Interaction with the South project is divided into three distinct processes:

1. When a reader opens the software it automatically scans the web for news and other situating information, including the local weather and financial data.
2. The South software evaluates readers via psychometric testing and generates appropriate and challenging content for them that complements the book.
3. The reader uploads the freshly bespoke content into the egg and goes to the South Bank with the book and egg. The egg will guide them through both the book and the location in ways that are unique to each occurrence of interaction.

evaluation of
readers and
computer based
narrative generation
[using a range of
processes including
language and image analysis]
egg content
generation

Data gathering and
Analysis
(Pre-processing)
Three distinct processes:




Above, the software generates bespoke content and tasks to help users explore the South Bank in a way that is personally meaningful to them. The content is held inside the interactive egg device. The central character of the book, Ivan Dâr, is an avian embryologist

.
Bespoke content and types of knowledge
Many of the evaluative procedures deployed by the South software and book involve the senses, and indeed the progression of the evaluations through the five senses is part of the underlying narrative of the assessment process. The emphasis upon sensory and embodied interaction in both the South book and software also enforces the central notion that the technology we use does not exist in isolation from the cultural or physical spaces in which we live and work. The question of what constitutes knowledge or intelligence in these tests is also challenged (it is also an important question in the context of claims about intelligence and computing), what, the book asks, do we mean by intelligence? Are there other types of intelligence or knowledge that computers and conventional research processes can deploy, such as:

• Embodied knowledge

• Tacit knowledge[1]

• Situated knowledge

The processes presented in the South book and software are designed to facilitate an exploration of these questions within and through the South Bank location. Ivan Dâr, the central character of South is framed as a man literally in search of himself, struggling with ‘breaking the boundary of his own skin’ (as Roger Callois puts it). Ivan Dâr endures a psycho-physical breakdown in which he cannot find himself because he appears to have no location in space. At times the book deliberately aims to bewilder or undermine its own readers, hinting at the possibility of space as an annihilating agency working against them, generating a hostile space by means of its aesthetic strategies and by asking readers to undertake paradoxical or impossible tasks, lying to them, issuing contradictory instructions or leaving exasperating lose ends. Ivan Dâr’s fictions return to the theme of his so-called sickness and the strategies he devises to operate within the city despite his phobias and disintegrating boundaries.
In this way the South project uses situated Artificial Intelligence techniques such as language and image analysis as well as pattern matching, psychometric and embodied evaluations to gain insights into individual readers and to individual situations as well as to more widely felt experiences, such as breaking news, world temperatures, and financial data. Real World Interaction allows us to explore modes of interaction with books that can draw upon the situatedness and embodiment of human readers, likewise, by deploying digital technology and evaluative procedures bespoke content can be combined with stable analogue content.




About Your Egg

The egg is a proto-novel, a book waiting to be hatched. We suggest you keep the egg close to you for the first few days. If you have spare folds of fat this is great, you can tuck the egg into them. Otherwise you might like to keep the egg in your pocket or under your jumper while watching television. When the egg has reached a temperature of about 67 degrees your book is ready to be hatched. Grab this book, head down to the Southbank, turn the egg on and take it for walk. By pressing the forward and backward buttons at the back of your egg you can experience the enhanced navigational range of this book. This will involve going up its stairs and through its secret tunnels, scampering over its many rooftops, clambering along ridges and down into its green valleys to find new and previously undiscovered pages. The book is designed to be smudged, thumbed, bumped, riffled through, used as a pillow, an occasional table and drawing book. It is also a form of compass. One way or another, (if you stay on Earth) this book will always point towards the South, just like your egg.


Above, an excerpt from South: a psychometric text adventure, Dare, 2010: 254. Below, tailor made content for readers. The egg guides people through the book in unique paths. The text is not designed to be read linearly, but structured in collaboration with individual readers and the South software.







Above and below, computer generated
content for a digital book, Eleanor Dare 2010


Conclusion

In order for computation to bring significant and worthwhile mediations to literary texts it is necessary to use digital technologies, not as passive vessels alone, but as medium specific tools. Devices such as Kindle and iPhones are a case in point. The computational abilities of these devices could be used to meaningful effect rather than as blank re-mediations of traditional book forms, mere holders of analogue facsimiles. The South project has been an effective proof of concept, evidencing that it is possible to create bespoke literary content while preserving those aspects of book form that so many people cherish, such as the materiality of books and the tactile intimacy of reading. My own methodology has involved paying attention to the particularity of programming structures and paradigms. I would contend that such an analysis has not been extensively engaged upon by previous generations of digital fiction authors., At the same time the structures and orthodoxies of programming have been submissively accepted and rarely evolved by such authors, hence perhaps the stagnation of such forms as IF (interactive fiction) and hypertext fiction.
It is perhaps a challenging but exciting prospect that writers of digitally mediated literature need to develop computational skills and, likewise, programmers involved in this field need to know how to write. Interdisciplinary initiatives such as those at Goldsmiths where I undertook my doctoral and masters research will enable programmers, technologists and writers to gain insights into each other’s domains, and in my case to become both programmers and writers. Where this is not directly possible it behoves researchers such as myself to facilitate and disseminate insights into computational structures and the literary possibilities of emerging technologies. The South book contains detailed information about how to hack the code I have written, enabling those who are interested to experiment with programming new kinds of writing and thus to encourage a future that will tolerantly investigate and benefit from diverse book forms.



­­­­­­­­­­­­­______________________________________________________________

The South book is available here in electronic and paper forms:
http://www.lulu.com/product/paperback/south/12924476
The South software can be downloaded from here:
http://www.doc.gold.ac.uk/~ma501ed/south/south_Download.html


Bibliography

Aarseth, E. J. Cybertext: Perspectives on ergotic literature. Baltimore: Johns Hopkins University Press, 1997.

Adam, A, (1998) Artificial Knowing, Gender and the Thinking Machine, London and New York: First Routledge.

Barad, K. (2007) Meeting the Universe Halfway, Quantum Physics and the Entanglement of Matter and Meaning, Durham, N.C.: Duke University Press.

Barthes, R. (1970) S/Z, Mass Market Paperback, Editions du Seuil.

Birkerts, S. The Gutenberg Elegies: The Fate of Reading in the Elecronic Age. New York: Fawcett Columbine, 1994.

Bolter, D. J. Writing space: The computer, hypertext, and the history of writing. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1991.

Butler, J. (1999) Gender Trouble, Routledge.

Cassell, Justine, Kopp, Stefan, Tepper, Paul, Ferriman, Kim & Striegnitz, K . (2007) "Trading Spaces: How Humans and Humanoids use Speech and Gesture to Give Directions." In T. Nishida (ed.) Conversational Informatics. New York: John Wiley & Sons, pp. 133-160
Cassell, J., Bickmore, T., Vilhjalmsson, H., Yan, H. (2001). "More Than Just a Pretty Face: Conversational Protocols and the Affordances of Embodiment." Knowledge-Based Systems 14: 55-64.
Dourish, P. 2001. Where the Action Is: The Foundations of Embodied Interaction. Cambridge: MIT Press.

Dreyfus, H. L. (1972) What Computers Still Can’t Do, A Critique of Artificial Reason,

Gallagher, S. (2005) How the Body Shapes the Mind, Oxford University Press.

Grosz, E. (1994) Volatile Bodies Toward a Corporeal Feminism, Bloomington: Indiana University Press.

Jacob, R. J., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., and Zigelbaum, J. (2008). Reality-based interaction: a framework for post-WIMP interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '08. ACM Press, New York, pp. 201-210.

Kelly, Kevin (2006) Scan This Book! Available here:http://www.nytimes.com/2006/05/14/magazine/14publishing.html?pagewanted=1 accessed 03/10/10
Landow, G. P. Hypertext 2.0: The convergence of contemporary critical theory and technology. Baltimore: Johns Hopkins University Press, 1997.

Landow, G. P. Hypertext: The convergence of contemporary critical theory and technology. Baltimore: Johns Hopkins University Press, 1992.

Mateas, M. and Sengers, P. (eds) (2003) Narrative Intelligence, Amsterdam: John Benjamins Publishing.

Suchman, L. (1987) Plans and Situated Actions: The Problem of Human-machine Communication, New York: Cambridge University Press.

Suchman, L. (2007) Human-Machine Reconfigurations: Plans and Situated Actions, New York: Cambridge University Press.

Updike, J. (2006) The End of Authorship available here:http://www.nytimes.com/2006/06/25/books/review/25updike.html?_r=1&scp=1&sq=updike%20+%20snippets&st=cse accessed 04/10/10

[1] Tacit knowledge is summed up by the scientist and philosopher Michael Polanyi in his statement "We know more than we can tell." (Polyani, 2009: 4)Tacit knowledge implies a multi-layered background of subjective and unformalised skills and knowledge, including cultural and embodied forms of knowing.