Thursday, 5 August 2010

DRHA rough draft:

Main introduction
Core questions
Structure of paper
Why and what (motivation)
Mini background and literature review
Specific interventions and tools offered by VAINS

The VAINS platform offers a range of tools to facilitate the exploration and interpretation of online art works. These tools have been constructed with a view to fulfilling the following objectives:

To deploy networked and subjectively situated software architectures in the context of recent interrogations upon the role of the curator in the online context (Krysa 2006).
To deploy subjectivity and embodiment as core resources for the curatorial practice.
To explore the validity and practicality of anti-representationalism (as delineated by Varela and Maturana among others) and its associated methodologies, including enactivism, situated and embodied cognition, within such software.

These goals have been validated by an analysis of relevant theoretical and methodological contributions to the areas of human-computer-interaction, embodiment, and situated cognition. We have researched both current and historical literature on these topics. As a result we have found valuable analyses in the works of Shaun Gallagher (2006, 2009), Donna Haraway (1991), Robbins and Aydede (2009), Bill Gaver and Gaver et al (1999, 2004, and 2008) and Phoebe Sengers (2006), Henri Bergson (1896, 1911), Hubert Dreyfus (1992), Julia Kristeva (1982), Elizabeth Grosz, Karen Barad (2007) and Lucy Suchman (1987, 2005, 2006). The analyses presented by these writers validate the proposition that human beings are entangled with their technologies and with complex, relational and temporally bound systems of agency. Hence, a core methodological commitment embedded in the VAINS project is the confrontation or re-framing of the body in online curation, this has been supported by an examination of the separation between computers and humans, and, indeed, the ready made separations that we project between subjects and objects (including visitors and online galleries). Another important aspect of this work has been to identify the significant material features of computational interaction while acknowledging that computers are not clearly separable from ourselves, but, like all human artefacts, are of us.

Enactivism, embodied and situated cognition

To clarify the relationship of the VAINS tools to enactivism, embodiment and situated cognition we would like to frame them within a historical context in which computation has been dominated by top-down, disembodied and propositionally based structures. Enactvism offered a radical break from this construction, emphasizing the way that organisms and human minds interact with their environments. These ideas are in many ways the precursors of situated cognition and embodied cognition, and are presented as an alternative, middle way or via media between extremes of relativist subjectivism, cognitivism, computationalism and Cartesian dualism.

The methodologies represented by enactivism and situated cognition offer the possibility of constructing an alternative form of digitally curated space, one that deploys embodied subjectivity and situatedness as valid and valuable resources in the generation of new creative insights and actions in the field of online arts.

The tools distributed by the VAINS platform enable, to quote Maturana and Varela, a means of understanding how ‘our world, as the world which we bring forth in our coexistence with others, will always have precisely that mixture of regularity and mutability, that combination of solidity and shifting sand, so typical of human experience when we look at it up close’ (Maturana, Varela, 1992:241). This position has been critiqued as potentially solipsistic (see Svenson, 1992) however, such a methodological standpoint should be seen as part of a wider move in computing towards a re-conceptualisation of the body and a move away from a rigid Cartesian split between mind and body, described as ‘a division of labor that was not simply theoretical and a problem for philosophers, but that was finding its way into the pragmatics of every day life’ (Gallagher in Robbins, Aydede, 2009:37). This reframing is an opportunity to integrate the body into new epistemologies and methodological approaches. As Grosz states in (1994), eschewing disembodied, computational models of cognition represents an opportunity to ‘displace the centrality of the mind, the psyche, interior or consciousness (and even the unconscious) in conceptions of the subject through a reconfiguration of the body’ (Grosz, 1994: v ii). But, in reconfiguring the body we might also seize an opportunity to reconfigure the inter-subjective and technological boundaries between bodies and computers.

The Abjection application

One of the first mobile applications we have offered VAINS visitors is called Abjection. This application encourages users (or user-researchers) to investigate the bodily traces they have left within their digital equipment. The identification of such visceral traces may be seen as an interrogation of the notion of the immateriality of our interaction with digital technology, and, perhaps even a challenge to the notion of a stable virtuality. More significantly still, and in keeping with Juia Kristeva’s framing of abjection, it is proffered by VAINS as a challenge to the stability and sovereignty of the self. Abjection, which we will define as the horror of the materials ejected by the body, (which we customarily consider to be unclean), is deployed within the VAINS mobile application to test the limits of the boundaries between ourselves and the technologies we use, between subjects and objects. The Abjection application takes its users through a series of investigative procedures; these involve analysing the various abject materials that we embed into the keypads and ear pieces of laptops and phones.

Offering mobile and other applications to VAINS visitors also tests the boundaries of the web, it acknowledges the fact that all websites are also part of a wider social and media ecology. VAINS offers both web-based and other tools, such as mobile applications that can be ‘taken away’ from the web and used in other contexts. These tools have the potential to include locative media such as GPS, messaging and haptic feedback (which we will describe in more detail in this section).
Above, a screen shot from the VAINS mobile application Abjection
Available here:

Collaborative filtering: abject traces and new conceptualisations of the interactive.

In addition to the visceral and bodily traces left by users of digital technology, we also invite visitors to consider the many other traces of themselves that they (often unwittingly) leave behind. Bruno Latour (2007) has framed these traces as representing a significant erosion of the differences between the private and the public:
The ancient divide between the social on the one hand and the psychological on the other was largely an artefact of an asymmetry between the traceability of various types of carriers: what Proust’s narrator was doing with his heroes, no one could say, thus it was said to be private and left to psychology; what Proust earned from his book was calculable, and thus was made part of the social or the economic sphere. But today the data bank of has simultaneous access to my most subtle preferences as well as to my Visa card. As soon as I purchase on the web, I erase the difference between the social, the economic and the psychological, just because of the range of traces I leave behind.
(Latour, 2007)

Collecting and instrumentalising data from users of web sites, whether covert or consensual, is now part of the materiality and medium specificity of the web. But the traces Bruno Latour writes of are arguably rarely made visible to the users who have left them. VAINS makes these traces part of the materiality and navigational structure of the platform.

Below, a screenshot from 3d interface that dynamically reacts to user evaluations of the VAINS content

The VAINS practice hinges around the tensions inherent in the construction of subjectivity, singularity and collectivism, but, as many of the writers we have referenced maintain, we cannot easily reach a consensus as to what a subject is or even if such an entity really exists. This ambiguity and fluidity is an instrumental presence within this practice, to quote Barbara Bolt, it is a practice in which ‘the materials are not just passive objects to be used instrumentally by the artist, but rather the materials and processes of production have their own intelligence that come into play’ (Bolt, 2006:1). This becomes a case in point in our deployment of collaborative filtering within the VAINS platform. In VAINS collaborative filtering assumes both non-instrumental and instrumental qualities according to the unpredictable materiality of the dynamic system at play. Non-Instrumentality is described by Lowgren as ‘aesthetic, ludic and social qualities’ (Lowgren, 2008)) and instrumentality as ‘usability’ and ‘usefulness’.

The VAINS platform gathers data online in order to create a user-based collaborative filtering system, using php and mysql. The algorithms we have constructed find closest matches, for mood, weather, gender, age, location and other specific and situating variables. They find what the closest co-users in the system liked as well as negative correlations, such as what users might not like. Such filtering may also be called opinion mining, or sentiment analysis, within VAINS it is framed as a complex means of investigating both subjective and collective responses to online artworks. The VAINS algorithms collect and generate multi-dimensional similarity metrics for VAINS visitors based on the data-set and probable interests of new visitors. These are dynamic and offer a constantly evolving picture of the content and how users react to and engage with it. However, in offering recommendations it is important to emphasise that the VAINS system is not framed as a truth making mechanism, rather it is framed as a provocation to contemplate the validity and putative neutrality of such data-mining processes. In this way visitors to VAINS are invited to look at works which the system might have assessed as of no interest to them, or works that might be considered as statistically more suited to a different age, gender or other social grouping.

Above and below, the VAINS platform works playfully with the notion of demographics and demographic targeting, it offers these practices as sites of exploration and provocation to its visitors.

Above, the rule of thirds applet – reveals compositional conventions in web pages and web based artworks.

Other tool offered by the VAINS platform create navigational paths and collaborative filters to artworks based on interfaces that are reflexive and game-like, deploying embodied feedback, such as reaction times and hand-to-eye coordination. These programs can dynamically impact on the overall structure of the VAINS navigation system and on each individual’s experience of the site. If visitors want to understand the mechanisms at play they are able to access the code we have written to generate these experiences. Examining the code will reveal the underlying computational logic as well as the logic of our methodologies.

Below, excerpt from the code for the Frustrater interface
Above, the frustrater interface, the links jump rapidly around the screen, confounding expectations of
usability and control, it is offered as a provocation to VAINS visitors.

Above VAINS reflex linking interface, visitors are taken to different links according to their physical reaction times to an on-screen stimulus. The prototype system collects reaction times and links them to user preferences so that it may make aesthetic or navigational predictions based on embodied reactions, such as the reflexes of visitors.

The forms of navigation and filtering we have discussed are often engaged with alternative computational epistemologies, these epistemologies challenge the propositional logic at the heart of what is called Good Old Fashioned Artificial Intelligence – a narrow conception of human intelligence and being in the world that denies the body and certainly denies the political and cultural significance of its own logic. In the future we plan to evolve more subtly embodied and situated navigation and interpretation systems by developing software that reacts to bio-feedback and to more complex networks of embodied and situating factors.

Textural or haptic interfaces

In addition to the tools we have already constructed, in the future we envisage using haptic technology (derived from the Greek word haptesthai, pertaining to the sense of touch). Such emerging technology will make it possible to design haptic systems that provide users with physical feedback in the form of actuators from pager motors in mobile phones to high resolution three-dimensional force feedback systems that facilitate the haptic simulation of objects and textures. These systems may become more widespread as mobile technologies increasingly embed gaming and haptic technologies as core features. The VAINS platform proposes deploying more haptic means of navigating and exploring online works and of engaging the body directly in such experiences. For example the Anycall Haptic mobile phone features a large touch-screen display like the iPhone. But, to quote the manufacturer, Samsung, ‘It enables users to feel clicks, vibrations and other tactile input. In all, it provides the user with 22 kinds of touch sensations.’ These touch sensations may be deployed by artists, designers and programmers to investigate subtle forms of sensory navigation in digital platforms.

Above, the Samsung Anycall Haptic
Illustrating a mock up for a proposed haptic device
for interacting with digital artworks, this can deploy 22
forms of haptic feedback.


The tools we have produced are still in process, indeed it is our recommendation that such tools always be ‘in process’, and that a process-based orientation is brought to this practice. This recommendation is consistent with both our experimental deductions (evidenced in the following section) and our methodological commitment to a dynamic, inter-subjective and networked system of continuously evolving interactions. The next section will describe two experiments we conducted in order to evaluate and explore both sensory and overtly symbolic means of interaction with the VAINS platform, it will outline some of the problematic aspects of user evaluation, these aspects echo the themes we have raised already in this paper of instrumentality and non-instrumentality, navigating a delicate via media between extremes of subjectivism and positivist realism, embodiment and a priori representation.

Iconisation and sonicification: testing hypotheses
Sonicification: Sounds users to choose to represent experiential responses to works

The section will discuss our testing of hypotheses concerned with participation, iconography and curation, it will also evidence some of the difficulties inherent in imposing notions such as assessment and user evaluation in relation to an arts project, it will discuss how we have utilized but also destabilized these concepts in the service of multiple interpretation, provocation and play. And that establishing a via media between instrumentality and non-determination is in keeping with the logic of our research process.

Evaluation methods:

Structured depth interviews, also Lee’s discussion with participants
Questionnaires and more creative forms of participatory writing.
Feedback interview.

THE Evaluation of User experience
In writing about the evaluation of user experience we have found the work of Bill Gaver (2006), Gaver et al (1999, 2004, and 2008) and Phoebe Sengers, (2006) particularly relevant. Evaluation within an arts context is an ambivalent and often neglected concept, we have therefore drawn upon wider disciplines to investigate how we might evaluate our work with VAINS and the experience that it engenders in others. Gaver and Sengers (2006) frame evaluation within a methodological strategy, or logic of research generation, that is characterised by a commitment to multiplicity of interpretation. As this section will show, part of the logic of our evaluation of VAINS visitors and the solicitation of feedback from them, is to allow visitors to reconstruct the system according to their own epistemological perspectives.

As Sengers and Gaver (2006) observe, there are costly limitations inherit in judging user experience against the simplified expectations of the artist, programmer or designer who has produced the system in question. As alternatives to single interpretation we show how the VAINS system has solicited complex, multi-layered and often contradictory interpretations, characterising them not as problems in need of solutions, but as creative and welcome resources within the context of an arts computing based project. Gaver and Sengers (2006) have outlined useful evaluation strategies that can accommodate multiple and complex interpretation of human-computer-interactions, in which ‘potentially competing interpretations can fruitfully co-exist’ (Gaver and Sengers, 2006:1). They also document how ‘design and evaluation strategies shift when we abandon the presumption that a specific, authoritative interpretation of the systems we build is necessary, possible or desirable’ (1), stating that it is difficult ‘to conceive of interaction without interpretation’ (1). The multiple meanings that are assigned to computational systems by their users emphasize the importance of interpretation within HCI. Although the notion of single use and single interpretation may be appropriate in some cases, though Sengers and Gaver point out that even in issues of road safety it is sometimes better to stimulate drivers and pedestrians into making their own, non-passive interpretations of safe behaviour, rather than telling them what to do.

In the case of VAINS we are producing a system that is open to interpretation on many levels, from the interface itself, to the overarching significance of the entire system, summed up in the questions ‘what role can it play in my life’ (2) and ‘what does it mean about me, my social group, my society, my culture’? (2). These questions, among others, are ones that we have asked a range of people in relation to aspects of VAINS, confirming the assertion by Gaver and Sengers (2006) that multiple interpretations are almost inevitable in relation to computational systems.

Testing icons

Within the context of VAINS it has been engaging and productive to allow for ambivalence and multiplicity of interpretation. With this ambivalence of purpose in mind, notions of utility are also challenged, and as, Gaver and Sengers state, ‘alternative values, such as curiosity, play, exploration, and reflection are also important from this point of view’ (3). Gaver and Sengers emphasize the importance of generating new strategies and methods for creating systems that embody these alternative values, such as ‘purposely blocking’ (4) and, thwarting ‘any consistent interpretation’ (4). This, Gaver and Sengers are at pains to point out is not the same as deliberately generating confusion at the level of usability, stating that ‘what the system does and how it can be controlled is obvious – but the ultimate purpose meaning and usefulness of the device is left open for users to decide’ (4).

Above, a participant documenting
their reaction to VAINS icons

In addition to the feedback forms illustrated above, other methods we have used to solicit feedback and user evaluation are:

Usability testing
Depth interviews.
Participatory iconisation, sonicification and texturisation.

It has also been valuable to analyse our own unconscious projections and meanings in relation to this work. The complexity involved in this approach has arguably been more suited to an arts-computing project than paradigms of unitary interpretation and idealised conceptions of impenetrable scientific objectivity.

Common approaches to evaluation in HCI are based on developing and testing against a priori evaluation criteria corresponding to the designers’ anticipated interpretation of a system. But in taking multiple interpretations into account, systems can no longer be effectively evaluated in terms of criteria generated from a single, authoritative interpretation.
(Gaver and Sengers, 2006:7)

Notes for final Conclusion
strengths and weaknesses,
future research
Discussion of findings and analysis – relate to literature etc

Check in paper:
Explicitly stated aims for investigation
Answer a well defined broad question
Link existing knowledge (lit and examples) to our investigation.
Well structured and appropriate methodological approach and means of testing hypotheses


Problems Section in Conclusion
We have discussed some of the issues involved in evaluation
Problems of intervention and innovation
It is important to acknowledge the problems we face in offering alternative structures to the orthodox modes of online and, indeed, offline, interaction we have become habituated too. In light of their innovation the alternative devices we have constructed for VAINS may be perceived as ‘interventions’, in contrast to the arguably ‘naturalised’ structures that we customarily deal with. This is one of the difficulties of an interdisciplinary collaboration, in which an artist-programmer (Eleanor Dare) is collaborating with an artist-curator (Lee Weinberg. Our respective methodologies, though often compatible, are also in many ways at odds with each other. For example, the notion of intervention has been identified by Lee as potentially opposed to an ethical curatorial practice. However, one might equally observe that curation per se is a form of intervention, that the placing and interpretation of artworks always represents a view from somewhere (to cite both Haraway and Nagel), and that these actions can never be neutral.These are ongoing dialogues within the context of VAINS and the tools we have developed are offered as portals or provocations to engender further debate about appropriate mechanisms and contexts for the curation of digital artworks.

A criticality of perspective is also maintained in relation to the idea of user evaluation. The use of ambiguous methods such as participant iconisation, texturisation and sonicification emphasise some of the illusory aspects of such evaluation, serving to outweigh the idealisations that can arise in notions of ‘end-user’ assessment.



Barad, K. (2007) Meeting the Universe Halfway, Quantum Physics and the Entanglement of Matter and Meaning, Durham, N.C.: Duke Unversity Press.

Bergson, H. (1911) Creative Evolution, London: Macmillan.

Bergson, H. (1896) Matter and Memory, New York : Zone Books (1990 edition).

Bolt, B. (2004) Art Beyond Representation, London: I.B Tauris.

Carter, Paul (2004) Material Thinking, Melbourne University Press.

Dreyfus, H. L. (1972) What Computers Still Can’t Do, A Critique of Artificial Reason, Cambridge, MA: The MIT press (revised edition 1992).

Gallagher, S. (2006) How the Body Shapes the Mind, Oxford University Press.

Gaver, Bill, Sengers, Phoebe (2006) Staying Open to Interpretation: Engaging Multiple Meanings in Design and Evaluation, available here:
accessed 16/09/09

Grosz, E. (1994) Volatile Bodies: Toward a Corporeal Feminism, Bloomington: Indiana University Press.

Haraway D. (1991) ‘The Cyborg Manifesto’, from, Simians Cyborgs and Women: The Reinvention of Nature, New York, Routledge.

Kristeva, J. (1982) Powers of Horror, Columbia University Press.
Latour, B.. (2007) Beware, your imagination leaves digital traces
Available here
accessed 05/06/10

Lowgren, J. (2008) Encyclopedia entry on Interaction Design. Retrieved 5 August 2010 from

Maturana, H, and Varela, F. J. (1992) The Tree of Knowledge: The Biological Roots of Human Understanding, Shambala.

Samsung Anycall Haptic:

Swenson, R. (1992) Galileo, Babel, and Autopoiesis (It's Turtles All The Way Down). Int. J. General Systems, Vol. 21, No. 2, pp. 267-269.

VAINS project web site:

No comments:

Post a Comment