Transmediale 2016

Event link:
http://2016.transmediale.de/content/archive-curate-educate-active-media-arts

warning: this is a performance of material first presented with Ellef Prestsaeter at the KHIO in Oslo last fall.
The texts and images we will used come from a variety of sources, some our own reworkings, some direct quotations.
A list of sources appears at the end of this document.

from a hard drive through a filing cabinet to the book, a book worms travels through 3 archives....


http://kurenniemi.activearchives.org/logbook/
Erkki Kurenniemi has documented his life  but not archived it in any traditional sense, and didn’t develop a  systematic model for what he calls a template for all human life. In his profound techno-enthusiasm, he relies on future quantum  computers to make sense of it all. By 2048, Erkki states that the technology will be ready for the advent of this new artificial form of intelligence. The quantum computer will sort by itself the documents he  has been recording, capturing, filming, photographing, drawing, dreaming and  talking about.


Lena video
https://vimeo.com/12774628

If  “all form is a face looking at us“, what does a form  become when it is plunged into the dimension of dialogue? What is a form that is essentially relational? [...] as forms  are looking at us, how are we to look at them?

http://docs.opencv.org/2.4/modules/objdetect/doc/cascade_classification.html
The word “cascade” in the classifier name means that the resultant classifier consists of several simpler classifiers (stages)  that are applied subsequently to a region of interest until at some stage the candidate is rejected or all the stages are passed.

http://www.multimedia-computing.de/mediawiki//images/5/52/MRL-TR-May02-revised-Dec02.pdfA cascade of classifiers is a degenerated decision tree where at each stage a classifier is trained to detect almost all objects of interest (frontal faces in our example) while rejecting a certain fraction of the non-object patterns. For instance, in our case, each stage was trained to eliminate 50% of the non-face patterns while falsely eliminating only 0.1% of the frontal face patterns; 20 stages were trained. Assuming that our test set is representative for the learning task, we can expect a false alarm rate about .00000096 and a hit rate about  0.98.

Lena story
http://www.cs.cmu.edu/~chuck/lennapg/lenna.shtml
http://www.cs.cmu.edu/~chuck/lennapg/playboy_backups/lena.html
https://web.archive.org/web/20140716193150/http://www.lenna.org/full/len_full.html
"Over the years I would go into these engineering and research labs and  see these images of this woman all over the place," Seideman recalls. "I  would ask who she was and they would say, 'She's Lena.' And I would  say, 'Who's Lena?' And they'd say, 'I don't know. Just Lena.'"  
"Just Lena," Seideman soon found out, was Lena Sjööblom, Playboy's Miss  November 1972. In the early Seventies, an unknown researcher at the  University of Southern California working on compression technologies  scanned in the image of Lena's centerfold. Since that time, images of  the Playmate have been used as the industry standard for testing ways in  which pictures can be manipulated and transmitted electronically. Over  the past 25 years, no image has been more important in the history of  imaging and electronic communications, and today the mysterious Lena is  considered the First Lady of the Internet.  
"The use of her photo is clearly one of the most important events in the  history of electronic imaging," Seideman said. Image compression is  what has made the World Wide Web the wildly popular communications  medium it is today. 


Kurenniemi Face Replace
audio: http://kurenniemi.activearchives.org/spectrum/

Through the legal imperative of contacting  the involved parties in the image, the deferred presence comes back  with a vengeance. The women who implicitly agreed to be filmed  in the context of a shared moment may well refuse to allow these images to be shown publicly. They may have accepted what the photographer was doing in the present, but may refuse once the photograph becomes an image, to be held  prisoner by the reference. In practice though, will anyone ever ask them? Erkki never did. But will the institution that controls the diffusion of the archive? Or will it decide on their behalf that it is preferable to hide the images?

If perception, memory, and consciousness are all considered matters of particular rhythm and intensity, then the diary logs make evident the pivotal role of the sexual in and for Kurenniemi's particular personal rhythm, for his individual way of being.

Therefore the problem with privacy law is not that it re-opens the negotiation over the image, that it re-attaches  the images to more people than their owner or producer. The problem  comes with how it is responded to. If the response to the legal  imperative is to divide the set of documents between the “safe”  documents and the “problematic” ones, to publish the first and hide the  last, then a demarcation line is produced. This line follows the division between what is private and what is public. The exact division  that Erkki has questioned all his life. Are recorded on the same tape, in the same movement his sexual life, his meetings at work, his family  meetings, his travels, his talks and his logbook. Another response would be algorithmic and often suggested by lawyers: to blur the parts of the  images where somebody can be identified. But what can lead to identification? How far do the traces of the reference in the picture extend? And when blurring, what else do we show other than the practice of censorship? Erasing means engaging with surfaces. A white rectangle over a sensitive region is a shape and a color. Every censor is a vandal and every act of vandalism produces ornaments.

If the images cannot be ‘shown’, – and  perhaps this is a blessing rather than a tragedy – what can be shown are  the relationships between them, as they can be narrated to us by agents  to which we lend our reconfigured eyes. They can be sensed like a  pulse, experienced as time capsules. Leaving aside the “retinal”  approach to the image, we are learning from probes and experiments how the computerized visual traces of Erkki’s life let us feel temporal  intensities, carnal distances and proximities. An image is an image is an image. But an image is also an archive traversed by voluble algorithms and their nonhuman points of view. 

Source: http://kurenniemi.activearchives.org/logbook/?page_id=528


Image of Erasmus censored
http://sicv.activearchives.org/logbook/wp-content/uploads/2015/06/Erasmus_censored.png
http://sicv.activearchives.org/logbook/wp-content/uploads/2015/06/120515-Censor1Photo-hmed-0820a_files.grid-6x2.jpg
http://sicv.activearchives.org/logbook/wp-content/uploads/2015/06/ancient-censor-8.jpg
http://sicv.activearchives.org/logbook/wp-content/uploads/2015/06/ancient-censor-7.jpg

Images of declassified documents
http://sicv.activearchives.org/logbook/wp-content/uploads/2015/06/recommend-01.png
http://sicv.activearchives.org/logbook/wp-content/uploads/2015/06/recommend-02.png

A man goes to see his optician and asks for a pair of reading glasses.
The optician: I already gave you a new pair last week.
The man: I have already read them.

They Live
“The plot of They Live is deceptively simple. Roddy Piper’s character, a  semi-homeless day laborer named Nada (Spanish for “nothing”) discovers a  pair of sunglasses that allow him to identify the extraterrestrials who  have infiltrated the Earth’s populace.”
In a fashion very similar to computer vision procedures, the glasses  first discard color information. They then convert or translate images  into text. More precisely into orders. Watching the images makes you a  slave who obeys these injunctions without knowing it. Using the  glasses, you are then able to read instead of watching.  It is interesting to note that Carpenter does not choose to display the  text as a subtitle or comment to these images, but to replace them.  Using the glasses, the modes are switched. They cannot coexist. The  glasses are not only a sort of OCR for ideology, they also detect faces.  Or more precisely, reverse-detect them. When seeing an alien with the  glasses, its face becomes a dead mask, the alien looses its faciality.

Image books spines
arkiv book
Guttorm Guttormsgaard
The artist Guttorm Guttormsgaard has created over time a collection of tens of thousands of objects with the intention of  “documenting necessary impulses to keep one’s spirits up.” The archive is located in a former dairy in Blaker, a village 40 km northeast of Oslo.His archive includes a very rich collection of printed matter and printing equipment, in which the Bible coexists with the Quran, artists books with comics, propaganda flyers with hardware catalogues, notebooks with dictionaries. Guttorm scrutinises every graphical detail of his books, follows the lines of every engraving, turns their pages over and over. He admires layout and typography. But he hardly reads the texts. He loves the shape of words but he doesn't trust their meaning.

OCR
http://guttormsgaard.activearchives.org/orderings/
An OCR program typically attempts to circumscribe the zones of an image where there is text, this is the detection phase. And then tries to decipher the characters located in these zones, this is the recognition phase. The words extracted are like potatoes, they carry with them the soil that surrounded them: bits of the images, background elements attached to their contours or visible through their holes. They never are pure semantics.  The shape of a letter is rarely closed and complete. There is a porosity with the surroundings, the image corrodes the letters contours. The OCR translates what we read as words in a dyslexic alphabet. It is able to correctly extract the part of the screen containing a word, but when it is asked to spell what it has recognized, it will output sometimes argumenro or augmento when we read argumento. The OCR shows how there is always an r in every t waiting to appear, argumenro always haunts argumento. As if an unknown language was ready to surface behind every familiar word. This unkown language is not really unreadable as we can still reconstruct the words most of the time. What it becomes is unpronounceable.

video: randomwalk02
... when the alienation of human beings from their images reached critical  proportions. For this very reason, some people tried to remember the  original intention behind the images. They attempted to tear down the  screens showing the image in order to clear a path into the world behind  it. Their method was to tear the elements of the image (pixels) from  the surface and arrange them into lines: They invented linear writing.  
They thus transcoded the circular time of magic into the linear time of  history. This was the beginning of’historical consciousness’ and  ‘history’ in the narrower sense. From then on, historical consciousness  was ranged against magical consciousness.
The struggle of writing against the image runs throughout history. With writing, a new ability was  born called ‘conceptual thinking’ which consisted of abstracting lines from surfaces, i.e. producing and decoding them. Thus with the  invention of writing, human beings took one step further back from the  world. Texts do not signify the world; they signify the images they tear up. Hence, to decode texts means to discover the images signified by them.

Comparative contours http://sicv.activearchives.org/contours-view/test-plur-black-01.html
computationalvandalism

What does it mean to have these algorithms running through the pages of books, deforming words, caressing their lineaments with their itty-bitty lines?  What kind of madness is it to teach a computer to read not like a human, but like a machine learning to read?  What condition is it that a text has parallel readers, both human and machinic  – and what in a text cuts across these two scales and elicits anomalies in both of them?

It is more likely that we will find certain automata moving across and constituting certain forms of code and language.  Wriggling amongst these are algorithmic book worms.  Some of these are explicit, others we find moving across language like a deep pit of fragrant compost, churned up particle by particle, a myriad alien readers to collaborate with and be gleefully infested by.


More worms, less database


Material and inspiration from:

Ellef Prestsæter
Erkki Kurenniemi
Susanna Paasonen
Adam Harvey
Nicolas Bourriaud
Opencv documentation team
Rainer Lienhart, Alexander Kuranov, Vadim Pisarevsky
Wendy Hui Kyong Chun
Jeff Seideman
Chauncey Devega
Vilém Flusser
Matthew Fuller

-----



http://sicv.activearchives.org/mw/images/1/1a/Poctracingforprint.png
The  Scandinavian Institute of Comparative Vandalism (SICV) was founded by  Asger Jorn shortly after leaving the Situationist International in 1961.  It was the name of an association combining the forces of certain  artistic and political ideas, the French museum photographer Gerard  Franceschi and his Danish assistant Ulrik Ross, book printers and publishing houses, a Citroën, archeologists and art historians, a board  consisting of well-respected citizens from Jorn’s hometown Silkeborg, a  darkroom and an archival apparatus. For a brief period (1961-1965), this  unlikely assembly was held together and in motion by the sales of  Jorn’s paintings, which by then had started to fare well on the  international market. The output of the institute was paginated rather  than painted: for Jorn the codex was a site for the analysis, sequencing  and presentation of large quantities of heterogeneous visual materials. 


Through his “continuous collages” (the phrase belongs to Jorn’s friend  and collaborator, the archeologist P.V. Glob) Jorn wanted to trace image  migrations across space and time, including, most notably, what he  perceived as a specifically Nordic tradition going back to pre-Christian  times and significantly influencing European art writ large. Most  ambitiously, he planned the production of some thirty volumes of coffee  table books devoted to 10,000 Years of Nordic Folk Art under the auspices of the SICV

CABINET: http://sicv.activearchives.org/cabinet
The filing cabinet of the SISV contains the negatives, contact sheets  and enlargements of the photographs of the institute. Additionally one  can find the documents relative to the controversy between the  photographer Franceschi and the institute, maps, notebooks and various  documents relative to the institute. 



Vitrine
http://sicv.activearchives.org/vitrine/keeps/collage20160204_122253.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160204_130856.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160204_134743.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160204_173600.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160204_223919.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160205_151305.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160205_120138.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160205_110953.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160205_075424.gif
http://sicv.activearchives.org/vitrine/keeps/collage20160205_211409.gif
http://sicv.activearchives.org/vitrine/collage20160206_223045.gif
http://sicv.activearchives.org/vitrine/collage20160207_000537.gif

http://sicv.activearchives.org/vitrine/collage20160206_213336.gif
http://sicv.activearchives.org/vitrine/archive20160206_213336.gif





Outtakes


Associative Memory
http://kurenniemi.activearchives.org/logbook/?p=90


?? text...
 like four-eye machines made of elementary faces linked together two by two.
You don’t so much have a face as slide into one.
Scanner Darkly Mask Scene


Praat Annotations: http://activearchives.org/mw/images/9/93/Praat-annotations.png


============================================================================



Matthew Fuller: Computational Vandalism (Contour tracing)
http://sicv.activearchives.org/audio/computationalvandalism.webm


VIDEO: Scanners Part 1 07:30-08:30 (voices)

Praat Annotations: http://activearchives.org/mw/images/9/93/Praat-annotations.png
Spectrum sort (Kurenniemi) http://activearchives.org/wiki/Spectrum_sort 131:50-ish

(VIDEO: Scanners (ephemerol) FIND TIMECODE)

Image pyramid (GG)
http://guttormsgaard.activearchives.org/ephemerol/scanners.html


This movement into the image is, for me, precisely the same as travelling out into the world. Zooming and travelling are two sides of the same coin. It’s a question of seeing through the conventions, of seeing one’s own culture from the outside – or trying to go as far as possible into it.


*
VIDEO: Out of Focus
http://sicv.activearchives.org/logbook/out-of-focus/



Wolfgang: http://rwm.macba.cat/en/sonia/wolfgang_ernst/capsula   TIMECODE 01:43
*
Scanscapes (SICV)
http://sicv.activearchives.org/scanscape/README.html


*Like  the  blind, software sees while touching. And its vision is inseparable  from  movement. Considering its rhythmic dimension, scanning means more  than  digital conversion. Scanning has everything to do with slowness  and  speed, waiting and moving. The meaning of the word itself is   paradoxical: to scan   means to examine minutely and also means to look over quickly.  Scanning  is a redistribution of attention. What then if we look at the  scanner  not only as a practical digital photocopier but as a tool that  can be  extended to look at the entire archive, something that can (make  us)  scan and skim, minutely observe and look over rapidly. A scanner  gone  back to its etymological root from the Late Latin scandere   "to scan verse". As we know from poetry, verses are structures aimed  at  creating an aesthetic effect, but also to create a mnemonic one.  What  then is the scanner marking off? What are those mnemonic  structures  sensed by the comb of light-emitting fingers?


Blind Annotation: http://activearchives.org/wiki/Blind_annotation
for example.... http://kurenniemi.activearchives.org/aa/annotate/407/
*
*With privacy law, copyright, one can only look at an archive  blindfolded. Before tedious negotiations take place, images, sounds,  videos of the archive cannot be published. They still can remain in the  archive though.
*
*Can we therefore access the archive blindfolded?
*
*Being blindfolded implies the suppression of sight, but also implies a  concentration on other senses. Or implies a remapping of sight. Our  eyes are delegated to our hands, nose, to the epiderm and all these  informations combined produce a global image, a mental image.
*Using computer vision algorithms, we are seeing without eyes,  computer vision hides and reveals, hides and concentrates. A Braille  for the legal blind spots.


DATA GALLERY http://www.kurenniemi.constantvzw.org/db/records/images/view/3148

*Imagine a picture.
*An horizontal picture of 2592 pixels wide and of 1944 pixels height.
*Its print size is 36×27 inches.
*The picture was taken on the 06th of November 2004 at 21h56:37. The  document set contains 45732 pictures by Erkki Kurenniemi for the year  2004. Erkki took 223 pictures in 2004 between the hours of 9 and 10pm.  Of the 45732 pictures present in the dataset, Erkki took 33712 at night.

*In the folder where this file is located, there are 28 other pictures.  They have been taken between 21h56:32 and the next day at 19h21:18. The  folder Harrin bileet can be seen as a sequence of 21 hours 24 minutes 46  seconds of the life of a man of 63 years and 4 months at the date the  picture was taken.
*It took 10 of a 400th second for the camera to take the picture.
*The blink of an eye.
*
*[...]
*The level of skin colors is low, there is probably no nudity in the image. Or at least, nudity is not the main subject of the picture. In opposition to 63 percent of the pictures that contain a significant amount of skin colors. Two photographs in the same folder contain a large percentage of skin colors in their center. They are taken at 23h41 and 23h42, one hour and 45 minutes after the picture.
*There are 172 corners present. This level indicates that the picture may have been taken in a built environment. There are 3 faces positioned at (xx,xx), (xx,xx) and (xx,xx). They occupy a portion of xx% of the image. One at the lower left corner and two in the upper half of the image. The distance between the tree rectangles is in average xxx. The people are close without touching each other. The proximity of the photographer and the position of the face in other pictures prove that the photographer moves easily among the circle of people photographed.
*It is dark now. It is the end of an evening of November 2004. And the face in the center of the image receives more light than the one next to it at the right side. Two hands are close to the second face poorly lit. An image begins to form in the back of your mind, and we have only started counting.

*http://kurenniemi.activearchives.org/logbook/?page_id=521


(LATER: ?? Kurenniemi diary... )

GG: Orderings walk http://guttormsgaard.activearchives.org/orderings/walk.html

An interest in dispersed attention, such as the one modern art has shown throughout this century, may contribute to the deterioration of culture’s authoritarian patent makers, and surely it is this practice you would call comparative vandalism.
Détournement is a game born out of the capacity for devalorization. Only he who is able to devalorize can create new values. And only there where there is something to devalorize, that is, an already established value, can one engage in devalorization. It is up to us to devalorize or to be devalorized according to our ability to reinvest in our own culture.

Consider what happens when you run an image analysis procedure such as an edge detection or a (SIFT*) feature extractor, and then use a conventional means of visualizing the results drawnon top of the original. The result is an image dense with markings, edges of figures traced with the brutal pixelized edge of a computed curve or dotted with swarms of circular markers of various sizes and positions throughout the image at various points of (algorithmically determined) interest. The marks typically number from tens to hundreds. Visually, the language of the algorithmically annotated image resonates with the vandalized image, the political poster with sprayed-on beard or mustache, the marketing billboard covered with elements crossed out or covered by graffiti tags. There is an essential difference however: where the billboard vandal challenges the authority of the image (both of those depicted and the forces that arrange to place them in public), the marks of the algorithm carry themselves an authority borne from the often impenetrable layers of technique and software employed. On top of this, such techniques may well be further allied to authority by patents, software licenses, and other aspects of law. Furthermore, the images most frequently subjected to the algorithmic processes of analysis are typically banal ones, themselves collected by agents of authority and typically recorded with their subjects (relatively) unaware of their being taken. For instance, the closed circuit surveillance system in a shopping center or laundromat, or the passport control agent’s webcam.

Computational vandalism aims to challenge the authority of the algorithm and uses computation both as a means and a subject of critical investigation. It aims to expose the values enshrined in algorithms, to consider alternatives, and to feed them back upon themselves to make these processes more apparent in their operation, as well as to activate them as tools to investigate other kinds of images and contexts from their conventional uses.



Histograms: http://sicv.activearchives.org/histograms/ratios-colored.php
(showing histograms --- select an interval and click SWITCH)
*
* You are right to underline the authority of the algorithm. It is  nevertheless interesting to see that this authority is granted under a  very specific condition. The algorithm must in the end confirm human  perception. I always see a visual algorithm as a sort of Hercule Poirot  on a crime scene. Where his assistant Captain Hastings will immediately  decode the semantic layer, the obvious clues, the motives, Poirot always  seems concerned with absurdly pointless details: a slight change of  color in a carpet, the position of a finger, etc.
*
Feature browser (SICV)
http://sicv.activearchives.org/features/04_sift_features.html

*Poirot seems to embody  the coolness of media archeology, indifferent to content. Let’s think  of the SIFT algorithm, how it looks into extremely narrow fragments of  an image to find patterns that could identify an image even when it is  rotated. When we look at the regions of the images selected by the  algorithm, they seem completely secondary to the human observer. But, in  the end, if we make a search for similar images, the results returned  by this algorithm seem to match very well what a human would have picked  up. In every Hercule Poirot ending, the tour de force is to make a  match between all these apparently absurd little clues and a narrative  where the murderer is punished and the social order is confirmed.


http://sicv.activearchives.org/features/05_collage.html

*In  every operation of computer vision, there is this expectation that the  gap between human perception and computer perception will be filled. But  what if it isn’t? What if it reveals that there is a zone of  intersection between human perception and computer vision, and that the  contours of this zone are problematic, fluctuating? Computational  vandalism is what introduces us to a world with a proliferation of  powerfully meaningless little clues that do not point towards any  murderer. Nobody is punished in the world of computational vandalists,  but its visual structure is denser. Computational vandalism is Agatha Christie smoking Asger Jorn’s cigar.


Video: UCF Bag of words "We don't have a vocabulary for visual images" TIMECODE

http://sicv.activearchives.org/logbook/the-past-is-woven-into-it/
http://kurenniemi.activearchives.org/video/thumbs.webm

“A photograph is a frozen moment, outside time. As Wittgenstein says it  is ‘a probability’, not ‘all probabilities’, what one sees in the blink  of an eye. But if you keep your eyes open you will see things move and  change, nature as a dynamic event, and it is this constant changing that  creates fuzziness on one hand but clarity on the other, because if you  only glimpse then you exclude all other aspects, you have no greater  clarity, you are blinkered.”
http://sicv.activearchives.org/logbook/francis-galtons-composite-portraiture-meets-wittgensteins-camera/

[…] A while ago I met an extremely interesting developer in Holland.  He was working on smart phone camera technology. A representational mode  of thinking photography is: there is something out there and it will be  represented by means of optical technology ideally via indexical link.  But the technology for the phone camera is quite different. As the  lenses are tiny and basically crap, about half of the data captured by  the sensor are noise. The trick is to create the algorithm to clean the  picture from the noise, or rather to define the picture from within  noise. But how does the camera know this? Very simple. It scans all  other pictures stored on the phone or on your social media networks and  sifts through your contacts. It looks through the pictures you already  made, or those that are networked to you and tries to match faces and  shapes. In short: it creates the picture based on earlier pictures, on  your/its memory. It does not only know what you saw but also what you  might like to see based on your previous choices. In other words, it  speculates on your preferences and offers an interpretation of data  based on affinities to other data. The link to the thing in front of the  lens is still there, but there are also links to past pictures that  help create the picture. You don’t really photograph the present, as the past is woven into it.

The result might be a picture that never existed in reality, but that  the phone thinks you might like to see. It is a bet, a gamble, some  combination between repeating those things you have already seen and  coming up with new versions of these, a mixture of conservatism and  fabulation. The paradigm of representation stands to the present  condition as traditional lens-based photography does to an algorithmic,  networked photography that works with probabilities and bets on inertia.  Consequently, it makes seeing unforeseen things more difficult. The  noise will increase and random interpretation too. We might think that  the phone sees what we want, but actually we will see what the phone  thinks it knows about us. A complicated relationship — like a very  neurotic marriage.






end credtis
Matthew Fuller
Asger Jorn
David Cronenberg
Hito Steyerl
Guttorm Guttormsgaard
Woody Allen
Robin Williams
Wolfgang Ernst
and other members of the SICV
 All reproduction, deformation, modification, derivation and transformation is permitted.


outtakes
Associative Memory