Notes for a conversation with Nicolas

http://sicv.activearchives.org/logbook/
http://sicv.activearchives.org/share/ways_of_seeing/Composed/yolox2-composed-episode-3-1-01.mp4
http://activearchives.org/wiki/Machine_Seeing_Ways_of_Seeing
https://unthinking.photography/themes/machine-vision/the-cat-sits-on-the-bed-pedagogies-of-vision-in-human-and-machine-learning

- We are generally interested in the work of contours, dissection, segmentation and boundary making in volumetric imaging. First of all we are wondering why you started to be interested in contours? What is/was the trigger to look in that specific direction and/or to that specific practice of contouring?
- We talked a bit about Slicer on the phone, and it continues to be an interesting software for us to look at. Here are some notes taken during the last weeks: http://pad.constantvzw.org/p/possiblebodies.slicer For example, Slicer apparently begun at the MIT Artificial Intelligence Laboratory http://www.csail.mit.edu/ and the Surgical Planning Laboratory, at The Brigham and Women's Hospital, Harvard medical school https://www.slicer.org/wiki/Slicer3:Acknowledgements -> we would like to check with you on your reading of the connections between the MIT AI lab, and the history of Computer Vision?
- We wonder about the operations of labeling, naming, ontologies (at this moment, a lot through the lense of pathalogical anatomy) in relation to segmentation or how ordering and boundary-making are connected (in digital practices).
- In general, if you have any understanding of the workings, histories and cultures of 'segmentation' algorithms such as GrowCut, watershed methods and Marching Cubes that all seem to combine 2D-to-3D metaphors and techniques? If you have any thoughts on this, we would be very happy to speak about this.
- It seems that biomedical imaging industries are making more and more use of 'machine learning', and much of the learnings are (still) from the Visible Human Project but also from living patients. But also in other areas, machines are learning about body (images): thinking of Michael's sicv post on (Not) Safe For Work for example.
How do you sense that machine learning is learning about bodies, and how does it produce body imaginaries and/or more direct bodily affections? Or to paraphrase your own words, how can we think productively about the fact that a generation of humans and algorithms are learning together to look at bodies? ;)
- Anything else that we did not manage to grasp but that you would be interested in speaking about with us?

-----------------------

From separating background and foreground to segmentation -- 
"Background and foreground, scraping and structured data. Computer Vision algorithms employed as interlocutors, to explore alternative interpretations, different orderings and seeing through other eyes of digital and digitised collections."
http://diversions.constantvzw.org/etherdump/erasing_the_background.diff.html
http://docs.opencv.org/trunk/d1/dc5/tutorial_background_subtraction.html

In medical imaging: defining organ boundaries, visually discerning anatomical elements or sometimes anomalies.
"if you don't have this reference you have to make the algorithm learn what counts as a difference: have a large amount of footage that teaches the algorithm  what makes a difference" (from: diversions notes)

Specific histories/practices of those techniques in CV (Computer Vision)? --> GrowCut algorithm, watershed algorithm, 
Trying to understand https://en.wikipedia.org/wiki/Marching_cubes and relation to contour, segmentation

Possible links to anatomy (animal dissection?)

on machine learning: 

---

Conversation 24/07/2017

NM: relationship to computer vision -- what's the model of vision that is used in CV. How do they come to talk about vision. 
reading through the history of how in computer science vision is being studied
came to an experiment in CalTech in 2007, where they show images to people for less than 400ms and ask them a description of what they saw and actually in 27ms many things happen like going from vision of abstract shapes and forms to very detailed understanding of nr ofunits, specific objects, etc. they have a theory that vision is hierarchical: temporary goes through a taxonomy -- interested not only in the theory but also in the way they did the experiment. 
people are asked to try machines and explain

FS are these images always "photographic" images? -- 

NM - Yes, surprisingly -- this is not seen as an issue. In psychology this depth is not so much of a problem, but flat images stand for seeing -- people were shown not only photographic images but mostly drawings. -- from the internet, somehow more "real" -- 
in 2007 they said those images were more real than others -- never use "reality" as a word.

FS so your looking at these ...
NM they considered that if you look at images coming from a search engine you are looking at images that are un-biased by the average
(...)
NM sift features https://es.wikipedia.org/wiki/Scale-invariant_feature_transform // by patterns, but babies don't know how to choose, so (...)
there is something very well equipped in human brain that make it understand in terms of lines.-- 
the "vision community" claim there is a part of vision called "early vision" -> what you see, if you're exposed to visual stimulus, this part of vision is said not to be accessible by consciousness
[related to gestalt?]
JR: is this related to subliminal? -> well, this is not unconscious, but non conscious. Pre-conscious.
this is the blackboxofvision -- what computer science is trying to model for computer vision: give the same tool as this part of human vision. 
experiment of the frog that they exposed to all kinds of equipment with electrical signals and showed that a frog is able to detect a fly before the info passes to the brain
cognition without consciousness
so first you need to anchor contours
very interesting things by Katherine Hayles -- interest on the non-conscious. If you look at this moment of 'vision', the complexity of what it is to "see" is reduced, but on the other hand it is nice to look into vision outside of the paradigm of consciousness, where many other ways of being in the world and looking at it can accessed. A possiblility for imaging through the eyes of other species.
if you talk about contours in terms of computation it'll be important to show where, contextualize computer vision.

FS we wanted to talk to you about contours. It came because somehow it seemed that the hightened attention in CV for contour that meets the heightened attention for segmentation and dissection in anatomy.
in the clinical situation, where the work of drawing boundaries around organs or tissues or anomalies -separate units- seems to happen in a blurry bunch of zones
it was extreme to see how the separation of organs is a collaboration between algorithms and anatomical projections. 
jump from non-conscious vision, but
wondering ...

NM how you can go from the interpretation of the pattern detector to a sign or a meaning. where the pattern detector says there is a probability and this is matching the tumor or x, it does not tell you there is a tumor. and then how the tumor has been correlated with a sufficient numer of patterns.
if you do pattern detection you can do (...)

You can craft your own model in pattern detection. in computer sciences you can. and if that work is narrow enough... but when it has different variations t becomes incredibly complex. 
then you have machine learning. Here the modelling does not come from computer science but from who did the training of the software.

number of segments is what is known

the accountability/responsibility is very hard to place/locate

in machine learning they were not using this to detect tumors or for situations where people can die, but they'd build trainingsets through amazon's mechanical turk -- when applying it in medicine you will want to include a very well trained team to develop the training set.

FS At the hospital we visited, it seemed there's 24h of imaging happening. But they seem to be very clear that in the backup, or treating as accumulative, the kept using the patient as the comparison field. the folders are organized by patient. it's in research where inter-person (and inter-species?) comparison happens.

NM i've been looking at contours from the Kurenniemi project. That was my first -- to not show image due to privacy issues etc, so looked at descriptions and extracted features, algorithmically.
also wanted to work with image materials from pdf where there were drawings and text. That was the first contact with contours: asking how many horizontal lines etc. 

FS so a big amount of horizontal lines would be a text

NM
yes

& Hough lines -- a pattern algorithm // estimate wether points or pixels where a line is a continous between different points. Most of the times lines are understood as continuous contrasts -- so you look for the most continous series of points. In drawn lines that works, but at the level of pictures you might have gaps. 

histericity??¿?

perception and muscles when tracing a line, when there is anticipation and also constrain. So of you look at a series of pixels, the computer cannot preview any constraint, so you need to introduce a constraint. a restriction of the movement of the line. [freedom of movement/channels in 3D/robotics?]

Norbert Wiener was working with the american army doing the systems to defence against airplanes attacks -- they had this problem of finding the probability for when they shot a target, what'll be the next attack. So the moment they shoot, and the time will vary.. 

not to train trajectory, but to narrow very much the possible next step by calculating (curve?)
finding these techniques and taking into account the trajectory of the airplane and also by shooting, they discovered also the feedback loop.
anything that happens in the environment you need to include in the loop, as it informs it.

for the tracing of the line, when the trajectory of the plane is always related to what it previously did -- this is the concept of histericity: when you do something to a material, it will keep to a trace of this action. A certain persistance. Not knows what will happen next but is also a temporary memory ofapressure you exerced to the material.

so the algorithm of this line also has some of this temporal issue.
the next point that will be added to the line will get more and more (...)

FS if we talk in terms of possible and probable...

NM yes, it's probability. because it is a discrete (...) so continuity is not part of it. Everything is central. It is side by side. 

FS what would be continuity outside of the discrete

NM when you do imaging you have a bitmap, but nothing tells you of the pages or one pixel and the other are continuous. 

FS i'm trying to transpose this to the troubles i had with the instantaneous generation of continuous-whole-3D model from a block of slices (at the hospital), which is in a way planes but lines in space.

...you talk about the beauty of the line looking forward and back. But in 3D I can only think of uninteresting probabilities.

NM lines is really quite related to the idea of making a sort of translation of something that happens in the world to a 2D space. 
but i've got the impression that in your project there is something about 3D. Lines are not there to separate 2 surfaces, but also defining/connecting a 3D model.

JR -- something on [registration -- intra-calibration or intra-comparing] 
(...)

NM when this process of calibration happens, the program that makes it has a sort of what a human body in 3D might be?

FS we are not entirely sure

NM what i discovered when looking at how the lines are being traced, you need to (...)¿??

FS from looking at slicer and the VHP i suspect that the VHP is the reference.

NM it's different if you have an expectation for what direction a line is going, than drawing a line in the wild

FS everytime we ask people if there are any set models, the answer is a no, but it does not seem you can import different models -- or what happens when you look at a horse through a MRI?

JR to put it more politically; how to point at the oneness of humanness in its finitude / humanness that is embedded in the algorithm.

FS also because there's always only one person in the MRI tube, but when you ask if there could be more, then they come up with jokes. Also the folders of images are ordered person by person

FS we would like to sit with one or two people, to really ask.

NM so...who to ask? :P

FS it's clear that research is very much tainted by the VHP dataset

NM for instance the question of whole ... It might be you never train anything youself at a clinical level, never actually feel like you are feeding a machine learning system but still you are part of the educating process, if only by using and validating the datasets.

[a missing conversation on ... temporality]

Contact at some point curator Katrina Sluis at the photographers gallery (Digital Programme) + https://unthinking.photography