Welcome to Constant Etherpad!
These pads are archived each night (around 4AM CET) @
http://etherdump.constantvzw.org/
An RSS feed from the etherdump also appears on
http://constantvzw.org/
To prevent your public pad from appearing in the archive and RSS feed, put or just leave the following (including the surrounding double underscores) anywhere in the text of your pad:
__NOPUBLISH__
Changes will be reflected after the next update at 4AM.
Notes for a conversation with Nicolas
http://sicv.activearchives.org/logbook/
http://sicv.activearchives.org/share/ways_of_seeing/Composed/yolox2-composed-episode-3-1-01.mp4
http://activearchives.org/wiki/Machine_Seeing_Ways_of_Seeing
https://unthinking.photography/themes/machine-vision/the-cat-sits-on-the-bed-pedagogies-of-vision-in-human-and-machine-learning
- We are generally interested in the work of contours, dissection, segmentation and boundary making in volumetric imaging. First of all we are wondering why you started to be interested in contours? What is/was the trigger to look in that specific direction and/or to that specific practice of contouring?
- We talked a bit about Slicer on the phone, and it continues to be an interesting software for us to look at. Here are some notes taken during the last weeks:
http://pad.constantvzw.org/p/possiblebodies.slicer
For example, Slicer apparently begun at the MIT Artificial Intelligence Laboratory
http://www.csail.mit.edu/
and the Surgical Planning Laboratory, at The Brigham and Women's Hospital, Harvard medical school
https://www.slicer.org/wiki/Slicer3:Acknowledgements
-> we would like to check with you on your reading of the connections between the MIT AI lab, and the history of Computer Vision?
- We wonder about the operations of labeling, naming, ontologies (at this moment, a lot through the lense of pathalogical anatomy) in relation to segmentation or how ordering and boundary-making are connected (in digital practices).
- In general, if you have any understanding of the workings, histories and cultures of 'segmentation' algorithms such as GrowCut, watershed methods and Marching Cubes that all seem to combine 2D-to-3D metaphors and techniques? If you have any thoughts on this, we would be very happy to speak about this.
- It seems that biomedical imaging industries are making more and more use of 'machine learning', and much of the learnings are (still) from the Visible Human Project but also from living patients. But also in other areas, machines are learning about body (images): thinking of Michael's sicv post on (Not) Safe For Work for example.
How do you sense that machine learning is learning about bodies, and how does it produce body imaginaries and/or more direct bodily affections? Or to paraphrase your own words, how can we think productively about the fact that a generation of humans and algorithms are learning together to look at bodies? ;)
- Anything else that we did not manage to grasp but that you would be interested in speaking about with us?
-----------------------
From separating background and foreground to segmentation --
"Background and foreground, scraping and structured data. Computer Vision algorithms employed as interlocutors, to explore alternative interpretations, different orderings and seeing through other eyes of digital and digitised collections."
http://diversions.constantvzw.org/etherdump/erasing_the_background.diff.html
http://docs.opencv.org/trunk/d1/dc5/tutorial_background_subtraction.html
In medical imaging: defining organ boundaries, visually discerning anatomical elements or sometimes anomalies.
"if you don't have this reference you have to make the algorithm learn what counts as a difference: have a large amount of footage that teaches the algorithm what makes a difference" (from: diversions notes)
Specific histories/practices of those techniques in
CV (
C
omputer
V
ision)
? --> GrowCut algorithm, watershed algorithm,
Trying to understand
https://en.wikipedia.org/wiki/Marching_cubes
and relation to contour, segmentation
Possible links to anatomy (animal dissection?)
on machine learning:
---
Conversation 24/07/2017
NM:
relationship to computer vision -- what's the model of vision that is used in CV. How do they come to talk about
vision
.
reading through the history of how in computer science vision is being studied
came to an experiment in CalTech in 2007, where they show images to people for less than 400ms and ask them a description of what they saw and actually in 27ms many things happen like going from vision of abstract shapes and forms to very detailed understanding of nr ofunits, specific objects, etc. they have a theory that vision is hierarchical: temporary goes through a taxonomy -- interested not only in the theory but also in the way they did the experiment.
people are asked to try machines and explain
FS are these images always "photographic" images? --
NM -
Yes, surprisingly -- this is not seen as an issue. I
n psychology this depth is not so much of a problem, but flat images stand for seeing -- people were shown not only
photographic
images but mostly drawings. -- from the internet, somehow more "real" --
in 2007 they said those images were more real than others -- never use "reality" as a word.
FS so your looking at these ...
NM
they considered that
if you look at image
s
com
i
ng from a search engine you are looking at image
s that are
un-
biased by the average
(...)
NM sift features
https://es.wikipedia.org/wiki/Scale-invariant_feature_transform
// by patterns, but babies don't know how to choose, so (...)
there is something very well equip
p
ed in human brain that make it understand in terms of lines.--
the "vision community" claim there is a part of vision called "early vision" -> what you see, if you're exposed to visual stimulus, this part of vision is said not to be accessible by consciousness
[related to gestalt?]
JR: is this
related to subliminal?
->
well, this is not unconscious, but non conscious. Pre-conscious.
this is the blackboxofvision -- what computer science is trying to model for computer vision: give the same tool as this part of human vision.
experiment of the frog that they exposed to all kinds of equipment with electrical signal
s
and showed that a frog is able to detect a fly before the info passes
to the brain
cognition without consciousness
so first you need to anchor contours
very interesting things by Katherine Hayles -- interest on the non-conscious.
If you look at this moment of 'vision', the
complexity of what it is to "see" is
red
u
ced, but on the other hand
it
is nice
to look into vision outside of the paradigm of consciousness, where many other ways of being in the world and looking at it can accessed.
A possiblility for imaging through the eyes of other species.
if you talk about contours in terms of computation it'll be important to show where, contextualize computer vision.
FS we wanted to talk to you about contours. It came because somehow it seemed that the hightened attention in CV for contour
that
meets the h
e
ightened attention for segmentation and dissection in anatomy.
in the clinical situation, where the work of drawing boundaries around organs or tissues or anomalies -separate units- seems to happen in a blurry bunch of zones
it was extreme to see how the separation of organs is a collaboration between algorithms and anatomical projections.
jump from non-conscious vision, but
wondering ...
NM how you can go from the interpretation of the pattern detector to a sign or a meaning. where the pattern detector says there is a probability and this is matching the tumor or x, it does not tell you there
is a tumor. and then how the tumor has been correlated with a sufficient numer of patterns.
if
you do pattern detection you can do (...)
You can
craft your
own model in pattern detection. in computer sciences you can. and if that work is narrow enough... but when it has different variations t becomes incredibly complex.
then you have
machine learning
.
Here
the modelling does
not come from computer science but from who did the training of the software.
number of segments is what is known
the accountability
/responsibility
is very hard to place
/locate
in machine learning they were not using this to detect tumors or
for
situation
s where
people can die, but
they'd build trainingsets through
amazon's
mechanical turk -- when applying it in medicine you will want to include a very well trained
team to develop the training set
.
FS
At the hospital we visited,
it seem
ed
there's 24h of
imaging happening. But they seem to be very clear that in the backup, or tr
e
ating as accumulative, the kept using the patient as the comparison field.
the folders are organized by patient. it's in research where
inter-person (and inter-species?)
comparison happens.
NM i've been looking at contours from the Kurenniemi project. That was my first -- to not show image due to privacy issues etc, so looked at descriptions and extracted features, algorithmicall
y.
also wanted to work with image materials from pdf where there were drawings and text. That was the first contact with contours: asking how many horizontal lines etc.
FS so a
big amount of horizontal lines would be a text
NM
yes
& Hough lines -- a pattern algorithm // estimate wether points or pixels where a line is a continous between different points. Most of the times lines are understood as
continuous
contrast
s
-- so you look for the most continous series of points.
In
drawn
lines
that works
, but at the level of pictures you might have gaps.
histericity??¿?
perception and muscles when tracing a line, when there is anticipation and also constrain. So of you look at a series of pixels, the comp
uter
cannot preview any constr
a
int, so you need to introduce a constraint.
a restriction of the movement of the line.
[freedom of movement/channels in 3D/robotics?]
Norbert Wiener was working with the american army doing the systems to defence against airplanes attacks -- they had this problem of finding the probability for when they shot a target, what'll be the next attack. So the moment they shoot, and the time will vary..
not to train trajectory, but to narrow very much the possible next step by calculating (curve?)
finding these techniques and taking into account the trajectory of the airplane and also by shooting, they discovered
also
the feedback loop.
anything that happens in the environment you need to include in the loop, as it informs it.
for the tracing of the line, when the trajectory of the plane is always related to what it previously did -- this
is the concept of histe
r
icity:
when you do something to a material, it will keep to a trace of this action. A certain persistance. Not knows what will happen next but is also a temporary memory ofapressure you exerced to the material.
so the algorithm of this line also has some of this temporal issue.
the next point that will be added to the line will get more and more (...)
FS if we talk in terms of possible and probable...
NM
yes, it's probability. because it
is a discrete
(...)
so continuity is not part of it. Everything is central. It is side by side.
FS what would be continuity outside of the discrete
NM when you do imaging you have a bitmap, but nothing tells you of the pages or
one pixel and the other are
continuous
.
FS i'm trying to transpose this to the troubles i had with the instantaneous generation of continuo
u
s-whole-3D model from a block of slices
(at the hospital)
, which is in a way planes but lines in space.
...you talk about the beauty of the line looking forward and back. But i
n 3D
I
can only think of uninteresting probabilities.
NM lines is really quite related to the idea of making a sort of translation of something that
h
appens in the world to a 2D space.
but i've got the impression that in your project there is something about 3D. Lines are
not there to separate 2 surfaces, but also defining/connecting a 3D model.
JR --
something on [registration -- intra-calibration or intra-comparing]
(...)
NM when this process of calibration happens, the program that makes it has a sort of what a human body in 3D might be?
FS we are not entirely sure
NM what i discovered when looking at how the lines are being traced, you need to (...)¿??
FS from looking at slicer and the VHP i suspect that the VHP is the
reference.
NM it's different if you have an expectation
for what direction a line is going,
than
drawing
a line in the wild
FS everytime we ask
people if there are any set models,
the answer is
a no, but it does not seem you can import different models -- or what happens when you look
at
a horse
through a MRI?
JR to put it more
politically; how to point at the oneness of humanness in its finitude / humanness
that is
embe
d
ded in the algorithm.
FS also because there's always only one person in the MRI tube, but when you ask if there could be more, then they come up with jokes. Also the folders of images are ordered person by person
FS we
would like to
sit with one or two peopl
e, to really ask.
NM so...who to ask? :P
FS it's clear that research is very much tainted
by
the
VHP
dataset
NM for instance the question of whole
... It might be you
never train anything youself at a clinical level, never
actually feel like you are feeding
a machine learning system
but still you are part of the educating process, if only by using and validating the datasets.
[a missing conversation on ...
temporality]
Contact at some point curator Katrina Sluis at the
photographers gallery
(Digital Programme) +
https://unthinking.photography