JR: 'triangulating software with theory'

3D Slicer http://slicer.org

Mission
"3D Slicer is an open source software platform for medical image informatics, image processing, and three-dimensional  visualization."

"3D Slicer ("Slicer") is an open source, extensible platform for image visualization and analysis. Slicer  has a large community of users in medical imaging and surgical navigation, and is also used in fields such as astronomy, paleontology, and 3D printing."

"Support for multi-modality imaging including, MRI, CT, US, nuclear medicine, and microscopy."

"3D Slicer is a free open source software application for medical image  computing. As a clinical research tool, 3D Slicer is similar to a  radiology workstation that supports versatile visualizations but also  provides advanced functionality such as automated segmentation and  registration for a variety of application domains. Unlike a typical radiology workstation, 3D Slicer is free and is not tied to specific hardware. As a programming platform, 3D Slicer facilitates translation  and evaluation of the new quantitative methods by allowing the  biomedical researcher to focus on the implementation of the algorithm, and providing abstractions for the common tasks of data communication,  visualization and user interface development. Compared to other tools  that provide aspects of this functionality, 3D Slicer is fully open source and can be readily extended and redistributed. In addition, 3D  Slicer is designed to facilitate the development of new functionality in  the form of 3D Slicer extensions."
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3466397/

from the about: "Slicer is NOT an FDA approved medical device."

License
BSD license, but specific for 3D? (Medical use?)
"maintained by l, Inc. ("Brigham")" ?? CHECK
"4. The Software has been designed for research purposes only and has not been reviewed or approved by the Food and Drug Administration or by any other agency. YOU ACKNOWLEDGE AND AGREE THAT CLINICAL APPLICATIONS ARE NEITHER RECOMMENDED NOR ADVISED. Any commercialization of the Software is at the sole risk of the party or parties engaged in such commercialization. You further agree to use, reproduce, make derivative works of, display and distribute the Software in compliance with all applicable governmental laws, regulations and orders, including without limitation those relating to export and import control."
https://github.com/Slicer/Slicer/blob/master/License.txt

"Slicer License"

Code
github repo
QT (ref. Blender?), Python, C++
active
loads of grants for the development: Cancer, Prostrate, Brain tumors, Orthognathic Surgery, Liver, Neurosurgery (money coming in until 2020)
Possibility to use OpenCV (plugin)


Research vs clinical use, observing vs. intervention

"Slicer continues to be a research package and is not intended for clinical use":
clinical vs. research, preparing interventions though

The presence of 'sculpting' in these softwares. it seemed, as they somehow hover between 'creative' tools and 'research', 'discovery'

Slicer/segmentation is tied into the idea of 'robot surgery' ('surgical navigation') in many ways, to begin with its history in the MIT AI lab.
Surgical navigation belongs to the field of computer assisted surgery https://en.wikipedia.org/wiki/Computer-assisted_surgery

Principle 1: "Creating a virtual image of the patient". Method: segmentation.
https://upload.wikimedia.org/wikipedia/commons/thumb/3/37/StereolithographiemodellSchaedel.jpg/220px-StereolithographiemodellSchaedel.jpg
Bone segment navigation: https://en.wikipedia.org/wiki/Bone_segment_navigation

JR: 'This reminds me of Haraway's speech on her jaws in the documentary, when she explains how she discovered in a conversation with her ortodonthist that orthodoncy is actually based on the sculptural greek canon, but not "fleshy" anatomy even!'

See "Using 3d modeling techniques to enhance teaching of difficult anatomical concepts": interacting with the models is more effective when it has an effect, ie. intervene into the models. These models are not just readable, but also writable (Waldy)?


Datasets

Comes with loads of sample data, MR-scans and ultrasound (all 2D) 
The datasets so far are all human: heads, torso, braintumor ... some of the main tutorials are done by a Brain Imaging Scientist
super anatomical canon

Data acquisition: Check how to work with Ultrasound? https://www.slicer.org/wiki/Slicer3:4DUltrasound_4D_US

Licenses? Anonymity?
Some included.
Other datasets around?
Paleontology: http://www.ohio.edu/people/witmerl/lab.htm

Collage/assemblage?

Atlases
The alphabet industries of course: https://en.wikipedia.org/wiki/ZygoteBody + https://zygotebody.com/
Google health: https://en.wikipedia.org/wiki/Google_Health


Slicer + segmentation/dissection

'segmentation' seems to be a big thing in these applications
If this segmentation work that seems very central to the imaging, if it is just because this software can, or that they want it from the software if you get what i mean. Need to look at the history of that practice.

Segmentation vs. discretisation?

"The term “cut” is a bit of a misnomer,  yet it is used to describe the process of grinding away the top surface  of a specimen at regular intervals. The term “slice,” also a misnomer,  refers to the revealed surface of the specimen to be photographed; the  process of grinding the surface away is entirely destructive to the  specimen and leaves no usable or preservable “slice” of the cadaver."
"NLM (National Library of Medicine) itself has started an open source project, the Insight Toolkit, whose aim is to automatically deduce organ boundaries from the data."
https://en.wikipedia.org/wiki/Visible_Human_Project

"Welcome to the National Library of Medicine Insight Segmentation and Registration Toolkit (ITK). ITK is an open-source sofware system to  support the Visible Human Project. Currently under active development,  ITK employs leading-edge segmentation and registration algorithms in  two, three and more dimensions." https://itk.org/Insight/Doxygen/html/index.html

"The goals for ITK include: Supporting the Visible Human Project." https://itk.org/itkindex.html

Segmentation is the process of identifying and classifying data found in a digitally sampled representation. Typically the sampled representation is an image acquired from such medical instrumentation as CT, MRI or ultrasound scanners.

Understand how Radon transform makes segmentation/watershedding lowhanging fruit

Software constellations + histories

Project started in 1998 ... so 20 years old!
Slicer begun at the MIT Artificial Intelligence Laboratory http://www.csail.mit.edu/ and the Surgical Planning Laboratory, at The Brigham and Women's Hospital, Harvard medical school https://www.slicer.org/wiki/Slicer3:Acknowledgements (check Nicolas for MIT history?)

1997: Slicer started as a research project between the Surgical Planning Lab (Harvard) and the CSAIL (MIT)  https://www.slicer.org/w/images/5/51/3DDataLoadingandVisualization_Slicer4.5_SoniaPujol.pdf

Development: multi-institutional!  

ITK

ITK-SNAP
https://www.youtube.com/watch?v=Fe1zyS2DC4k
From 2d to 3d
segmentation (visually discerning anatomical elements)
'growing' the placenta
post processing ... cutting off pieces with the sculpting tool
'looking for places where the segmentation accidentally leaked'
dedicated to segmentation as a task

MITK
http://mitk.org/wiki/MITK
diffusion imaging

3D-printing
3d printing needs the opposite move, to go from solid object to slices.
slic3r http://slic3r.org/ --> preparing 3D-printing. 2000 reprep
'slicing' is also a process used to prepare for 3d printing so by nature, this software is ready for that?
g code -- contains also info on speed, heat, ...

The Visible Human Project

How exactly does the software grow with/from this?
-- ITK?

Catherine Waldby: The Visible Human Project: Informatic Bodies and Posthuman Medicine http://pad.constantvzw.org/p/possiblebodies.informaticbodies



Communities of use

Used in mixed environments, ie medical professionals AND amateur. -- ref Waldy

A community of people that like to 3D print bones and skulls etc.

Motivation: Getting a sense of your own body, where things are https://en.wikipedia.org/wiki/Proprioception (body schema, tool use?)

Described as a brain-imaging tool, used for astronomy: https://arxiv.org/pdf/astro-ph/0506604.pdf
The Open Source Paleontologist (3D printing a dinosaur head from mri scans) http://openpaleo.blogspot.be/2008/12/3d-slicer-tutorial-part-iv.html (it seems to be the only one)
Clinical use for animals ok? https://sonovol.com/ -> based on Slicer
"We have applied 3D Slicer to research and commercial applications  ranging from pre-clinical animal studies, to surgical planning and  guidance, to medical robot control, to population studies." https://www.kitware.com/platforms/#3d-slicer


Machine Learning, Computer vision

Seems mainly in the way algorithms are applied?
Look into OpenCV with Nicolas? http://activearchives.org/wiki/Machine_Seeing_Ways_of_Seeing

MM: Segmentation is done with a 'watershed algorithm' (ref. Open CV)
Open CV tutorial on Watershed http://docs.opencv.org/3.1.0/d3/db4/tutorial_py_watershed.html

Excellent explanation of what the GrowCut algorithm does: "The process takes two inputs and creates one output. The first input is  an image to be segmented; the second is a "swipe" image, a mask of seed  points for clusters. The algorithm grows clusters until no more changes  are made." http://im.snibgo.com/growcut.htm

A paper on GrowCut and segmentation in medical environments (interesting test images): http://graphicon.ru/oldgr/en/publications/text/gc2005vk.pdf

--> Dutch Radiologist Quarterly on AI

Notes from the interface

Zooming + rotation: scale, time, movement

JR: 'Super weird how Slicer brings movement in, on its interface i mean. Did you see the vertical slider? Vertical for time!?'

Travelling through the body - Waldy: Anatomy treats the body as a landscape.

Slicer is also used in archaeology (one person?) -- a very interesting time displacement -- unity of individual whole human body, organic... but not alive and probable enmeshed with earth/ground/soil and other materials and beings? Astronomy (one test) is nice too, from the body to the stars. From the cell to the cosmos.

http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly

Documentation, sources, credits etc. mixed with operators for functionalities.

https://www.slicer.org/w/images/5/55/SlicerWelcome-tutorial_Slicer4.5.pdf
if always combines 3 2D windows with one 3D viewer

https://www.slicer.org/w/images/e/e0/Slicer4.5minute_SoniaPujol.pdf
A Slicer scene is a MRML (Medical Reality Modeling Language) <- Inspired on VRML? From virtual to medical reality? Seriously?!
The scene file and datasets have been saved as an ‘.mrb’ (Medical Reality Bundle) file. The MRB file format is Slicer’s archive file format. 
Slicing never ends: Clipping! Spinning!

https://www.slicer.org/w/images/5/51/3DDataLoadingandVisualization_Slicer4.5_SoniaPujol.pdf
'3D Slicer: an open-source platform for translating innovative algorithms into clinical research applications'
can save .stl, .obj ...

https://www.slicer.org/w/images/d/d5/RegistrationTutorial_3DSlicer4.5_spujol.pdf
Registration -- 2 scans per person
3D registration happens also immediately, automatically?
Register 2 persons to one image: BSpline transform https://en.wikipedia.org/wiki/B-spline
[not sure I got it right, but seems you can easily average multiple images, across sets of scans and subjects]

https://www.slicer.org/w/images/4/48/FastGrowCutTutorial.pdf
Editor = Generic Anatomic Colors (are they always the same? Is there a standard for this?)
Hmmm. can't make it work

https://www.slicer.org/w/images/6/65/WhiteMatterExplorationTutorial_SoniaPujol_Slicer4.5.pdf
https://www.assembla.com/spaces/slicerrt/documents/bmRQGEzzur54v-dmr6CpXy/download/bmRQGEzzur54v-dmr6CpXy

How to segment multiple vertebrae in spine CT for 3D printing https://www.youtube.com/watch?v=Uht6Fwtr9hE

Realistic training
Spine phantom
Speckling
Islands
Scissors tool
Smoothing, doing away with holes
Subtracting, adding – human body becomes available for graphical operations

How to approach Slicer as an agential cutter?

Notes from reading the Slicer forum

DICOM = Digital Imaging and Communications in Medicine is a standard for storing and transmitting medical images. https://en.wikipedia.org/wiki/DICOM#History
Definitions of DICOM http://dicom.nema.org/medical/dicom/current/output/chtml/part03/PS3.3.html

---------------------

My Interior Cavities https://www.youtube.com/watch?v=vXPQP8OjCa0

---------------------

Shadow ... ?
Segmentation and recognitions
Patterns?
"Clique potentials are used to model the social impact in labeling."
https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm -- shortest path (one way segmentation works)

---------------------

http://tomopy.readthedocs.io

This proprietary architecture captures an enormous amount of acoustic data from each transmit operation and performs digital beam reconstruction along with mathematically optimized focal processing to create real-time images with exceptional resolution and uniformity.
https://www.usa.philips.com/healthcare/resources/feature-detail/nsight.html

“MAGNETOM Vida: Embrace human nature at 3T”
NOT cutting

radio-opaque contrast agent
CT ‘non-invasive’ manner – changes DNA = cancer risk
Plant research: checking roots without disturbing them
http://www.plantphysiol.org/content/suppl/2011/12/20/pp.111.186221.DC1

Tomographic reconstruction
https://en.wikipedia.org/wiki/Tomographic_reconstruction
http://visielab.uantwerpen.be/computed-tomography-and-astra-toolbox-training-course

Non-destructive
Computed tomography
Tomos = section/slice
Computed ? not destroying it.
Rontgen … [naming practices of naming a technique after yourself]
Image of his wife’s hand
Eine Neuen Art von Strahlen
Radon Transform: mathematical idea that makes Tomography 3D
EMI (!) Godfried Hounsfield, first CT machine
attenuation: process of x-ray beams passing through the body
attenuation value, related to density
“object atoms will be in its way”
“exponential relations are more difficult to compute”
darkfield, flatfield – modelling

Two domains:  


Forward and backprojection, transformations between the two domains
Segmentation: different materials (not just organs)

attenuation
[ah-ten?u-a´shun]  
1. the act of thinning or weakening.
2. the change in the virulence of a pathogenic microorganism induced by passage through another host species, decreasing its virulence for the native host and increasing it for the new host. This is the basis for the development of live vaccines.
3. the change in a beam of radiation as it passes through matter. The intensity of the electromagnetic radiation decreases as its depth of penetration increases.

Only needing to do 180 degrees, xrays passing through
filtered backprojection – flipping the imaging process on itself
first scanners too slow to deal with movement – so: brain (or related to the challenge?)
helix – imaging beam moves around and forward

"Charles S. Peirce introduced in the late 19th century the notion of  abduction as inference from effects to causes, or from observational  data to explanatory theories. Abductive reasoning has become a major  theme in contemporary logic, philosophy of science, and artificial  intelligence. This paper argues that the new growing branch of applied  mathematics called inverse problems deals successfully with various  kinds of abductive inference within a variety of scientific disciplines.  The fundamental theorem about the inverse reconstruction of plane  functions from their line integrals was proved by Johann Radon already  in 1917. The practical applications of Radon’s theorem and its  generalizations include computerized tomography which became a routine  imaging technique of diagnostic medicine in the 1970s"


The scanner story BBC
https://www.youtube.com/watch?v=u_R47LDdlZM
“doctors can hinge our bodies open at any point”
accessibility
x-ray = shadowgraph
Teaching students the difference of having medical imaging https://youtu.be/u_R47LDdlZM?t=10m37s
Reconstruction is already needed for computing the slices, to construct the volume back is only later (when?)
part II: “it is as if the computer can’t believe the density of the bullet in the brain”
“We’re slicing through the patient’s body and looking at it as if it was a treetrunk”
Fabricated data bodies: Reflections on 3D printed digital body objects in medical and health domains
https://link.springer.com/article/10.1057/sth.2015.3
digital health technologies
three-dimensional imaging technologies

MRI data consist of a series of cross sections of a three-dimensional object. Like a black-and-white photograph, each cross section has regions of dark and light, and the boundaries between those regions may indicate the edges of anatomical structures. Then again, they may not.
Determining the boundaries between distinct objects in an image is one of the central problems in computer vision, known as "image segmentation." But general-purpose image-segmentation algorithms aren't reliable enough to produce the very precise models that surgical planning requires.

Human factors
Typically, the way to make an image-segmentation algorithm more precise is to augment it with a generic model of the object to be segmented. Human hearts, for instance, have chambers and blood vessels that are usually in roughly the same places relative to each other. That anatomical consistency could give a segmentation algorithm a way to weed out improbable conclusions about object boundaries.
The problem with that approach is that many of the cardiac patients at Boston Children's Hospital require surgery precisely because the anatomy of their hearts is irregular. Inferences from a generic model could obscure the very features that matter most to the surgeon.
In the past, researchers have produced printable models of the heart by manually indicating boundaries in MRI scans. But with the 200 or so cross sections in one of Moghari's high-precision scans, that process can take eight to 10 hours.
(...)
Pace and Golland's solution was to ask a human expert to identify boundaries in a few of the cross sections and allow algorithms to take over from there. Their strongest results came when they asked the expert to segment only a small patch —one-ninth of the total area—of each cross section.
In that case, segmenting just 14 patches and letting the algorithm infer the rest yielded 90 percent agreement with expert segmentation of the entire collection of 200 cross sections. Human segmentation of just three patches yielded 80 percent agreement.
"I think that if somebody told me that I could segment the whole heart from eight slices out of 200, I would not have believed them," Golland says. "It was a surprise to us."
Read more at: https://phys.org/news/2015-09-mri-heart-scans-3d-printed-physical.html#jCp

https://phys.org/news/2007-03-d-medical-imaging-stars.html

Sinograms -> "what the x-ray sees" (= backprojection)
Radon Transform: putting it all together

Laminograms: added up backprojections -- blurry image reconstructions

https://www.youtube.com/watch?v=MA2y_2YySq0

reverse radon projection https://www.youtube.com/watch?v=qgEvSh8U_ok

watershed meets reconstruction meets segmentation https://www.youtube.com/watch?v=K5P5rjDiZzk

-----

16/10/2017

MEMORAD JAARGANG 22 - NUMMER 3 - NAJAAR 2017
https://www.radiologen.nl/system/files/bestanden/publicaties/nvvr_mr_22.3_web_2.pdf

Radiologist quarterly on AI: 
Radiology = fist medical profession to become obsolete as a result of computing/automation

"fourth industrial revolution"

AI (?) mainly used in mammography (US -- FDA? why?)

"vertically experienced cat" https://www.youtube.com/watch?v=QzkMo45pcUo

CAD = 

PACS = Picture, Archiving and Communication Systems -- feeding the cloud
radiologist as medisch datamanager

https://eusomii.pro/
radiology informatics -- "beeldvorming"

https://research.google.com/teams/brain/healthcare/
digital pathology
"we built an automated detection algorithm that can naturally complement pathologists’ workflow."

edge detection -- segmentation -- 
Radiomics, radiomic feature https://en.wikipedia.org/wiki/Radiomics
"volumes of interest"
"curse of dimensionality" https://en.wikipedia.org/wiki/Curse_of_dimensionality

Pixelshine -- opwaarderen van lage-dosis beelden. Hoge resolutie, maar wordt de botbreuk weggepoetst?

"Ik denk dat veel mensen de correlatie met de causatie verwarren"

Healthineers (Siemens)

MRI scans as unstructured data -- patients generate their own data too (only one mention of this though the self-referencing radiogooglist appears)

"Quantib’s clinically-validated algorithms have been tested on over 10,000 scans in the Rotterdam Scan Study dataset."
"biomarkers and tools for cardiovascular, neuro, neurovascular, musculoskeletal, abdominal, oncology and lung images with the help of our advanced software."

Algorithms before computers
https://www.youtube.com/watch?v=pqoSMWnWTwA


https://philpapers.org/rec/GIBSTE


Laura Kurgan

Writing on Slicer
segmentation, biomedical, bridge to ML + CV
FS: Writing on Slicer (needs to happen!) Possibly with help of Martino, Seda (?)
flattening and re-injecting volume
JR interested in re-composition/re-construction
computational efforts to calibrate.

"the Transmarcations program reminds me of the book Purity and Danger (1966) by  anthropologist Mary Douglas, which has a few lines that I often come  back to in my work/life. I'm not sure if you are familiar with it, but i  find it a beautiful analyze of how dirt is matter out of place. Meaning  that the idea of dirt is proof that we have a system of categories for  where things belong, as in a pair of muddy boots in the hallway is fine,  but put them in the bedroom and they are dirty. She argues that our  strongest category is our own body, and that things that transition that  category become the most dirty, as in hair on our head versus a strand  of hair in food, or body liquids that are dirty as soon as they leave  our body. So the more something is considered dirty the stricter of a  category it has transitioned."

https://lukeoakdenrayner.wordpress.com/2017/12/18/the-chestxray14-dataset-problems/