This is an old revision of the document!
* To build a dense prediction model * To begin reading current papers in DNN research
For this lab, you will turn in a report that describes your efforts at creating a pytorch radiologist. Your final deliverable is a notebook that has (1) deep network, (2) cost function, (3) method of calculating accuracy. You should also report on how much of the data you used.
Your notebook should also include an image that
shows the dense prediction produced by your network on the
pos_test_000072.png
image. This is an image in the test set that
your network will not have seen before. This image, and the ground truth labeling, is shown at the right. (And is contained in the downloadable dataset below).
Your notebook will be graded on the following:
The data is given as a set of 1024×1024 PNG images. Each input image
(in the inputs
directory) is an RGB image of a section of tissue,
and there a file with the same name (in the outputs
directory) that
has a dense labeling of whether or not a section of tissue is
cancerous (white pixels mean “cancerous”, while black pixels mean “not
cancerous”).
The data has been pre-split for you into test and training splits.
Filenames also reflect whether or not the image has any cancer at all
(files starting with pos_
have some cancerous pixels, while files
starting with neg_
have no cancer anywhere). All of the data is
hand-labeled, so the dataset is not very large. That means that
overfitting is a real possibility.
The data can be downloaded here. Please note that this dataset is not publicly available, and should not be redistributed.
For this lab, you will implement a virtual radiologist. You are given images of possibly cancerous tissue samples, and you must build a detector that identifies where in the tissue cancer may reside.
For this lab, there are two primary tasks:
Part 1: Implement a dense predictor
In previous labs and lectures, we have talked about DNNs that classify an entire image as a single class. Here, however, we are interested in a more nuanced classification: given an input image, we would like to identify each pixel that is possibly cancerous. That means that instead of a single output, your network should output an “image”, where each output pixel of your network represents the probability that a pixel is cancerous.
Part 1a: Implement your network topology
Use the “Deep Convolution U-Net” from this paper: U-Net: Convolutional Networks for Biomedical Image Segmentation (See figure 1, replicated at the right). This should be fairly easy to implement given the
conv
helper functions that you implemented previously; you
may also need the pytorch function torch.cat
and nn.ConvTranspose2d
Note that the simplest network you could implement (with all the desired properties) is just a single convolution layer with two filters and no relu! Why is that? (of course it wouldn't work very well!)
Part 1b: Implement a cost function
You should still use cross-entropy as your cost function, but you may need to think hard about how exactly to set this up – your network should output cancer/not-cancer probabilities for each pixel, which can be viewed as a two-class classification problem.
You are welcome to resize your input images, although don't make them so small that the essential details are blurred! I resized my images down to 512×512.
I used the scikit-image
package to handle all of my image IO and
resizing. NOTE: be careful about data types! When you first load
an image using skimage.io.imread
, it returns a tensor with uint8
pixels in the range of [0,255]. However, after using
skimage.transform.resize
, the result is an image with float32
entries in [0,1].
Don't forget to whiten your data. And remember that if your data is stored as a numpy array, be careful about the data type: if you try to whiten it while it is still a uint8
, bad things will happen.
You are welcome (and encouraged) to use the built-in tensorflow dropout layer.