Extra Homework 1

This is an extra homework: its score can replace a regular homework score if it is higher. There will be no late multiplier: May 14, 10:00am is a hard deadline.

1. Implement the following, general residual block:

   class ResidualBlock(torch.nn.Module):
       """
       A general wrapper to implement a residual block.
   
       Arguments
       ---------
       residual_map : `Callable[[dict], dict]`
           Residual map to wrap.
           It should change the tensor at the `"features"` key.
           The output tensor should have
           the same number of feature dimensions as the
           input tensor.
   
       Calling
       -------
       Instance calls require one positional argument:
       batch : `dict`
           The input data dictionary. Required key:
           `"features"` : `torch.Tensor`
               Tensor of features.
   
       We output the dictionary
       where the value tensor at the "features" key
       is the sum of the value tensor at the input dictionary
       plus the value tensor at the dictionary
       output by `residual_map`.
       """
   

2. Using the residual block, construct the following CNN, with all convolution kernel sizes (3, 3), to be trained on CIFAR-10, just like in Notebook 0425:

   1. Layer normalization, 2d convolution from 3 to 16 feature dimensions, dropout.
   2. A residual block of layer normalization, 2d convolution from 16 to 16 feature dimensions with "same" padding, dropout.
   3. 2d convolution from 16 to 32 feature dimensions with 2 stride.
   4. A residual block of layer normalization, 2d convolution from 32 to 32 feature dimensions with "same" padding, dropout.
   5. 2d convolution from 32 to 64 feature dimensions with 2 stride.
   6. A 2d pool layer.
   7. A residual block of layer normalization, linear from 64 to 256 feature dimensions, ReLU, linear from 256 to 64 feature dimensions, dropout.
   8. A linear layer from 64 to 10 feature dimensions.
3. Train the model on CIFAR-10.