The idea for a Projects section was inspired by the great discussion around @JWalker’s thread Increasing the number of images increases the loss and the thought that maybe it would be nice to have somewhere dedicated to open ended discussion, progress reports, etc.
I am always conscious of the fact that I use Q&A sites (e.g. stackexchange) and forums to get quick answers and prefer short threads on simple topics for that purpose - but I’m really liking the to-and-fro of ideas etc. we have here, so I wanted to have my cake and eat it, so to speak.
Anyway, that’s why I’m doing this here…
The original motivation is to do what I said in the thread above - do some investigation of double descent using a model and dataset from a real paper so that I can compare results.
But I’ve now decided that doing this is going to help me think about translating the layer diagrams I see everywhere into PL or other models, get a better feel for structure, and practice component/block reuse in PL for bigger models - [added later] i.e. develop a workflow for larger PL projects.
Bigger models? I have some very grand ambitions - and ease of use is what drew me to PL in the first place. I can code, in a non-coder way, but code isn’t the priority for me - it’s what it does and I won’t want to be debugging stupid typos, wrong loop limits, etc. if I can avoid it!.
Windows 10 Home, 64-bit, build 21H2, PerceptiLabs 0.13.5; Python 3.8.10, TF2.5.0, GPU 1080 Ti, CUDA 11.2.2 & cuDNN 126.96.36.199 via conda-forge channel
No, I’m just about to end this intro post and actually do some stuff now that I have added a new model to CIFAR-10 dataset I had already loaded up.
1st task - build ONE block and then think about how to efficiently copy/paste and update the #filters (or whether it’s just as easy to do it by hand)
i.e. turn this code from github into a set of PL components…
def make_basic_block_base(inputs, filter_num, stride=1): x = tf.keras.layers.Conv2D(filters=filter_num, kernel_size=(3, 3), strides=stride, kernel_initializer='he_normal', padding="same")(inputs) x = tf.keras.layers.BatchNormalization(axis=BN_AXIS)(x) x = tf.keras.layers.Conv2D(filters=filter_num, kernel_size=(3, 3), strides=1, kernel_initializer='he_normal', padding="same")(x) x = tf.keras.layers.BatchNormalization(axis=BN_AXIS)(x) shortcut = inputs if stride != 1: shortcut = tf.keras.layers.Conv2D(filters=filter_num, kernel_size=(1, 1), strides=stride, kernel_initializer='he_normal')(inputs) shortcut = tf.keras.layers.BatchNormalization(axis=BN_AXIS)(shortcut) x = tf.keras.layers.add([x, shortcut]) x = tf.keras.layers.Activation('relu')(x) return x