Perceptilabs A.I colorization

My next little project: An A.I colorization model in perceptilabs :slight_smile:

Instead of writing i decided to make a video where i breafly explain how the model works here: https://youtu.be/3j1sARksK1k

The result is far from perfect as i’m using the same app for inference as for the super-resolution model, which splits the whole image into chunks. Also the dataset is rather small at about 6000 imgs for the task. It does still do well on some images :slight_smile:

2 Likes

That’s fun! Glad to see you are still pursuing practical modelling (I’ve been deep in papers on trying to work out how to make the best possible models: beyond overfitting and into interpolation. Robert and I have been debating (loosely) what else is going on when e.g. there is weight decay. I am thinking about experiments now…

It’s doing rather well for such a small dataset - it’s interesting to think about why the same thing can be coloured so differently: the fence that is mostly wood coloured except for the section with a strong blue tint. (Is that a shadow on the road or a change in the stone texture/pattern).

If you can work out why same thing ends up a different colour/shade depending which segment it’s in I think it will be rather revealing/impressive!

Working in RGB space or tempted to try out HSV/HSL? :wink:

I think the problem with things being differently colored is the way i do the inference here. I was lazy and just used the same app as for the upscaling :laughing: which uses a “rolling window” approach. And so the model color each window separately. I guess maybe it could do better with a larger dataset though…

I’m working in YUV colorspace so the model only needs to predict 2 channels instead of 3 (RGB) :slight_smile:

YUV… so you predict the red and blue projections… and recalculate the luma from them?

Not sure whether/when this could be done in PL, but saw this for GAN face restoration :slight_smile: - thought you might be interested! Linked in post

The luma is not recalculated, I just take the input luma channel and stack it together with the predicted red and blue projections :slightly_smiling_face: Then it’s converted to RGB to match the target.

Ah, I guess it can be done in PL, but as GAN is removed for now it would be a bit cumbersome, i think :sweat_smile:

1 Like