Turning off previews

It’s taking too long to generate previews as I develop the resnet18 build, so I turned off previews.

I switched view to the model hub, then returned to the model and got this, i.e. previews still off but spinning circles for a minute or so… then the screen refreshed to the collapsed component view consistent with the preview off setting.

Is this expected behaviour? Is there any way to completely turn off preview generation while I work on structured & code? (Of course I really do want the previews when its done, but for now they’re not necessary.)

How big is the dataset? I know with mine on first load I have to wait 10-20min for things to stop loading in from the old spinny hdd (watch your disk resources in task manager) once done they’ll show but after switching screens they’ll take awhile.

@JulianSMoore Turning off previews only turns off the frontend rendering of them, which in some cases alleviates some lag. The previews are still internally calculated though to get some of the good stuff such as the dimensions of each component and errors.
That the spinners are showing even though they are closed is a bug, but be aware that the computation is still happening behind the scenes.
It also sounds like you might want the computation to stop entirely when the previews are closed, is that right? :slight_smile:

Hi @RBR

Thanks for the input… this is the PL Cifar10 dataset; 60k images. Each image is 32x32 and about 1KB, so a lot of file reads initially, all of which are (PC, not PL) inefficient because of their size.

Excellent tip re resource monitor… I did take a look for the 50s it takes to get previews refreshed but I didn’t notice any major image/csv access - I did see a lot of reading from Chrome’s IndexDb where PL caches a lot of stuff though.

But, you did say “on first load” and, yeah, there’s no avoiding however long it takes to pre-process a whole dataset (I’ve just downloaded ImageNet100 which is ~18GB - just in case I feel like trying it with ResNet18 - that would be an interesting stress test. I downloaded to spinning rust too rather than SSD because I seem to be a data hoarder :wink: )

Interestingly, Chrome wanted to restart for an update while I was looking into this, and I let it…

I saw then that PL is also reading “latest-training-results.pkl” from other models as part of whatever happens in the browser, but simply switching to the model hub and back to a model doesn’t seem to have any major disk access issues at first glance.

@robertl has now clarified that previews off only turns off the front end display so I think the original question is answered now.

Thanks @robertl

A question to ponder internally :slight_smile: (not necessarily a good use of time to educate a user on internal details that can change)

Dimensions: why would they be calculated from “preview” rather than “output”?

I had been looking at custom components and thinking about special processing (e.g. size changes) that might be done on the preview to speed things up*. It seems like an unnecessary dependency from the naïve user’s perspective.

And, this happens even when there have been no structural changes (AFAICT). I could imagine a “dirty” flag on a component that would affect it and everything downstream whenever there’s a change that affects/could affect dimensions so that, if a component has not been dirtied the preview doesn’t need to be regenerated.

(* size change before doing other stuff; for example I was considering XOR of input & output after reducing size to 32x32 to minimise computations)

It also sounds like you might want the computation to stop entirely when the previews are closed

Not if the computation is essential to having a valid model :wink: But it would be nice if unnecessary computations could be avoided and if preview computation was only for the benefit of previews and so could be entirely turned off…

1 Like