After data src change, Input settings still show old file path

PL V0.12.24 Windows 10 Home 64-bit 21H1

The data settings were used to change the source file… and the “current” settings reflect the previous change, but the settings panel still shows the original data file.

NB PL server was shutdown and restarted between source change and re-check… because seeing the settings panel I thought the src change had failed.

The dataset is 112k 224x224 RGB images (@Birdstream’s super-res x4); previews were present before logout and restart server, but it takes a phenomenally long time (~5 minutes on a fast PC) to regenerate the previews on reopening the model after the restart… it’s as though it is (pre)processing the whole dataset again.

I would have said this was because I didn’t save the model before logging out etc., but PL did save the src file change or are we seeing a cache/save effect here? During the preview regeneration time CPU usage (i8700k) was at ~50% across 12 cores!

On the other hand - see second image below - the model is clearly not “dirty”, so the absence of a previous save was not a factor.

It does take a while to open the model, yes! Could it be that the model uses MinMax normalisation on both input & output? But I wouldn’t say it takes 5 mins on my machine, more like a minute and a half… Do you keep the dataset on SSD?

Thanks for letting us know @JulianSMoore!
I’ll add it to our bug list, here is a Canny ticket: :slight_smile:

1 Like

No @birdstream, the dataset is on spinning rust… I already have too many datasets and my data SSD is My Precious, so I have offloaded some things to real disks.

That said, the disk concerned is a Windows storage NAS built from 4 disks and I know generally that it can sustain ~150MB/s transfers… I guess the small size of the images is the issue here.

The real point is the need for some sort of progress/status update every 10s or so because sometimes things fail, and when there is no progress info at all, how long does one wait before calling it? I have had that happen (in other apps!), clicked cancel on something at precisely the time it was finishing and effectively nullified my own success and had to start again.

It’s going OK so far, though the air from the PC is noticeably warmer at the moment :wink:

1 Like