Error when running the exported model in python

I exported my model in tensorflow format to use it in python (Thanks @robertl for the script example), but when I run it, I get this error:

UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node StatefulPartitionedCall/deep_learning_conv__convolution_1_keras_3/Relu}}]] [Op:__inference_restored_function_body_2460]

and this warning:
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
Traceback

In the Pl’s Model Hub, the model appears as trained. Is there any file that I missing or something?

Hi @CamiloRR

The first error is definitely a CUDA installation issue - I used to get that :wink:

Wherever you have CUDA installed (and I strongly recommend in your python environment - even if you also have an OS level installation) either you are missing cuDNN completely or the version is incompatible with the rest of the CUDA toolkit.

I posted a “run book” for CUDA builds a while back - take a look at those and/or note that I use this setup successfully with TensorFlow 2.5 (PL V0.12.x) - anmd TF 2.6 (non PL) (assuming your GPU is a supported nVidia card)

call conda install -y -q -c conda-forge cudatoolkit=11.2.2
call conda install -y -q -c conda-forge cudnn=8.1.0.77

Let me know how you get on!

PS the second error doesn’t seem to affect inference as I had that error too when serving @birdstream’s model, which worked without the “training configuration”. I don’t really know what it means (yet!)

UPDATE See StackOverflow for some info about this

2 Likes