The OpenCV DNN module only supports deep learning inference on images and videos, but it supports loading many popular deep learning frameworks such as TensorFlow, (Py)Torch, DarkNet, Caffe, ONNX. - DNN opencv module: A Definitive Guide / A Comprehensive Guide / DNN

It is able to run caffe NN, see

• samples/dnn/openpose.cpp example

PyTorch / YOLO

Pytorch model have to be converted to ONNx first

The following code contains the description of the below-listed steps:

• instantiate PyTorch model
• convert PyTorch model into .onnx
• read the transferred network with OpenCV API
• prepare input data
• provide inference
• get colored masks from predictions
• visualize results

Running pre-trained YOLO model in OpenCV - overview of exporting pre-trained YOLO family models from PyTorch and deploying them using OpenCV’s DNN framework.

• Why does YoloV8 perform poorly when exported to .onnx
  • you need to handle the pre-processing that is done in python to create the model input.

Note that this involves a number of steps that are not necessarily trivial and need to be done in the same order as what was done to the input when the model was trained. Resize needs to use anti-aliasing. Normalization needs to use the right range (pytorch is usually float from 0..1, tensorflow is usually float from -1..1). The ordering of the data needs to be correct (channels first vs channels last, RGB vs BGR).

Conversion of TensorFlow Segmentation Models and Launch with OpenCV

The key concepts involved in the transition pipeline of the TensorFlow classification and segmentation models with OpenCV API are almost equal excepting the phase of graph optimization. The initial step in conversion of TensorFlow models into cv.dnn.Net is obtaining the frozen TF model graph. Frozen graph defines the combination of the model graph structure with kept values of the required variables, for example, weights. Usually the frozen graph is saved in protobuf (.pb) files. To read the generated segmentation model .pb file with cv.dnn.readNetFromTensorflow, it is needed to modify the graph with TF graph transform tool.

.tflite

Not supported yet, but should be convertible to .pb

$ bazel run -c opt //tensorflow/lite/toco:toco -- --input_file=palm_detection.tflite --output_file=graph.pb --input_format=TFLITE --output_format=TENSORFLOW_GRAPHDEF

Conversion of PyTorch Segmentation Models and Launch with OpenCV

The key points involved in the transition pipeline of the PyTorch classification and segmentation models with OpenCV API are equal. The first step is model transferring into ONNX format with PyTorch torch.onnx.export built-in function. Further the obtained .onnx model is passed into cv.dnn.readNetFromONNX, which returns cv.dnn.Net object ready for DNN manipulations.