I found I had this same error,
I think the problem is that in the ofxCv code, calling the overloaded version of toOf that takes a mat and an ofImage actually just calls the PixelsRef version internally.
The problem is it uses the function ofImage.getPixels() instead of ofImage.getPixelsRef()
I changed this line of code from
toOf(mat, img.getPixels()); to ttoOf(mat, img.getPixelsRef()); inside the ofxCv addon (line 290 in ofxCv Utilities.h) and That solved the problem for me.