ofImage resize is so inefficient,any ideas?

one option is that you might try using opencv to resize the image. I know with opencv for example, there are different interpolations that can be faster / slower such as nearest neighbor, linear, bilinear, etc