How to manipulate an image using the mesh vertices you've detected?

I have detected a mesh of vertices (joined by OF_PRIMITIVE_LINES) based on the luminosity of each pixel.

Result3-neighbour10.jpg750x600 256 KB

I am animating this mesh by adding a simple jitter movement. In the screenshot below, I have placed the original image below the mesh so you can also see the background.

When the jitter movement increases you can start seeing the lines of mesh. What I would like to do instead is use the vertices in the mesh point to locations on my image and manipulate the pixels on the image (or texture of my image?)

I am reviewing the OpenGL basics but meanwhile it would be great if you can point me to the right direction to achieve this.

Thanks!