I finally cracked it. You have to wiggle the front of the handle out, which often requires lifting the handle, then rotate the handle anti-clockwise...
I need to get all four door handles removed so that I can paint. I have had the front passenger handle off in the past but I am working on the front...