Hi KIm & Dave,
I had a look last night on New Zealand's Birth, Death and Marriage website to pin-point a date of birth and parents' details for James Rayner. I've searched from the dates between 1865 to 1890 and attempted different spellings of the surnames to no avail. See below -
The only entry that caught my eye was James Edgan Alphonso Rayner, due to having a father by the name of William but further research of this name indicated that this person died at the age 7 months.