The exact situation of the eponymous holy well is disputed. Here on its page about Holywell Manor House Balliol College Archives state:
The Holy Well has been assigned at various dates to as many as four difficult sites. Of these two lie in the present garden of the Manor, the one immediately to the north of Holywell Church, the other eighty yards to the east of the Manor House, sometimes called Jenny Newton's well. A third site is beyond Napper's Bridge, 100 yards to the north-east of the Manor; while a fourth would identify the Holy Well with Crowell, the pond at the junction of Holywell and Long Wall. It is not possible to decide between these. Hearne, who favours the first site, says; 'The well at St Cross was much frequented and brought a vast quantity of money to the place. This well is on the north side of the Church in the garden that belongs to the Manor House and is quite different from another well more eastwards which is now frequented by some people and the water of it reckoned good for the eyes and some other sores; but this well is of very little standing in respect of the other and ought not to be confounded with it.'
Anthony Wood wrote as follows about the well in his City of Oxford:
On the north side [of St Cross Church] betweene it and the Mannor-house is an antient well, from whence the parish took its name called “Halywell,” though now, more properly, called “Holy well.” Upon what account it had that epithite bestowed upon it, whether for the imployment of the water therof about sacred uses for the church as is before said, or els that by the reputed holiness therin in respect of the miracles it wrought and the like, I am in doubt.
He also wrote (under St Cross Church) that the well was “holy from the supposed holynesse of its water therin, having bin antietntly and lately employed about holy uses done in this church”.
According to Dr Plot in his Natural History of Oxfordshire, Richard Fitzjames, the Warden of Merton College from 1482 to 1507:
built a faire house over it of stone, with a roof to it of free stone, about the year 1488, a token of which bounty remaineth over the door therof at this time, viz., a dolphin naiant carved on a shield with another coat adjoyning sometimes quartered by the former, being the arms of the said bountiful and worthy Doctor.
In the eighteenth and nineteenth centuries the former holy well was known as the cold bath. The Gossiping Guide to Oxford for 1878 recorded:
The Holy Well, from whence the parish takes its name, was dedicated to SS. Winifred and Margaret. It is now covered up by the boundary wall of the Cemetery. The spring war remarkably cold and seldom froze. Antony à Wood says, “I find many persons yearly relieved by these wholesome waters to this day” (about 1675). The water was deemed a specific for ophthalmic complaints.
The website of Balliol College Archives comes down more emphatically on the site of the well here:
The site of the holy well is next door to St Cross church in the garden of Holywell Manor, Balliol’s graduate centre. You can see the site of the well (or spring) if you look through the wooden gate in the wall between St Cross church and the entrance to Holywell Manor. It’s covered by a grating in the pavement. The water table has dropped, and the well is now more of a muddy puddle!
On 3 May 1890 a letter from Herbert Hurst was published in Jackson's Oxford Journal where he wrote of his conversations with the Revd John Rigaud, including:
The holy well at Holywell about 40 years back was a clear and beautiful basin for bathing in; myself and Dean Liddell, of Christ Church, bathed in it often about 1834, but the draining of the Cemetery and other parts around gradually lowered the water to three or four inches depth. I superintended in after years the building of the Sisterhood's chapel over the spot.