Full Idea
As Kreisel has remarked, what is important is not the existence of mathematical objects, but the objectivity of mathematical statements.
Gist of Idea
What matters in mathematics is its objectivity, not the existence of the objects
Source
Michael Dummett (Frege Philosophy of Language (2nd ed) [1973], Ch.14)
Book Reference
Dummett,Michael: 'Frege Philosophy of Language' [Duckworth 1981], p.508
A Reaction
[see Maddy 2011:115 for the history of this idea] It seems rather unclear where Frege stands on objectivity. Maddy embraces it, following up this idea, and Tyler Burge's fat book on objectivity.