Full Idea
A simple argument makes it clear that all mathematical arguments are abstract: there are infinitely many numbers, but only a finite number of physical entities, so most mathematical objects are non-physical. The best assumption is that they all are.
Gist of Idea
There is an infinity of mathematical objects, so they can't be physical
Source
James Robert Brown (Philosophy of Mathematics [1999], Ch. 2)
Book Reference
Brown,James Robert: 'Philosophy of Mathematics' [Routledge 2002], p.12
A Reaction
This, it seems to me, is where constructivists score well (cf. Idea 9608). I don't have an infinity of bricks to build an infinity of houses, but I can imagine that the bricks just keep coming if I need them. Imagination is what is unbounded.
Related Idea
Idea 9608 There are no constructions for many highly desirable results in mathematics [Brown,JR]