Full Idea
The idea that objects do not possess secondary qualities intrinsically rests on the thought that they do not figure in the physicist's account of the world; ..as they are causally idle, no purpose is served by attributing them to objects.
Gist of Idea
We say objects possess no intrinsic secondary qualities because physicists don't need them
Source
Howard Robinson (Perception [1994], III.1)
Book Reference
Robinson,Howard: 'Perception' [Routledge 2001], p.61
A Reaction
On the whole I agree with this, but colours (for example) are not causally idle, as they seem to affect the behaviour of insects. They are properties which can only have a causal effect if there is a brain in their vicinity. Physicists ignore brains.