Full Idea
The problem with reducing arithmetic to ZFC is not that this theory is inconsistent (as far as we know it is not), but rather that is not completely general, and for this reason not logical. For example, it asserts the existence of sets.
Clarification
ZFC is Zermelo-Fraenkel set theory with Choice
Gist of Idea
As a reduction of arithmetic, set theory is not fully general, and so not logical
Source
A.George / D.J.Velleman (Philosophies of Mathematics [2002], Ch.4)
Book Reference
George,A/Velleman D.J.: 'Philosophies of Mathematics' [Blackwell 2002], p.90
A Reaction
Note that ZFC has not been proved consistent.