Full Idea
What sets Zermelo's modelling of arithmetic apart from von Neumann's and all the rest is that he identifies the primitive of arithmetic with an appropriately primitive notion of set theory.
Gist of Idea
Zermelo's model of arithmetic is distinctive because it rests on a primitive of set theory
Source
David Lewis (Parts of Classes [1991], 4.6)
Book Reference
Lewis,David: 'Parts of Classes' [Blackwell 1991], p.111
A Reaction
Zermelo's model is just endlessly nested empty sets, which is a very simple structure. I gather that connoisseurs seem to prefer von Neumann's model (where each number contains its predecessor number).