This is a refreshing way to frame the argument. I've often had similar thoughts when vegans tell me humans are the only species to drink milk into adulthood; I think to myself, "we're also the only species to play chess, compose music, build architecture, and so on". So yes, the "nature" argument is a double-edged sword. I'm not a vegan, and don't intend to become one. I do, on the other hand, absolutely support the (unfortunatley latent) message that food production needs desperately to move away from a big, industrial model. That model is only "necessary" for making small numbers of people rich; it isn't required by the species.
But I have to say that I find the moral arguments about diet myopic. Living beings simply cannot live on inorganic matter. And it is, in my opinion, only a matter of time before we begin to realize more profoundly that plants are as much "living" beings as animals. Sounds silly, I know, but humans do maintain a morally questionable hierarchy of life, such that the more human-like a being is, the more morally biased we are toward it (e.g., primates > quadrupeds > fish > insects). Yet trees have been shown to live in communities, to communicate, and even to help one another survive and thrive. So it's quite likely that it's primarily our particular temporal niche (i.e., the rate at which we demonstrate adaptive behaviour), not to mention our mistakenly individualistic world view, that makes us think of plants as "less alive" than animals. In this light, any moral argument will eventually reveal that life itself is "immoral". Cruel and expressly inhumane factory farming should be outlawed, no doubt. But I suspect that the morality of eating is beyond our control and that moral judgements grounded in diet should be relegated to the dustbin of history.