Viv: For instance, statistics is an incredibly useful tool, but it works within margins of error that would give a physicists or someone working in fluid dynamics a heart attack.
As a practitioner, I would say statistics is a common description of one or more mathematical subfields and a science of how they are applied in various situations.
The definition is very vague because statistics is an umbrella term which cover many different types of practice some which are in mutual conflict in various ways.
At any rate, the core of most (I cant think of a counter example right now) areas of statistics is pure math and is only inexact in the sense it may be applied approximatively. A lot of good physics, incidently including much fluid dynamics, boil down to statistical considerations cf. the role of statistical mechanics and for the sake of example it's application to turbulence in fluid dynamics. Scientific inference, i would claim, also fall within statistics.
At any rate a misapplication of statistics or crude approximation may give anyone a heart attack, however the same hold for any misapplication or crude approximation made within any other physical field. As an example one could consider a newtonian approximation to a non-newtonian fluid, or laminar assumptions to a turbulent flow and so on in fluid dynamics.