It's what is or isn't vital. A strong test is whether the elements pertinent to a given genre are in fact necessary to the tale. Not a few "genre" stories and novels could easily be rewritten with only minimal changes in some other genre, or even as "mainstream" (which is, of course, itself a genre). In such tales, the "genre" elements are merely wallpaper, and could be changed at will. Only if the genre-relative elements are essential to the tale--that is, the tale is basically impossible without them--is the tale of that genre. Many less-able writers write "genre" fiction that is just a plain story with fancy curtains and wallpaper. An example that always pops up in my mind is David Gemmell's "Jerusalem Man" series, always listed as "fantasy" but in plain fact just westerns. Glen Cook's "Black Company" tales, after the first set (which is basically one extended novel") are effectively Vietnam war stories. And so on.