I don't think I've seen any serious discussion of Harry Potter as big L literature but I'd imagine that's because it obviously isn't Literature.
Says you. There are quite a few people who think it is literature with a capital L, and she has won literary awards for the series. This is what the critics are often objecting to, and what Bond is complaining about. For the third book in the series, Rowling won the prestigious literary Whitbread Book of the Year Award for children's fiction, (now called the Costa Book Awards,) which put her on the shortlist of nominees for the main prize. This caused an enormous hue and cry, with one judge threatening to resign if she won. (She came in second after the poetry nominee.) So there has been much serious discussion of the series as literature, both as its own and as a children's/YA series. And there will continue to be.
That said, again, Bond's view of a cohesive, organized, exclusive "literary apparatus" is decidedly 1960's, as was Grossman's approach, which several people critiquing the column complained about. While people do still come through the old college degree/MFA, write for specific magazines, get book deal path, an enormous amount of them don't. It is true that some people cling to the idea of that social class system, as Bond is doing. Bloom is certainly one of them -- he was scarred by the radical changes that took place in academia in the 1960's and 1970's. Which makes his old man ranting attractive to the media, because it stirs up controversy for them. What techniques there are for marketing fiction as literary sometimes make use of this idea, but there is also the whole body of non-white, "ethnic" fiction, and other marketing techniques that are the exact opposite to this. More money is made now by dropping the poor pitiful literary fiction in the gutter for the elites approach than in the other direction. Plus, it's very hard to claim that "literary" writers are being ignored when they are getting multi-million dollar multi-book contracts. (And yet, even within our SFF field, the idea persists.)
The arguments you are having about Modernists versus Realists is symptomatic of the same debates that go on all the time, in academia, media, pals around the table, and shows exactly why there is not nor never will be a set standard criteria for "literary." I will say this: Twain didn't have a problem with non-realism in his own work. He was not a Modernist, but a part of the Victorian age. His objection to Cooper was that he thought the guy wrote purple prose. And Pynchon, who uses genre elements, whether you want to call him modernist or postmodernist, is a lauded darling of literary critics and the like, so it sort of blows the argument of a Realist/Modernists, don't use genre dominance out of the water right there.
1) The medium is not the message: This is the low-brow/high-brow culture divider. Whatever form a creative work takes -- horror movie, comic book, shaving cream sculpture, etc. -- that form does not automatically preclude it from being considered a work of substance and value -- art -- in one direction or another. As you know, what was considered low brow over time has a tendency to become high-brow -- opera, Shakespearean theater, ballroom dancing, big band music, the Beatles are now the grand emeritus of rock artistry, etc. You can argue that you don't think in time shaving cream sculpture will ever be considered high culture, but they said the same thing about opera. The medium does not keep a particular work out of the running. When people claim it does, it's often code for "the young people like this and so it must be awful."
2) The publisher is not the message: This is the literary/genre divider with genre being one sort of fiction (elf/spaceship/sex.) People do a lot of their literary categorizing by publisher -- where the book is sold in the bookstore, what kind of cover art, whether it comes out first in paperback or hardcover, tie-ins, children's/YA or not, etc. Small press is often considered better quality than large press. Certain "status" publishers like Knopf are seen as larger signs of quality (that would be the leftovers of Bond's literary apparatus.) And reviewers -- including genre reviewers -- pay a great deal of attention to who the publisher is in choosing what to review, (though that is now one important thing that has changed over the last ten years.) It's judging art through packaging and marketing (which gives the publishers all sorts of ways to "trick" people on authors.)
3) Extrapolation is a weak argument: People cherry pick data and extrapolate entire systems from it. You read a couple of tie-in books and they are awful, and your friend read one that was awful and a lot of people say that they are awful, so therefore, tie-in novels are awful. You've heard that all these women-written urban fantasies are romances and you read one that seemed kind of romantic to you, so all the women-written novels are romances. A non-fan read a popular SFF title and didn't like it, and so talks about how fantasy fans want elves and a particular kind of comforting storyline, etc.
In particular, people will pick a popular title, such as say, Stephanie Meyer's Twilight, and from it, and possibly their reaction to reading it, begin to extrapolate all sorts of theories about what this means about what teenage girls read or why children's/YA appeals to teens and adults. In trying to combat #1 and #2, Grossman extrapolated in his column. But it's a weak argument because it's wildly inaccurate and anyone can poke holes in it by pointing out contradicting data. Which is why those making the argument then try to claim that such data doesn't count.
4) Style and approach do not define art: There are a lot of people who are convinced that there is one Right Way to Write Fiction, as Hal Duncan puts it, and try to argue that this narrow formula is how one should determine what is good or literary. That all fantasy should be dark, gritty and realistic to be considered good is one of those. That fiction should be written for adults only to be art is another. That fiction should not have violent action in it, that it should not have spaceships in it, to be literary, and on and on. The whole character versus plot debate that Grossman is objecting to is one of these, with what is character-focused and what is plot-focused wildly extrapolated and vaguely argued, which is why it doesn't really work to cast the Modernists, one movement, as Darth Vadar.
But Grossman was also trying to cast them as one strand in the panoply of fiction literature, to hammer at those who believe the Modernist vision, or post-modernist, etc. was the style and approach that defined art. So points for him.
Sorry, but it doesn't matter how many critics you have, as you know. And as for readers, the more people who love or think highly of a work, the more you have who do not. Many others do regard it as Literature, and in my opinion it is. Until one of us is appointed king of the universe, that's pretty much where the matter lies, though certainly we can debate it. 
