Robert Holdstock is an author I consider challenging. I read his novel Celtika years ago and found that it reminded me of Gene Wolfe with it's weird, wandering plot and the way it had so many mythic themes layered throughout it.

Michael Swankick is also very challenging. Iron Dragon's Daughter was just so weird and trippy that I wouldn't even know where to begin if I wanted to analyze it.