According to this tribute published on Princeton University's web page, Milton Babbit's views were innovative indeed:
"Serious modern music, he argued, was not for the average listener — it could be understood only by those with training, similar to learning how to appreciate works in mathematics, physics or philosophy."
Ultimately, to the musician who was willing to take the time to decipher this music, this would be a rewarding listening experience.
Part of the development of this music stemmed from Babbitt's own fascinations with math and music theory, leading to the development of a particular use of the 12-tone scale in music. This idea originated from Austrian composer Schoenberg, and Babbitt took it to the next level. The 12 notes of the scale, ordered in a particular way, formed the basis for the piece.
Among other things, Babbitt also embraced electronic music, which excited him with its possibilities. And even though his own music was probably the most inaccessible, he also embraced music like jazz, pop, Broadway, and more. To him, different kinds of music deserved to be heard, and was ultimately worthy of respect.
He also taught Stephen Sondheim, Broadway composer and lyricist of such hits as West Side Story, Gypsy, and Sweeney Todd.
The classical music world will miss him greatly.