I did it twice and it worked out fine. In my first start-up, everyone said the technology was impossible, and the market real, but too small. The second time, everyone said no one needed it.
But … you need to go a little deeper than that.
A key thing in both of these cases is IF the product really worked AND there were changes to the market (which we could already see coming), THEN the start-up thesis did make sense with real, live customers we knew and interviewed.