As shown in this diagram, are brown dwarfs are not only far away from the star, but also partly in less than an Earth distance (1 AU). However, there are hardly any brown dwarfs because they lose gas in each cycle, then that the star receives.
You will thus be swallowed by little from their star.