Customize Styles on Book level - can't switch if off again

In SAB, you can change a Style for a single (or several) books by clicking “Styles” > “Customize styles for this book”. However, I can’t seem to turn off this option again. If I click “Use default styles for this book”, at first the original view (blank window with no options) is restored. But when I later return to the book, the custom options are visible again and appear to be used. This happens on Windows and on the Mac.