After I finish a Tails session and shutdown my computer, is it okay for me to boot to Windows and use a regular browser?

Yes

Would this compromise my anonymity in any way?

No

Would it perhaps be better for me to unplug my modem and plug it back again before using the internet with Windows?

No

On a related note, is it necessary to keep your regular HTTP/HTTPS browsing activities and your Hidden Service (as a user) activities separate?

That's one hell of a complex matter. All manner of browser based and network based attacks are possible. Either read up on Tor until you can answer it yourself of take is as "yes"

You can find vastly more comprehensive answers to your questions in the (excellent) documentation.