Matt, I have never had that happen on here--a switch from insert mode to overstrike. I didn't even think there was a way to do that, since it always seemed locked in insert mode and has never changed for me. You might try going into your UserCP, Edit Options, then scrolling all the way down to the bottom of the page, to Message Editor Interface. Try changing your editor and seeing if that helps.
About the SpellCheck, it seems to be working fine for me here. Was it just not coming up, or was it not spell checking?
Kevin could probably tell you more about the why's and wherefore's of *that*, since he's the one who broke it.