I do have a good grasp on casual polite speech and not sounding like a ten year old thug though. I'd be completely lost in a very formal setting, but can communicate with normal people pretty well without coming off as sounding like a rude tween. Generally Japanese people understand that foreigners who speak their language don't necessarily 'get' formal speech. They'll correct someone who is speaking in a rude way and put up with it pretty well if the person seems unaware, and will usually give them the benefit of the doubt unless it's obvious they're trying to be a jerk.
Mostly, they accept semi-formal casual as polite when dealing with foreigners. It's how we usually learn the language to begin with as it's the most commonly used format for daily life.
I have an American accent that I'm told is not nearly as heavy as most Americans and am better off than most who learned by way of language courses. I have been told I'm actually easier to understand than most foreigners because of how I enunciate everything. I sound a little odd, but it's not a bad habit because it forces me to speak clearly.