Although on this site I focus more on techniques to improve reading ability, since in the case of Japanese reading is a greater barrier to fluency than speaking, there are a lot of decent ways to improve your speaking and listening ability even if you aren't in Japan.

My main recommendations in this area include using podcasts, for listening practice, along with Skype as a way of having conversations, and then the Pimsleur CDs for some specific speaking and listening practice.