It depends what you mean by learning Japanese. If it means gaining some conversational ability, then I believe most people would rate it well. If it means becoming profecient or near native level, then I doubt Duolingo could help you do that on it's own.
What exactly are your goals? If I want to work in an English or multi-lingual environment and learn some Japanese to help with daily life, Duolingo might get you there. If you want to become proficient in the language, work in a Japanese-only environment, and be able to deal with banks, real estate agents and doctors yourself, you'll probably want to study at a university or language school, or at the very least self study with some actual textbooks.
I'm using it and have been in Japan in Dec. It got me started and provided a good platform to work from, but I also immersed myself as much as possible, by watching japanese TV and listening to podcasts,... And also used other resources.
I got by, but was far from understanding everything/talking fluid and think that without the immersion it would have been even worse.