The base definition of inbox zero is that you should spend as little time (close to zero) managing your mail as possible. It doesn't mean your inbox should always be at zero messages; although doing it right means that you should have zero unread messages in your box.
There are two reasons why you need inbox zero:
An overwhelming inbox can even lead to you deleting messages without reading them, which may then turn out to be important later. Also, a full inbox can result in worry and weigh on your mind. Some people find the little number by their inbox particularly distracting; for those people inbox zero is a particularly important technique.
Achieving inbox zero is not (just) about getting rid of that little number. It's about having your email not be a major part of your day. Paradoxically, one of the ways to achieve it is to keep your email program closed while working on other stuff.
The core of inbox zero is to follow five principles: delete, delegate, respond, defer, or do. So, basically, you engage in a form of triage. The inventor of inbox zero, Merlin Mann, suggests that you do the following:
Nothing gets left in your inbox. You also resist the temptation to deal with the complicated matter in your inbox right away (unless it really is urgent). As an alternative to the top of each hour, you can also open your email client when you have finished a specific task and use dealing with email as a change of pace. Some people find they are able to automate some of the email triage with rules and smart boxes, but this doesn't completely eliminate the need to check your email.
Like many things, it's easy to do inbox zero for a little while, then you're likely to fall back into old habits. Consider setting an alarm for when it's time to check your email. Don't become obsessed with it; if you worry about it, then you're right back to your brain living in your inbox. Also, be aware that it doesn't work for everyone; you don't need to keep it up if it's not helping.
Inbox zero can have a profound effect on your work, with the most important factor being that emails cease to be unplanned interruptions in your day. If you try to deal with every email immediately, as soon as it comes in, you will inevitably end up multitasking, reducing your concentration and increasing cognitive load, which lowers effective intelligence. (This is also why you should close social media while focusing on a task).
Inbox zero also makes it harder to miss important emails and, perhaps paradoxically, easier to respond to emails in a timely manner. Because you set aside specific time to deal with complicated emails, you make sure they get done. Finally, letting email messages pile up in your inbox can eventually slow down your computer.
Inbox zero is one way to master your mail box and improve your productivity. Try it and see if it works for you.