Timing
nWhen adding functionality
nDuring debugging
nFor code review
nTo avoid the third strike
There are several prime opportunities for refactoring.

When you are changing the code anyway, you may as well improve it a little, too.

Often bugs are a result of complex code that should be simplified.

If others are goings to look over your code, then it should be as understandable as possible.

Fowler says that someone told him that the first time you do something, you just do it.  If you have to do it a second time, you don’t like it, but do it anyway.  The third time you do something similar, you refactor, probably with an emphasis on reuse.  This he calls the third strike.