Refactoring is a process that does not bring any value to the product.
You should consider doing refactoring when:
- product is in active development state
- adding new feature require more time as planned
- you can spend extra time for refactoring
If you have checked all three points you can begin refactoring.
At first do following things:
- write how product should be designed
- create a criteria when you new code is good enough
- make an estimation how much time it would take
If you can determine time that would be spent on refactoring or this time is enormously big, then you better to write a new product.