Once properly initialised, the timers then depend only on one thing; Crystal oscillator. Check if there is anything wrong with your crystal and its coupling capacitors.
Also in your code make sure there should not be a delay in your timer’s ISR which is more than the timer’s firing period itself.
If you are trying to make a watch, its better to use RTC chip.