For example if I have a mining base that gives me 5MP per turn. But I am not producing any ships or social projects what happens to that 5MP ? Is it lost forever or is it accumulated for later use ?
Those 5MP are "lost".
Production is turn-based. Each turn you have a certain amount available to you, depending on how you've built up each colony and your spending-ratio. If you don't use the production, then you'll lose it for this turn, but you're also saving the money, that would have been spent to finance the production.
The only difference is, if your colony is not building any planetary improvements, but is constructing ships. In that case, the money you would have spent on social production gets applied to your military production. However, your bonuses from your Military Production ability don't get applied to this money, meaning, that your military production is still lower than if you had funneled all that money into military production directly.
If it is lost forever and assuming that the mining base is a finite resource is it wise to only start mining that base when you have money to use those resources ?
Mining bases provide infinite resources, so you don't to worry about running out. However, the amount of resources you receive gets diminished, the further away from the target planet the base is. It also takes a long time to fully upgrade even one mining base (55 weeks), so you really should start building them early, if you want to make use of them.
I hope this explain it.