I have 100Gb mysql db. Trying convertation to postgre, but it takes about forever.
Total 219283353 chunks, in first 30min about 200-400k done with ETA 2days, then it slow down and ETA became 20+ days. So no way to complete this. Traffic on VM ~50Kb\s.
Tried default chunk size and 100Mb with --chunk-size key.
Is it works as intended or more looks like bad perfomance\memory leak?
next 27, mariadb 10, postgre 15.
not sure if the following can help you, but if it is not foreclosed that postgresql settings could be the bottleneck
may have a look here
don’t think so, PG is already tuned. Tested with single node and cluster installation.
have you seen this in the forum here?
That seems like an absurdly long time. You sure it would have actually taken that long and that it wasn’t just a bad estimate?
And if you don’t mind my asking, what’s the motivation for switching to postgresql?
That looks interesting, thanks.
3rd party migration probably my only way, since i’m not only one have problems with occ perfomance.
--chunk-size option expects an integer value. It defaults to
1000 but you might try
10000. If you specified
100Mb I think it would have been cast to
100 further slowing things slow (I would expect anyhow).
What does the CPU or Disk I/O utilization look like on the app server and the db servers?
hm, i found somewhere that
--chunk-size can be 1-100 with default 10.
Any resources utilization is about zero.
I’m looking at the code.
It defaults to
'the maximum number of database rows to handle in a single query, bigger tables will be handled in chunks of this size. Lower this if the process runs out of memory during conversion.',
You might be mixing up the term “chunk size” with the one used for WebDAV file transfers. That’s completely unrelated.
Across all servers involved? Can you provide output from
htop and/or at least the process statuses from a
That only one key that required for this method and not exist at official doc)
Tested with chunk size 10000.
Before calculation of
oc_filecache table, with default size i wait for about 5min. Now it takes about 20min.
First time failed by timeout, 2nd started.
Looks much better, but still very slow. Also process a bit slow down after first million.
chunked query, 21929 chunks
2370000/219283353 [>---------------------------] 1% 47 mins/3 days
Utilization also not high.
postgre process often go to idle status, between chunks.