Bug #695
closedhelp.libreoffice.org is slow
0%
Description
I'm uploading wiki help to help.libreoffice.org. I started it a week ago, and it is still less than halfway done. It is extremely slow, last time it was done in 36 hours, and now, after 168 hours we are still far away from completion. Typical error messages:
Error: 3: internal_api_error_DBConnectionError: Exception Caught: DB connection error: User 'libreofficehelp' has exceeded the 'max_user_connections' resource (current value: 240) (localhost)
Error: 3: internal_api_error_DBQueryError: Database query error
Also, if I go to https://help.libreoffice.org, it takes 30-60 seconds to load a page.
Updated by Alexander Werner over 9 years ago
- Status changed from New to Feedback
Regarding loading speed:
The response time of help.libreoffice.org varies between 60-150ms at my location.
How are you doing the help upload?
Updated by Andras Timar over 9 years ago
I'm using a script, see:
http://opengrok.libreoffice.org/xref/help/upload-wiki.pl
On Sun, Oct 5, 2014 at 3:22 PM, The Document Foundation Redmine
<generic@redmine.documentfoundation.org> wrote:
Updated by Andras Timar over 9 years ago
Upload started: 2014-09-29
Upload ended: 2014-10-28
326543 wiki pages were uploaded, 1.5 GB. This is not the amount that should take a month. I think wiki is low on resources. Uploading 4.2 help took only 1.5 days approximately in February.
Updated by Florian Effenberger over 9 years ago
The load on the machine at this very moment is quite high, at about 8
Not sure if this is a generic problem, as otherwise, we have no
load/performance issues
I assume that with the VMs currently being migrated to our new, powerful
hosts, that problem should vanish soon anyways
Andras, when is the next time a wiki upload is due?
Updated by Andras Timar over 9 years ago
Next wiki upload is due in 6 months approximately.
On Wed, Oct 29, 2014 at 9:03 AM, The Document Foundation Redmine
<generic@redmine.documentfoundation.org> wrote:
Updated by Florian Effenberger over 9 years ago
- Status changed from Feedback to Closed
If there is no urgency, I close this ticket, because in ~2-3 months, the
migration is done in any case. If there is a need before, let me know.
Updated by Andras Timar about 9 years ago
I'm uploading the wikihelp again (master update, 4.4, and 4.3 update). It is even slower than 6 months ago. The last 50 files went up in 1.5 hours. With this pace, it will take half year to finish the job.
Updated by Florian Effenberger about 9 years ago
- Assignee set to Christian Lohmaier
Cloph, I remember we throttled the database for the wiki. Can you maybe
have a look?
Updated by Andras Timar almost 9 years ago
20% done in 2 weeks. 190.000 files to go...
On Thu, Apr 9, 2015 at 9:21 PM, The Document Foundation Redmine
<generic@redmine.documentfoundation.org> wrote:
Updated by Florian Effenberger almost 9 years ago
Alex has raised the limits yesterday, so it should be working - do the
problems still persist?
Updated by Andras Timar almost 9 years ago
It is better now, speed is 4x than before. It is still very slow
compared to last year's performance. Upload in 1.5 months is better
than upload in 6 months, but... You know what I mean. :)
On Tue, Apr 21, 2015 at 11:18 AM, The Document Foundation Redmine
<generic@redmine.documentfoundation.org> wrote:
Updated by Florian Effenberger almost 9 years ago
Cloph, can you completely remove the MySQL limitations for the moment,
or did we do so already? (Not sure whether we lowered or removed them.)
If not, you mentioned something moving from MyISAM to InnoDB or vice
versa. Could this affect the speed of this?
Updated by Florian Effenberger almost 9 years ago
- Status changed from Closed to In Progress
This one slipped through as the status was closed - reopening so we don't lose track
Updated by Florian Effenberger almost 9 years ago
This ticket sadly fell over from Monday's team call, as it was closed;
asked Cloph to have a look when he has time, but might slip to next week
- sorry for that
Updated by Christian Lohmaier almost 9 years ago
- Status changed from In Progress to Resolved
problem solved with the following changes (update is running right now and the db-queries don't skyrocket/the system load is fine and processing the upload is now one or two seconds instead of minutes):
# disable the update of the l10n cache in the api-instance of the helpwiki $wgLocalisationCacheConf['manualRecache'] = true;
This prevents triggering X transactions (that on top are comparably slow) for each update when batch-uploading the new files.
And on the public facing wiki:
# use cdb file based cache instead of db file cache (l10ncache falls back to db if there is no cache directory defined) $wgCacheDirectory = "$IP/cache";
In case it is unacceptable that opening a page on the public-facing page takes a couple of seconds (should in any case be less than 5 Seconds with those changes), a manual refresh of the cache can be done after the import is done.
Setting to resolved.
Updated by Florian Effenberger almost 9 years ago
- Status changed from Resolved to Closed