Project

General

Profile

Actions

Task #3440

closed

Test workaround patch for fixing wiki subpage moving memory use explosion

Added by Beluga Beluga about 3 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Normal
Category:
Wiki
Target version:
Team - Q1/2021
Start date:
Due date:
% Done:

0%

Tags:

Description

Currently we can not move (rename) any translated page alongside with its translation subpages. The memory use goes through the roof.

There are a bunch of related Phabricator tasks for this issue:
https://phabricator.wikimedia.org/T20585
https://phabricator.wikimedia.org/T260812
https://phabricator.wikimedia.org/T268840

Niklas has a workaround patch that we might test against 1.35.1:

commit f6b28f090f81b513a54387e4a20f5551b8251aef (HEAD -> master)
Author: Niklas Laxström <niklas.laxstrom@gmail.com>
Date:   Tue Dec 1 08:57:50 2020 +0100

    Workaround for T268840

diff --git a/includes/deferred/DeferredUpdates.php b/includes/deferred/DeferredUpdates.php
index 7f506bfeaf..b62c0282ac 100644
--- a/includes/deferred/DeferredUpdates.php
+++ b/includes/deferred/DeferredUpdates.php
@@ -495,8 +495,8 @@ class DeferredUpdates {
                if ( self::pendingUpdatesCount() >= self::BIG_QUEUE_SIZE ) {
                        // If we cannot run the updates with outer transaction context, try to
                        // at least enqueue all the updates that support queueing to job queue
-                       self::$preSendUpdates = self::enqueueUpdates( self::$preSendUpdates );
-                       self::$postSendUpdates = self::enqueueUpdates( self::$postSendUpdates );
+                       // self::$preSendUpdates = self::enqueueUpdates( self::$preSendUpdates );
+                       // self::$postSendUpdates = self::enqueueUpdates( self::$postSendUpdates );
                }

                return !self::pendingUpdatesCount();

This patch is hoped to be the proper fix, but it only applies to master for now, so let's ignore it: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/596082/

For testing the effect on perf, one could try renaming this page as we used it in our previous test:
https://wiki.documentfoundation.org/Videos/_Convert_color_code

Actions #1

Updated by Guilhem Moulin about 3 years ago

  • Priority changed from Urgent to Normal
Actions #2

Updated by Guilhem Moulin about 3 years ago

  • Status changed from New to Feedback

Beluga Beluga wrote:

Currently we can not move (rename) any translated page alongside with its translation subpages. The memory use goes through the roof.

Many thanks for the follow-up with upstream!

For testing the effect on perf, one could try renaming this page as we used it in our previous test:
https://wiki.documentfoundation.org/Videos/_Convert_color_code

Tried to move these 155 pages on the test wiki before and after applying Niklas' patch, using runJob's default memory limit. Before the patch the job was aborted due to memory exhaustion; trying again after applying the patch the pages were properly moved AFAICT. \o/

I did not apply the patch to the production wiki yet. Is it suitable for inclusion in 1.35.2? :-)

Actions #3

Updated by Beluga Beluga about 3 years ago

Guilhem Moulin wrote:

Beluga Beluga wrote:

For testing the effect on perf, one could try renaming this page as we used it in our previous test:
https://wiki.documentfoundation.org/Videos/_Convert_color_code

Tried to move these 155 pages on the test wiki before and after applying Niklas' patch, using runJob's default memory limit. Before the patch the job was aborted due to memory exhaustion; trying again after applying the patch the pages were properly moved AFAICT. \o/

I did not apply the patch to the production wiki yet. Is it suitable for inclusion in 1.35.2? :-)

Great! Let's keep our fingers crossed for https://gerrit.wikimedia.org/r/c/mediawiki/core/+/596082/ to land soon.

Actions #4

Updated by Beluga Beluga about 3 years ago

Niklas commented on IRC that we probably need to wait until 1.36 to get the real fix. So applying the workaround to prod starts to sounds like an attractive proposition.

Actions #5

Updated by Guilhem Moulin about 3 years ago

Beluga Beluga wrote:

Niklas commented on IRC that we probably need to wait until 1.36 to get the real fix.

We're following LTS releases and will skip 1.36; for us it probably mean waiting another 2-ish years.

So applying the workaround to prod starts to sounds like an attractive proposition.

I read that as a request and applied the change :-) Please close this after a little while if you spot any regressions, and shout out if you do.

Actions #6

Updated by Beluga Beluga about 3 years ago

  • Status changed from Feedback to Closed
Actions

Also available in: Atom PDF