Category talk:Pages using DynamicPageList3 parser function

DPL

I want to remove as many of these as possible so we can eliminate DPL as being part of the cause of wiki slowness. -Chieftain Alex 06:38, 21 September 2021 (UTC)

But cluttering Special:WantedPages isn't the solution either. As Template:Ifexists describes: "This template is similar to as it checks the existence of the given page. The difference is that the checked page is not added to Wanted pages.".

So if we set for example {{#ifexists|{{{line}}} (specialization)|{{{line}}} (specialization)|Profession}} }} then at least 15 trait pages (12 major, 3 minor, sometimes 1 proficiency) create the link <line> (specialization), resulting in 45 <line> (specialization) (15 links) and soon 27 <line> (specialization) (16 links) in WantedPages.

What about rewriting the Template:Ifexists instead? The documentation mw:Help:Extension:ParserFunctions##ifexist links to the wikipedia template w:Template:Linkless exists which uses the magic word PROTECTIONEXPIRY instead of a dpl approach. --Tolkyria (talk) 07:42, 21 September 2021 (UTC)

Adjusted Trait infobox by a simple #switch, trait lines don't get renamed, new trait lines are elite specialization which are suffixless anyways.

Added edited templates table below to keep the overview. --Tolkyria (talk) 08:06, 21 September 2021 (UTC)

There are actually two different linkless-ifexists versions using different magic words:

• w:Template:Linkless exists using the magic word PROTECTIONEXPIRY
• Template:Exists using the magic word anchorencode Edit: Transcludes the whole page which 1. is extremely expensive when transcluding large pages and 2. sets SMW properties again.

I haven't figured out yet which one is better, but the first one seems to be used more often. --Tolkyria (talk) 08:49, 21 September 2021 (UTC)

Oh god what a morning fuckup, I had forgotten about the wantedpages list. Rewriting ifexists sounds way superior and far easier to implement. Let me revert my changes from this morning.. -Chieftain Alex 17:04, 21 September 2021 (UTC)

Ifexists updated as suggested. -Chieftain Alex 17:14, 21 September 2021 (UTC)

Since you already brought this up... I wouldn't stop here. I think in some templates the ifexists check is just an excuse for not implementing/specifying it properly. For example:

• Does the template tl really need an ifexists check or can we replace it with {{#if: {{{link|}}}|<!-- set to no -->{{{1}}}|[[Template:{{{1}}}|{{{1}}}]]}}. We could simply set this and track nonexisting (but used as parameter 1) templates with WantedPages and set link=no afterwards.
  There are basically two main usage types:
  1. They are used on template/category pages to describe dependencies, those templates all exist, the check is redundant.
  2. They are used on talk pages to describe potential new templates, there we can easily set link=no. Especially if the idea is dropped and the template will never be implemented (Example: Why do we have to check three years after the abandoned discussion if the never implemented templates exist?).
• Does the template item infobox really need an {{ifexists|{{{skin}}} (skin)|...}} (simplified code) check? For those who haven't seen it yet, I also started a discussion there: Template talk:Item infobox#Skin template code. The same holds for armor/weapon/back item infoboxes. In my opinion being precise isn't hurting anyone, having clear rules how to edit even simplifies it for new editors.

In the end we have to permanently check (on each page rebuild) if a certain page exists just because we have been lazy exactly once, namely the first time we set the parameter. Even if it only slightly improves the wiki speed, I would go for it. --Tolkyria (talk) 18:01, 21 September 2021 (UTC)

Agreed on the tl template, I was looking at that this morning when I was removing some usage cases I didn't like the look of. Tbh, if the page doesn't exist, it shouldn't be linked with tl, but with nowiki.

Replacing the names for the skin pages with the actual skin page wouldn't actually be that difficult, we can run the parameter through the output of a substituted arraymap with something like this I think. -Chieftain Alex 18:19, 21 September 2021 (UTC)

This morning the wiki was at >22k DPL pages, and now its down to 16k, with surely further jobs to process in the job queue. This feels like a really good thing to have done so far. -Chieftain Alex 20:41, 21 September 2021 (UTC)

I was going through the armor consumable skins from the item infobox and cleaned those up, weapons and back items still misisng, but mimicking the current template behaviour with a bot is even better.

The ifexists breaks on something like Nonexistent page/{{{1}}} or Nonexistent page{{!}}1 stating "infinity" internally. To be fair, the old ifexists broke also but by design it was treated as nonexisting page, now it's treated as existing page. I found this in tl, probably not used correctly, so I think we can ignore this; and in the template Babel trying to replace #ifexist with ifexists (Honestly, I don't understand this template, transcluded once, otherwise linked!?, no idea what this is about).

Removed #ifexists checks from the templates League infobox and Collection nav. --Tolkyria (talk) 22:50, 21 September 2021 (UTC)

Down to 5,053 where 4,895 pages should be the template Historical content. --Tolkyria (talk) 13:59, 22 September 2021 (UTC)

I'm not sure the Historical content template is using the ifexists properly anyway. E.g. Aetherized Axe (PvP) is currently tracking the last bot edit as its removal date.

As an aside, 22k to 5k, of which most is from one template? You are both freaking incredible. Greener (talk) 14:35, 22 September 2021 (UTC)

Mostly ifexists. Tolkyria spotting that mw/meta use different versions to do the same thing was pretty neat. -Chieftain Alex 16:09, 22 September 2021 (UTC)

The #dpl on Historical content uses allowcachedresults=true, hence using outdated information by ~1 hour according to the documentation, so it should be less expensive.

But following up Greener's comment: Is it better to have a wrong historical date than no historical date? Not really, for me the cases where the correct date is displayed don't outweight the ones which are simply wrong. Greener's example was probably the combination of page move + page edit (that's I guess how addfirstcategorydate=true interacts with title={{PAGENAME}}, the page name obiviously changed in the move process and thus newly added to the category). Page moves are a normal wiki process, old historical pages get moved while new pages take up their place, so having a function that cannot take this into account is kinda missing its purpose. But one can go even further in this example: with a creation page date in February 2017 it never had a chance to get the actual date, namely April 2014, see PvP Locker.

Summed up: Should we get rid of the #dpl date check in the template Historical content and only rely on the manually set parameter {{{date|}}}? --Tolkyria (talk) 09:14, 23 September 2021 (UTC)

I think the odds of us having added the category on the right date are fairly slim, i'd support removing the dpl from the historical content template. -Chieftain Alex 20:55, 23 September 2021 (UTC)

No objections so far, thus I removed the #dpl historical date addition from historical content, down to 375 pages now plus 40 pages in Category:Pages using DynamicPageList3 parser tag (<DPL>) which are now game updates (necessary), pet stats and many user pages (for example pre smw skill tables).

Maybe we should also have a look at Category:Pages using DynamicPageList3 dplreplace parser function where I think most of the #dplreplace are used:

• in the infobox descriptions to get rid of the sic categorization, namely {{#set:Has game description={{#dplreplace:{{{description|}}}|\[\[Category:Text errors\]\]}}}}. However, I have no idea yet how to avoid this replacement, maybe define a global #var:property declaration and which is checked in the template sic and omits it there.
• in sold by to get rid of language-based thousand separators as a number property stored in records can't be obtained in plain format, see {{currency|{{#dplreplace:{{#explode:@@@|}}|[^0-9]*}}<!-- remove anything that isn't a digit -->. Theoretically we could avoid this by changing the property type from number to text and hence avoid any thousand separators by default (although the input doesn't allows anyone either, which the number property don't care about).

--Tolkyria (talk) 13:27, 27 September 2021 (UTC)

Maybe sic shouldn't categorize and we use something like:

<div class="smw-ul-columns" style="column-count:4;">{{#dpl:uses=Template:Sic|ordermethod=title|mode=category}}</div> to create a "category" of pages that use the template. I know it's ugly but it'll cut down on the replace function. --BuffsEverywhere (talk) 15:09, 27 September 2021 (UTC)

• Game updates - could potentially make that SMW. We would however have to "property up" the game update pages (or add more categories). Would have to add a template to each game update page too if we chose to add properties/categories.
• Template:Time - I believe the non-cached property this used to be used for was removed several DPL updates ago, so any occurrences of this could be replaced with #time: if required for the time value. Could be removed completely if only present for the non-cache ability.
• Template:Sic - dplreplace is purely to stop any smw lookups on the erroneous content from also appearing in the same category (e.g. ask for skill descriptions on a profession page). Tricky. I like being able to add the sic template to bad descriptions, but I think that removing the category is probably not that intuitive based on how we do everything else on the wiki. Not sure how we could avoid that, other than expressly asking people not to include sics in description parameters + to use a separate template in the #Notes section.

-Chieftain Alex 16:04, 27 September 2021 (UTC)

Wait, while I claimed that I have no idea yet, basically everything is there, no need to go wild.

• Game updates - {{#ask: [[Category:{{PAGENAME}} updates]][[~Game updates/20*]]}} does the job (arraymap + mainspace transclude or result format + mainspace transclude).
• Sic - Set in infobox: {{#vardefine:property declaration|true}}{{#set:Has game description={{{description|}}}}}{{#vardefine:property declaration|false}} and in sic: {{#if: {{NAMESPACE}}{{#ifeq: {{{categorize|y}}} | y || do not categorize }}{{#ifeq: {{#var:property declaration}}|true|do not categorize}} || [[Category:Text errors]] }} should work in theory unless some wierd cache interaction bug decides that it doesn't work (plain sandbox test says that it works).
• Time - should be okay, idk yet.

--Tolkyria (talk) 16:36, 27 September 2021 (UTC)

(Reset indent) Created Template:Pet table to remove some inline dpl, but it'll need some SMW properties setting to replace with something useful. -Chieftain Alex 10:36, 28 September 2021 (UTC)

Now updated to SMW, slight shortfall in that we don't quite document the skills/pets fully but its workable. -Chieftain Alex 12:43, 28 September 2021 (UTC)

Sic refuses to receive the variable value defined in the infobox, no idea how to replace the #dplreplace there. --Tolkyria (talk) 15:51, 28 September 2021 (UTC)

Maybe I found a solution for sic. I think due to caching issues, we can't define a variable in the infobox and then use it in sic that is used in an infobox parameter description; but we can define a variable in sic and interact with this variable in the infobox. Namely, define {{#if: <!-- namespace/categorization check -->||{{#vardefine:sic cat|[[Category:Text errors]]}}}}{{#var_final:sic cat}} and use it with #var_final at the end of the page creation. Thus, in the infobox we can avoid category declaration in the game description (and hence in the property Has game description) by setting {{#vardefine:sic cat|}} and categorizing it at the same time directly. In all other cases where no infobox is involved the categorization happens at the end.

However, is this really better than calling #dplreplace to get rid of the category? --Tolkyria (talk) 18:17, 28 September 2021 (UTC)

FYI, the new smw monthly game update approach causes a template time due to /2020-02-25 (of course it had to be this game update... after all this time it's still haunting me), tuning down game update icon to the core functionality solved it.

Furthermore, not sure what we did, but wiki is extremely slow for me. --Tolkyria (talk) 21:06, 29 September 2021 (UTC)

As suggested by Tolkyria above, I changed the property type of Property: Has item value which allowed the removal of the dplreplace call in sold by, and this reduced the size of Category:Pages using DynamicPageList3 dplreplace parser function from over 15k to 4.4k. --BuffsEverywhere (talk) 08:15, 30 September 2021 (UTC)

Edit: Regarding sic, perhaps this will work? {{#replace:{{#replace:{{sic}}|[Category:Text errors]|}}|[]}}. --BuffsEverywhere (talk) 08:58, 30 September 2021 (UTC)

That would probably work; though it might not be ideal for the reason outlined here. Nightsky (talk) 16:43, 30 September 2021 (UTC)

I think you mean {{#replace:{{#replace:{{description}}|[Category:Text errors]|}}|[]}}, how/why should we replace something from {{{sic}}}? Using the mw parser function #replace obiviously does not work due to the 1000 character cap: Sic alone has already ~200 characters (e.g. 4 sic calls together with some text); there are item description novels with ~1500 characters. As I already said, the best would be to disable the categorization in the infobox description while keeping it everywhere else. If we don't add it, then we don't need to replace it. --Tolkyria (talk) 17:40, 30 September 2021 (UTC)

Yes that's what I meant but I didn't know that #replace has a character limit. It's not documented! --BuffsEverywhere (talk) 18:19, 30 September 2021 (UTC)

Changing Property: Has item value to text caused search operators ">" and "<" to work on it like a string - no longer greater/less than but an alphabet order (so 574>42000 for example), effectively breaking some vendor templates. Can't think of any way to fix other than revert, any other suggestions before that? DJemba (talk) 22:11, 30 September 2021 (UTC)

Edit: Wouldn't formatnum:<>|R work to get rid of separators instead? DJemba (talk) 22:16, 30 September 2021 (UTC)

the fun bit about semantic mediawiki is depending on the language you view the wiki in, the separator automatically changes and this interacts differently with the parser functions for calculations (Hence the R option of the formatnum function doesn't play nice for the different types of possible separator). Out of interest, are there any useful times when we'd want to specify a minimum or maximum quantity for currency? -Chieftain Alex 22:30, 30 September 2021 (UTC)

Pages such as Chisel use it in {{Vendor query table}} to specify the items, + it doesn't have to be necessarily max/min, just finding items cheaper than X or ordering them based on the price would now be a problem. DJemba (talk) 22:34, 30 September 2021 (UTC)

That doesn't seem like a particularly good way of querying it. Can't we just query by item name or id? e.g.

{{vendor query table|other=[[Sells item.Has canonical name::Chisel]]}}
{{vendor query table|other=[[Sells item::Chisel#Fine||Chisel#Masterwork]]}}
{{vendor query table|other=[[Sells item.Has game id::23692||23698]]}}

for sorting, I suspect that Guild Wars 2 Wiki talk:Requests for technical administration#Sorting pages with numbers might fix that such that we can still sort the text numerically (not 100% sure on that)

fwiw I dug out the 2016 bug report. -Chieftain Alex 22:57, 30 September 2021 (UTC)

There's already a parameter that queries for all subobjects sold by vendors, use {{vendor query table|item variant={{subst:PAGENAME}}}}.

Might be wierd if smw:Help:$smwgEntityCollation set to numeric wouldn't address this problem. --Tolkyria (talk) 23:06, 30 September 2021 (UTC)

Ifexists

I have just realised that the version using PROTECTIONEXPIRY counts as an expensive parser function, and thus only provides 100 valid results on a page. I am now tempted to swap it for the other version Tolkyria linked, i.e. {{#ifeq: {{anchorencode:{{lc:[[:{{{1|defaultFalse}}}]]}}}} | {{anchorencode:{{lc:{{:{{{1|defaultFalse}}}}}}}}} | {{{3|}}} | {{{2|}}} }}. -Chieftain Alex 16:56, 27 September 2021 (UTC)

Don't do it now, further tests are required, but right now after I preformed some tests this approach seems to be super fishy. I think it checks if the linked page is equal to the transcluded page (which is only the case when the page doesn't exist). But since it transcludes the page in this process:

1. it gets really big, I ifexists-checked 20 large pages and got Post-expand include size: 2,097,151/2,097,152 bytes and thus a template timeout.
2. this is even not enough, the transcluded pages sets smw properties again!

Try {{#ifeq: {{anchorencode:{{lc:{{:Meteor Shower}}}}}} }}

Abort! Abort! Abort! --Tolkyria (talk) 18:43, 27 September 2021 (UTC)

Comparison:

• #ifexist — expensive parser function, limited to 100 calls, creates Special:WantedPages entries.
• #dpl — expensive, we want to get rid of this, however the question is how expensive is this compared to the others.
• PROTECTIONEXPIRY — expensive parser function
• anchorencode — extremely expensive in certain cases, sets SMW properties again
• ... — is there anything cheaper?

So far for me the conclusion of this is that an ifexists check is always expensive. The best would be to completely avoid it if possible by shifting the responsability to the wiki editor. While it may be a little bit more work for the wiki editors, they have to perform this check exactly once, in contrary to the template which has to check it on each page rebuild over and over again. --Tolkyria (talk) 08:18, 28 September 2021 (UTC)

Lol metawiki garbage templates. Good spot and glad we didn't randomly choose that one.. -Chieftain Alex 08:54, 28 September 2021 (UTC)

I can't tell how (in)?expensive it is but may i suggest {{#ifeq: {{#ask: [[<!-- Pagename. -->]]|format=count}} | 0 | <!-- If NOT exists. --> | <!-- If exists. --> }}.

The question is how much do we trust SMW 3.2.X. There was a time before SMW 3.0.0 where we couldn't rely on the count query to return the correct result (most likely due to outdated entities not being deleted). Furthermore, each query creates an entry (sets the property Has query, check Browse properties in your sandbox where you have tried it). Currently ifexists is used on ~13,000 pages which would mean 13,000+ new query entries.

One could even reduce it to {{#if: {{#show: {{{1|}}}|default=fail}}|<!-- does not exist -->{{{3|}}}|<!-- exists -->{{{2|}}}}}, but then I again I have experienced that a query for a nonexisting page returned an empty value rather than the default value, as the query itself after saving it once somehow defined the non-existing page internally (I think this was most likely before SMW 3.0.0).

Good idea, but I think a pure mediawiki approach is safer. --Tolkyria (talk) 16:29, 28 September 2021 (UTC)

I don't know about trusting it in this regard but i didn't realize it would create a query everywhere it's used. And 13,000+ additional queries don't sound desirable.

And it's sad if smw is truly as unreliable on this as you mention.

I agree, it's probably safer. Nevermind my suggestion. Nightsky (talk) 17:05, 28 September 2021 (UTC)

Edited templates

Template	Old code	New code	Notes
#dpl related edits
Template:Ifexists	{{#if: {{ #dpl: skipthispage = false | redirects = include | title = {{{1|}}} | noresultsheader = no page found }} | {{{3|}}} | {{{2|}}} }}	{{#if: {{PROTECTIONEXPIRY:edit|{{{1|}}}}} | {{{2|}}} | {{{3|}}} }}	Copied from w:Template:Linkless exists.
Template:Historical content	{{#dpl:skipthispage=no|title={{PAGENAME}}|category=[..]| addfirstcategorydate=true [..]}}	-	Removed.
Template:Pet table (new)	{{#dpl: | category=... pets | ... }}	{{#ask: [[:+]] [[Has context::Pet]] [[Is historical::N]] {{#if: {{{1|}}} | [[Has pet family::{{{1}}}]] }} | ... }}	Individual direct usage replaced with a templated "pet table" template, and then converted the template to smw.
#dplreplace related edits
Template:Sold by result format Template:Vendor query table result format Template:Collection table [[Template:Currency for result format]] Template:Armor vendor list result format	{{currency|{{#dplreplace:{{#explode:@@@|(}}|[^0-9]*}}|{{#explode:@@@|(|1}}}}	{{currency|{{#explode:@@@|(}}|{{#explode:@@@|(|1}}}}	Property:Has item value type was changed from number to text to get rid of thousand separators (user language dependent, see w:Decimal separator) in order to avoid replacing it afterwards; note that in a record the plain format smw operator "#" does not work.
Parser function #ifexist and template ifexists related edits
Template:Collection nav	{{#ifexist: {{#var:achievement-page}} | {{#if: {{#show:{{#var:achievement-page}}|?<property>}} | [..] }} }}	{{#if: {{#show:{{#var:achievement-subobject}}|?<property>}} | [..] }}	I can't see any reason why an ifexists check is requried for an achievement subobject.
Template:Item infobox	{{#arraymap: {{{skin|}}} |;|@@@|{{ifexists|@@@ (skin)|{{cname|@@@ (skin)}}{{#set:Has skin=@@@ (skin)}} [..]|{{cname|@@@}}{{#set:Has skin=@@@}}|<br>}}	{{#arraymap: {{{skin|}}} |;|@@@|{{cname|@@@}}{{#set:Has skin=@@@}}|<br>}}	Run through all skin parameters and added the suffix (skin) if necessary, removed ifexists.
Template:League infobox	{{#arraymap: {{{organizer|}}} |,|@@@| {{ #ifexist: @@@ | [[@@@]] | @@@ }} |<br />}}	{{#arraymap: {{{organizer}}}|;|@@@|@@@|<br>}}	Just link it directly on the few pages that use it.
Template:Trait infobox Template:Trait infobox/historical	{{ifexists|{{{line}}} (specialization)|{{{line}}} (specialization)|{{{line|Profession}}}}} }}	{{#switch: {{{line}}} | Arcane | Corruption | Retribution = {{{line}}} (specialization) | #default = {{{line|Profession}}} }}	Trait lines don't get renamed, new elite specialization trait lines are suffixless anyways.