Widget talk:Gem Store magic

Two things

1. Why's this called "Gem Store 2" while there's no "Gem Store 1" or "Gem Store"?
2. This gave me an idea but you might be having the same here; i can't quite tell. What do you intend this to do? Nightsky (talk) 18:36, 4 February 2021 (UTC)

If this becomes reality, we should probably also rework how we store the smw gem store information, the copy-paste-subpage-entry-moving-approach is so tedious. E.g. I'm thinking of storing all information (available and not available in the gem store) on one page silently (probably two-four pages to properly handle it and avoid time out, 900+ items) rather than awkwardly spreading to several gem store category subpages and the storage subpage. This could be combined with a timebased approach, allowing up-to-date data and furthermore automatising the gem store history templates on the item pages (problem: backward compatibility, might be tedious). Then the gem store tables themself are created with smw queries. --Tolkyria (talk) 19:27, 4 February 2021 (UTC)

"I'm thinking of storing all information (available and not available in the gem store) on one page silently" We talked about this with DJemba and another user not so long ago actually, and came to the conclusion that the idea is great and would allow easier work with the actual/temporary stuff instead of having xx subpages and having to move it all the time back and forth. DJemba said he can have a look at it in the future, but I just wanted to chim in and say that there are definitely plenty of people who agree with this if you wanna do it. ~Sime 19:47, 4 February 2021 (UTC)

Describing my smw gem store redesign a little bit more: the template could take the following parameters

• the usual parameters item, cost, amt (or qty for quantity might fit better)
• the new parameter date of the format: <start date>,<end date>(, <discount optional>) allowing multiple entries separated with new lines.

This parameter would set the properties Has start date (or Has release date) and Has end date of property type date to allow searching for all available items each day with:

{{#ask: [[Has vendor::Gem Store]] [[Has release date::>Febuary 4 2021]][[Has end date::<Febuary 4 2021]]|?...}} <!-- would create today's available items -->

where we set always available items to a very far away date, e.g. January 1, 2100 to always catch them. Note that a reoccuring item might have several start and end dates.

In order to associate a start date exactly with one end date (and probably with a discount), we would furthermore need to create a record with the properties Has gem store start date and Has gem store end date (and probably Has gem store discount) of type text (to ensure the format year-month-day as we can't force plain format in records, otherwise wiki-language dependent!). Note that sorting the record by the first entry alphabetically (actually the only way we can sort an array), the start date with the format year-month-day, would provide exactly the wanted order.

The only problem is maintaining those, that's where hopefully this widget comes into action by simply taking the currently stored wiki data and adding the new gem store items and their dates; and to recreate all the previous years 2012-2020. --Tolkyria (talk) 20:02, 4 February 2021 (UTC)

Looking even further into this topic, I can say the following: the approach I described above is extremely powerful as it shifts the Is historical property to date properties to dynamically define its historical status and thus heavily relies on the correctly and permanently added dates. Given the correct usage it's always up to date and provides lots of automatically generated information, in contrast to the current weak implementation that always exists whether we update it or not, sure it's outdated and the gem store history has to be created manually on each page (and hence it is outdated soon too) but it does not fail if stop updating it.

As I already said, that's exactly the point, the date approach requires almost all start and end dates from the years 2012-2020 (currently the start or end date parameter of the {{Gem Store history table row}} is set to unknown more than 600 times) and an easy way to ensure that we are able to update in the future. Any difficult or tedious way will ultimately result in destroying the whole gem store documentation, hence for this date-based approach we would need a simple and quick way, somehow crawling and extracting the data from the gem store website.

Summing up: High risk, but incredible high reward.

Some numbers, currently we document:

• 926 items in gem store, 1098 item subobjects stored (due to items sold in different quantities), where 247 are currently available (probably outdated) and 685 aren't and 166 are historical.
• 676 unknown used in {{Gem Store history table row}}.
• 15 discounts are set to ??.

--Tolkyria (talk) 22:37, 4 February 2021 (UTC)

Adding my few thoughts:

• I assume this widget is continuation of our Discord discussion, where it was (finally) discovered that Gem Store data can be found in the game files, making them possible for some sort of automation.
• I'd also love to see this happen. I'll try to help, feel free to ping me if you want something from me.
• "Has start date (or Has release date)" - I am definitely for "start date", I don't want this to eventually mess with story releases
• For backwards implementation, we can try to check older Anet posts, older Reddit Gem store mining posts, eventually we could try old youtube videos if all else fails.

DJemba (talk) 23:22, 4 February 2021 (UTC)

(An extra edit: Looks like all missing discounts are from the 2014 sales, which can be found on Reddit. I'll put that in tomorrow if no one else does before.)

Honestly, I have no idea what discussions are going on at the discord. Well, the template is bascially done, see my sandbox, probably some errors may appear when using it but otherwise it's straight forward. See this sandbox for the raw wiki template usage code.

Regarding the backwards date implmentation: The wiki user Ojimaru excessively spent time into adding all dates, not sure if these dates are there if he/she didn't found it in autumn 2019. That's the tedious part and probably the one that never let's us implement it. --Tolkyria (talk) 23:30, 4 February 2021 (UTC)

For what it's worth the idea i had would have been a gui that would have parsed the wikicode from the individual gem store subpages and presented them in a way that would idealy make it easy to update things before then writing the updated information back. But based on the response below - point five in the first numbered list - this would seem to possibly end up duplicating an update mechanism so i shall refrain from doing anything like that at least untill the "possibly" is clear.

Guess there's lots of other plans already though so count me out. Nightsky (talk) 18:20, 5 February 2021 (UTC)

Go for it, my plan is way to ambitious, unifying the templates gem store table row and the spread out gem store history table row, given that there are around 700 unknown dates and the future permanent required maintenance. Alex' plan has the usual problem that the external names (API, gem store site, whatever) doesn't match the wiki page names, e.g. suffix (container) or (package), making it hard to establish links to the correct wiki pages.

But may I recommend to get rid of all this gem store subpages, declare the subobjects in one page (or two pages to avoid template timeout) and introduce an availability parameter (available, currently not available, historical), handling what the current subpages do. --Tolkyria (talk) 18:55, 5 February 2021 (UTC)

fwiw on the gw2 development discord a potential player-hosted api for gemstore info was mentioned that would mean we'd cut out reading the localstorage file ourselves. -Chieftain Alex 20:27, 5 February 2021 (UTC)

Update: I have discarded the date-approach I mentioned above for now. However, in order to improve the gem store editing I now propose the following: Gem Store/data (see also Talk:Gem Store/data) stores all currently available and unavailable gem store items on one subpage instead of ten subpages as we currently do (the historical gem store items would remain on an own subpage). There the items would be sorted alphabetically by the item name with a status parameter set either to "available" or "unavailable", allowing quicker gem store adjustments (rather than copy-pasting the templates from one subpage of another as we currently do). Furthermore, I think this should favour any future update, as dealing with only one page is so much easier than dealing with content that is spread out to ten pages at the same time.

It would be appreciated if you let me now actively on the Talk:Gem Store/data if this is actually an improvement or indeed a step back, and if you, as community, want to see it implemented. Otherwise, given not enough feedback I have to assume that the current {{gem store table row}}-approach on ten subpages is totally fine for the community. --Tolkyria (talk) 15:27, 6 February 2021 (UTC)

What this page was actually going to be

I was literally using this as a sandbox because I thought Nightsky was still tinkering with the current content of Widget:Test. Here's my code:

removed, see earlier revision https://wiki.guildwars2.com/index.php?title=Widget_talk:Gem_Store_2&oldid=2226717#What_this_page_was_actually_going_to_be

Basically I want sutgon from reddit's code in a widget and I'm willing to play for a few hours. -Chieftain Alex 23:13, 4 February 2021 (UTC)

But yes other than the read the uploaded file bit, everything is still missing. Dunno what to do with the sqlite3 file atm, I just want that json blob - maybe I can search in the data for a certain pattern of encoded braces/brackets + extract the bit between them.

In terms of the content I wanted in the widget:

1. Drop the localstorage file from %TEMP% into the widget.
2. Read the data and extract the json blob to give the current gemstore items start/finish dates.
3. Steal some of the content from Widget:Gem Store catalogue to give me item names.
4. Display a table of what's therefore in the gemstore right now.
5. Possibly at a stretch some form of automatic update for any gemstore data storage pages.

-Chieftain Alex 23:15, 4 February 2021 (UTC)

Ah i see. This tells me my comment here (last parahraph) must have not been clear enough on me stopping trying to do anything with that. Either that or you simply haven't had seen it by then; can't tell. You can replace Widget:Test with this if you want to. Nightsky (talk) 18:20, 5 February 2021 (UTC)

Active logic

Don't currently know how to pull out the list of currently active things, at the moment I've got the logic slightly wrong and it says there's discontinued stuff which is active. -Chieftain Alex 13:05, 14 February 2021 (UTC)

Need to get Silver to add the searchIndex endpoint too (mirror of https://2gemstore.staticwars.com/const/gw2/build111179/GemStore/en/searchindex.560f728d.js), then I can do it fully on this widget.

• For each uuid in searchindex.guid (233 items)
• Confirm uuid exists in the items object (all 233 items exist)
• If a date start or end is provided (226 with start, 40 with end, 7 with neither), respect that date (set as unavailable if too early to start or too late to display), defaulting to available.

That's it. -Chieftain Alex 00:56, 19 February 2021 (UTC)

Got the diff working now, but highlights inconsistencies between the data already on the wiki + the data being spat out by the widget, I will expand what it's doing to provide (A) a table of all items, (B) a table of available items, and (C) the wikitext for all items, (D) the wikitext for new items, (E) the wikitext for the gem store/data diff. -Chieftain Alex 20:47, 23 February 2021 (UTC)

Done. Only hard bit will be doing the first update of Gem Store/data. (the way I've done it at the moment is to ignore prices, this will end up with a discount price and a normal price on the same page) -Chieftain Alex 00:47, 24 February 2021 (UTC)

Bugs

1. My trick to handle suffixed page names doesn't work because although the package might identify the itemid, the actual wiki page is not the item, its for the package. -Chieftain Alex 00:48, 25 February 2021 (UTC)

2. Section autodetection is annoyingly not working as intended for Infused Samurai Package which it thinks is a town service in preference to a style combo. -Chieftain Alex 01:09, 25 February 2021 (UTC)

Ad 1. You should be aware of the type, if I correct, namely either "item" or "package". Couldn't you use smw to ask for the wiki context? So if the type is "package" then you could use one of the following to determine the page name:

1. checking if the <name> wiki context is "Item"

IF {{#show: <name>|?Has context}} = Item THEN name <- name (package) ELSE name <- name

2. directly search for the context "Gem Store combo"

{{#ask: [[Has context::Gem Store combo]][[Has canonical name::<name>]]}}

Both somehow require the existence of the wiki page (and the according infobox) to properly work, where the first one requires only the item page but not the gem store combo page. --Tolkyria (talk) 09:42, 25 February 2021 (UTC)

Missing/duplicate entries

1. Using the cache from today, the generated output is missing 2020 Extra Life Donation Bundle, Synergetics Gyrocycle Roller Beetle Skin, Volcanic Throne, Ultimate Dye Pack, and Home Instance Contract Pack.

2. It is creating duplicate entries for Living World Season 2 Complete Pack and Guild Wars 2: Path of Fire—Deluxe Upgrade. These say "cost = (unknown)", probably because my account already has them.

3. There are also duplicate entries for items that are on sale such as Mad Realm Mounts Pack, but that may be okay. ----BuffsEverywhere (talk) 03:20, 7 May 2021 (UTC)

If you have something unlocked on your account, then it will not appear using your dat file. For 2 for example, do you have PoF + LWS2 all unlocked? I'm not sure

Duplicate items for things on sale was intentional, I thought one data entry for each price might be useful. -Chieftain Alex 06:45, 7 May 2021 (UTC)

When i used my seldom opened NA account, I had the following additional rows:

• Black Lion Garden Plot Deed
• Daydreamer's Finery Outfit
• Frostfire Appearance Package
• Frostfire Glider
• Frostfire Outfit
• Grenth's Regalia Outfit
• Mad Realm Mounts Pack
• Skyscale Ascension Mounts Pack

Removed:

• Item Booster
• Synergetics Gyrocycle Roller Beetle Skin
• Volcanic Throne

Plus a bunch of other status changes where the price was the same but the item isn't available I guess.

Could do with some help fixing the diff function "wikitextMerge" I think. -Chieftain Alex 07:18, 7 May 2021 (UTC)

Perculiarities

What exactly are the percularities? Is the wiki interfearing with js somehow? From what i'm seeing the following, which is a slight variation of the code not far back in the history, should otherwise work:

$('#filelist').on('change', function onChange(event) {
	console.log('File(s) browsed');
	
	// Prevent default behavior (Prevent file from being opened)
	event.preventDefault();
	
	// Use the FileReader API, and bind a function to run when it's finished reading
	var reader = new FileReader();
	reader.onload = function(event) {
		var data = reader.result;
		console.log(data);
		processFileData(data);
	}
	
	// Read it
	var files = $('#filelist').get(0).files;
	reader.readAsBinaryString(files[0]);
});

Nightsky (talk) 22:07, 20 February 2022 (UTC)

Feel free to have a go, I couldn't get this working last time. -Chieftain Alex 22:34, 21 February 2022 (UTC)

I added the above so it "might" work now. Where might is in quotes because it seems to always run into the "No valid leaf table found" error for me, so i can't test if it does now or not (other than; it still gives me the same error as before so it should work for everyone that didn't run into that error before), but it should. Nightsky (talk) 19:22, 22 February 2022 (UTC)