0
\$\begingroup\$

Does any body know how the next change ID increments on the POE API?

I think I have figured out the the API returns the latest, system wide next_change_id for the latest stash changes.. but in between refreshes the incrementations are a bit all over the place.

So these are the change id's reported within seconds of updates

97021553-101802592-95624331-110373614-103027614
97021579-101802666-95624405-110373692-103027683
97021579-101802668-95624405-110373695-103027694
97021579-101802668-95624405-110373710-103027694

It looks like a GUID but it is not a GUID by any means.. when the game started these numbers where allot lower.

I have tried searching forums for a more in depth breakdown of how the API works. The game site has limited information, Reddit has some insight and source code links.. but no proper break down.

\$\endgroup\$
  • 1
    \$\begingroup\$ pathofexile.com/developer/docs/api-resource-public-stash-tabs and pathofexile.gamepedia.com/Public_stash_tab_API You don't generate those ids, you just append them to the URL and get the next set of changes and a new id. Rinse repeat. \$\endgroup\$ – ElDuderino Sep 20 '17 at 15:51
  • \$\begingroup\$ I said I already read the games site version of it.. and has limited information "as in insight" to the mechanics of the change ID. I want to know how far away from the latest change ID I am. The API downloads the change in 1000ms or so.. and I have been doing that for 2 days and not hit real time yet... and I started at the latest id from poe stats. \$\endgroup\$ – Piotr Kula Sep 20 '17 at 19:07
1
\$\begingroup\$

I don't have any hard evidence but from tinkering with the change id myself I came to the conclusion that it represents 5 different ids. I assume they use 5 different databases/shards (maybe one for each region?) and each has its own identity column. As already mentioned you only need to set this as an initial value if you want to write a crawler. After the first request you should use the value that was provided with the response.

\$\endgroup\$
  • \$\begingroup\$ That is very interesting.. that was kind of in the back of my mind that so its interesting you say that too. The thing probably what I should have said I would like to know how close to live I am possible. I have been downloading for 3 days now.. and not hit the end (blank page) ID yet... Am I behind.. am i very far behind? Can I create several workers to download the data? its a mystersy? :D \$\endgroup\$ – Piotr Kula Sep 20 '17 at 19:05
0
\$\begingroup\$

So this is not officialy documented any where but from what I have managed to gather is that the "number" is divided into five parts. Each part belongs to a shard. for example, US,EU,JP...

Each number is actually the number up updated "public and/or private" item changes recorded on the shard.

97021553-101802592-95624331-110373614-103027614

So shard[0] as some people like to call it has seen 97million and a bit item changes.

You can actually just get data from each shard by setting the other shard numbers at extremly high numbers.

0-9999999999-9999999999-9999999999-9999999999

will return a set of stashes and items ONLY belonging to shard[0] and the next change ID is

2524-9999999999-9999999999-9999999999-9999999999

Which means THIS request returned 2542 items from the start of the game, and on that next ID you get the next DELTA SET of data.

4888-9999999999-9999999999-9999999999-9999999999

This subset managed to return 2364 items for shard[0] (4888-2524) which is a similar data set size.

Now the normal API returns a number that looks a bit strange and is hard to follow because the API fills a buffer of about 5mb of uncompressed data, then calculates the new number just for your call, based on all the data it could get from ALL the shards. You then need to follow your change id's get the next set of data in sequence.

If you call a number that is over the amount of items seen on all shard you just get an empty response, with that change ID and no items. This is possible, if you can manage to catch up to real time.

But that is only possible if you do some clever maths your self.

And the maths is what makes sites like poe.trade able to det data indexed within seconds and is their "secret sauce"

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.