Firstly, thanks for this nice addon. As a completionist and "list" geek, I am sure I will be using this a lot. Also, I like its interface and sorting feature. Just one thing though, is there a possibility to reduce the time it takes to make searches? It takes more than 20s to do a search. I noticed it did an initial scan at the beginning which crippled the game which is fine for once but having to wait 20+ seconds for every search is kinda annoying. I tried another addon, Ludwig, much simpler but it generates the item lists immediately when you select the category. Must be related to how the data is handled but I hope that something can be done for ItemDB to increase its search speed.
Another suggestion: It would be nice to add "All" option in the Rarity Filter to select/deselect all rarity options.
There is indeed a huge difference in how ItemDB and Ludwig handle the data. Ludwig comes with a huge precomputed database which it seems to gather from wowhead. ItemDB in contrast uses the data stored in the client. Unfortunately, the way how that data is stored and accessed has changed over the years and calling the API for over 80k different items in a short time can mess up the client. :/ As a result, I throttled these API calls "a bit", which results in the slow initial scan that causes the client to load data from disk/server. The search itself has also been throttled since 6.x to prevent the client from hanging. However, it seems like by now things are running a bit smoother again, so with the last commit, I undid some of that throttling and search should be 10-25 times faster ;) I'm now checking 2500 instead of 100 items per frame, but that could drop frame rate a bit. Also it could break in combat with "script run too long"...
And while I was at it, I also added your suggestion regarding the rarity filter.
Thanks for the detailed reply! My thinking was that the initial scan was getting the info from the client which generated a static database, then the searches were taking the info from that database. And since Ludwig also has a large number of items in its static database (without client interaction but a large database), I wondered if ItemDB's search could be made faster.
What I would ideally do is to give the user the option to do scans manually when the user wants, not every time the user logs in. And keep the scanned item database in a local static database as Ludwig does. Then do the searches normally which may speed up things a lot. I don't think items are changing very frequently, especially a while after expansions or major patches. So, I would prefer to be able to do the scan when I want, and then use the generated database (which is stored in the addons or WTF folder I don't know which way it should be done) and use that static data till I do another manual scan. In that case, for example, I would do the scan only after patches etc.
Anyway, thank you very much for your continuing effort on this and I will be looking forward to your next update.
The main problem is that ItemDB uses a lot more data than Ludwig and thus doesn't maintain its own database but uses client API to access this data. Unfortunately Blizzard changed how this API behaves over the years and thus the initial scan is mainly used to force the client into slowly loading all item data (which would otherwise be done as required). Quickly loading all this data at once doesn't work as it causes the client to stop loading data at all at some point thus also breaking general functionality in other parts of the UI. Furthermore, there are two kinds of items regarding this loading process. From the more than 80k items, a large amount is now stored in the client, only a few thousand have to be loaded from server. Unfortunately, the server will not always return data for those, only if there was previously some legitimate reason to load it, e.g. someone looted the item or has it in his bags. It's a complicated situation and I've been struggling a lot to keep the addon working despite all those changes from Blizzard, while maintaining the initial design of not having a static database to work on. Keep in mind that the addon exists since vanilla, when Blizzard introduced the GetItemInfo API, so I don't intend to change that part.
In theory, it would be possible to cache the data for all client side items, sure, but even with a compression that would allow for individual data access, like Ludwig has it, the amount of data would be huge. Remember that Ludwig neither has item level nor vendor value or the data used for the advanced filters.
I hope this is not abandoned moving into Warlords of Draenor. It works great, and is vastly superiour to AtlasLoot (Which locks up the client when searching!)
I've already recommended this to everyone I know, and they love the addon. They've said its just like using the Auction House, except we can see literally every item that is in the game.
The only issue that appears to come up is the lack of a Minimap button, i'm not sure if this is a bug in the current version? Leaving the only way to access this addon via manually typing in /itemdb which was not stated anywhere here.
Since I just updated last month and don't intend to not get WoD, no plans to abandon, either ^^
ItemDB used to lock up the client until recently, too. There's lots of work to be done for a search and it seems like Blizzard intentionally slowed down some functions if you call them too often...
Regarding the missing minimap button, ItemDB comes with a data object for use with any display that supports LibDataBroker. Since I have no options so far, I was just too lazy to add a minimap icon so far ^^' Maybe if I find some time... ;)
I just pushed a new version to beta status, see "Other Downloads" above or WoWAce. This new version is working with 5.x but takes a few minutes to be ready (data object's tooltip will say "status: READY" and search/reset buttons will be enabled). This is due to the items having to be queried from server almost every session. But on the bright side, this now gives access to ALL items :)
Well, by now I know some more again about stuff is handled now then I did when I wrote the last post. A large part of the items are stored client-side to begin with but will not be loaded unless queried (~61k). The rest of the items is requested from server and requires that those have been loaded by the server during its current session (eg due to dropping, a player owning them, ...). Unfortunately, it seems like scanning for those can put your client in a state where it wont receive any more item data from the server, at least temporary, which can cause unlootable quest items. Still looking into how to fix that (interface reload and looting then works though).
Looks great! Only thing is it gums up the game while scanning. Any way to make the scanning a manual thing and maybe cache the data in a saved variable until the user says rescan?
I'm still messing with the values in my local version to get things running smoothly. Like I said in the other comment, I'm still working out how exactly the client does some things, mostly due to hardly anyone outside Blizzard knowing :/ Also note that it's not really scanning. It's causing the client to load the data from disc/server into the cache so it can be used e.g. for tooltips, filtering, ...
Are there plans to continue this addon? I loved having an alternative to AtlasLoot when digging around for upgrades ("Where can I upgrade my shield" is often more relevant than "what drops in this dungeon") and the news about transmogrifying gear is going to make this addon or something like it even more useful.
I seem to remember hearing that all*(read 95-99%) of the ingame item data is now stored locally as of the 4.0.3 patch. I have not looked into this myself but if it is true this should help you with a work around for the onload issues. I came across this information in an article about the new Atlasloot for 4.0.3. Another great addon. I hope this helps I miss having your addon on my LDB bar ready at a moment's notice to help answer my guildie's questions.
don't use this until it is fixed for 4.0.3a... even though I was resulting in no bugs when it would load according to bugsack with !buggrabber...
it would sometimes result in a disconnect when loading the character or after it loads you wait 2-3 seconds and bam you get disconnected from the game... over and over and over.
I checked my errors folder. nothing I checked the framesXML to see if it was spitting any errors... nope wasn't.
on a 5.9 rating scale if you want a hard number for windows rating system... I'm not saying its a bad addon, I would of loved it if I was able to use it but at its current version state.. it simply doesn't support 4.0.3a patch.
it will take me some time to fix ItemDB since it seems Blizzard changed the functions to access the cache so it now takes way more time to scan over it :/ So I'll have to rewrite the scanning backend...
I have experienced the same behavior as denlee0710 mentioned below. There are two spots where the loading screen pauses. The first is at ~25% full where it pauses for ~3-5 minutes. It usually proceeds past this to 100% loaded, but then stays there indefinitely. What information do you need or steps you want me to perform to aid in troubleshooting this issue? I happen to really like this addon.
Having this addon enabled would cause wow to be stuck in loading screen for a very long time, eventually getting disconnected and kicked back to login screen. Any chance we'll get an fix/update for this?
How do I use this Addon? is there an icon thats hidden or just not in my view? do i need another addon to call this one? do you have a manual or something?
Thanks!
Rollback Post to RevisionRollBack
To post a comment, please login or register a new account.
-
View User Profile
-
Send Message
Posted Apr 17, 2015Hi, I hope you did not abandon the update you were working on previously. Still looking forward to that :) Thanks.
-
View User Profile
-
Send Message
Posted Jan 7, 2015Hi,
Firstly, thanks for this nice addon. As a completionist and "list" geek, I am sure I will be using this a lot. Also, I like its interface and sorting feature. Just one thing though, is there a possibility to reduce the time it takes to make searches? It takes more than 20s to do a search. I noticed it did an initial scan at the beginning which crippled the game which is fine for once but having to wait 20+ seconds for every search is kinda annoying. I tried another addon, Ludwig, much simpler but it generates the item lists immediately when you select the category. Must be related to how the data is handled but I hope that something can be done for ItemDB to increase its search speed.
Another suggestion: It would be nice to add "All" option in the Rarity Filter to select/deselect all rarity options.
Thanks!
-
View User Profile
-
Send Message
Posted Jan 10, 2015There is indeed a huge difference in how ItemDB and Ludwig handle the data.
Ludwig comes with a huge precomputed database which it seems to gather from wowhead.
ItemDB in contrast uses the data stored in the client.
Unfortunately, the way how that data is stored and accessed has changed over the years and calling the API for over 80k different items in a short time can mess up the client. :/
As a result, I throttled these API calls "a bit", which results in the slow initial scan that causes the client to load data from disk/server.
The search itself has also been throttled since 6.x to prevent the client from hanging. However, it seems like by now things are running a bit smoother again, so with the last commit, I undid some of that throttling and search should be 10-25 times faster ;) I'm now checking 2500 instead of 100 items per frame, but that could drop frame rate a bit. Also it could break in combat with "script run too long"...
And while I was at it, I also added your suggestion regarding the rarity filter.
-
View User Profile
-
Send Message
Posted Jan 12, 2015Hey,
Thanks for the detailed reply! My thinking was that the initial scan was getting the info from the client which generated a static database, then the searches were taking the info from that database. And since Ludwig also has a large number of items in its static database (without client interaction but a large database), I wondered if ItemDB's search could be made faster.
What I would ideally do is to give the user the option to do scans manually when the user wants, not every time the user logs in. And keep the scanned item database in a local static database as Ludwig does. Then do the searches normally which may speed up things a lot. I don't think items are changing very frequently, especially a while after expansions or major patches. So, I would prefer to be able to do the scan when I want, and then use the generated database (which is stored in the addons or WTF folder I don't know which way it should be done) and use that static data till I do another manual scan. In that case, for example, I would do the scan only after patches etc.
Anyway, thank you very much for your continuing effort on this and I will be looking forward to your next update.
-
View User Profile
-
Send Message
Posted Jan 21, 2015The main problem is that ItemDB uses a lot more data than Ludwig and thus doesn't maintain its own database but uses client API to access this data. Unfortunately Blizzard changed how this API behaves over the years and thus the initial scan is mainly used to force the client into slowly loading all item data (which would otherwise be done as required). Quickly loading all this data at once doesn't work as it causes the client to stop loading data at all at some point thus also breaking general functionality in other parts of the UI.
Furthermore, there are two kinds of items regarding this loading process. From the more than 80k items, a large amount is now stored in the client, only a few thousand have to be loaded from server. Unfortunately, the server will not always return data for those, only if there was previously some legitimate reason to load it, e.g. someone looted the item or has it in his bags.
It's a complicated situation and I've been struggling a lot to keep the addon working despite all those changes from Blizzard, while maintaining the initial design of not having a static database to work on. Keep in mind that the addon exists since vanilla, when Blizzard introduced the GetItemInfo API, so I don't intend to change that part.
In theory, it would be possible to cache the data for all client side items, sure, but even with a compression that would allow for individual data access, like Ludwig has it, the amount of data would be huge. Remember that Ludwig neither has item level nor vendor value or the data used for the advanced filters.
-
View User Profile
-
Send Message
Posted Aug 19, 2014Hi Elkano,
I hope this is not abandoned moving into Warlords of Draenor.
It works great, and is vastly superiour to AtlasLoot (Which locks up the client when searching!)
I've already recommended this to everyone I know, and they love the addon. They've said its just like using the Auction House, except we can see literally every item that is in the game.
The only issue that appears to come up is the lack of a Minimap button, i'm not sure if this is a bug in the current version? Leaving the only way to access this addon via manually typing in /itemdb which was not stated anywhere here.
-
View User Profile
-
Send Message
Posted Aug 20, 2014Since I just updated last month and don't intend to not get WoD, no plans to abandon, either ^^
ItemDB used to lock up the client until recently, too. There's lots of work to be done for a search and it seems like Blizzard intentionally slowed down some functions if you call them too often...
Regarding the missing minimap button, ItemDB comes with a data object for use with any display that supports LibDataBroker.
Since I have no options so far, I was just too lazy to add a minimap icon so far ^^'
Maybe if I find some time... ;)
-
View User Profile
-
Send Message
Posted Sep 5, 2012I just pushed a new version to beta status, see "Other Downloads" above or WoWAce.
This new version is working with 5.x but takes a few minutes to be ready (data object's tooltip will say "status: READY" and search/reset buttons will be enabled). This is due to the items having to be queried from server almost every session. But on the bright side, this now gives access to ALL items :)
-
View User Profile
-
Send Message
Posted Sep 29, 2012WOW!!!! I never expected you to get this working with 5x. TYVM Sir!
-
View User Profile
-
Send Message
Posted Sep 30, 2012Well, by now I know some more again about stuff is handled now then I did when I wrote the last post.
A large part of the items are stored client-side to begin with but will not be loaded unless queried (~61k). The rest of the items is requested from server and requires that those have been loaded by the server during its current session (eg due to dropping, a player owning them, ...). Unfortunately, it seems like scanning for those can put your client in a state where it wont receive any more item data from the server, at least temporary, which can cause unlootable quest items. Still looking into how to fix that (interface reload and looting then works though).
-
View User Profile
-
Send Message
Posted Oct 1, 2012Looks great! Only thing is it gums up the game while scanning. Any way to make the scanning a manual thing and maybe cache the data in a saved variable until the user says rescan?
-
View User Profile
-
Send Message
Posted Oct 1, 2012I'm still messing with the values in my local version to get things running smoothly. Like I said in the other comment, I'm still working out how exactly the client does some things, mostly due to hardly anyone outside Blizzard knowing :/
Also note that it's not really scanning. It's causing the client to load the data from disc/server into the cache so it can be used e.g. for tooltips, filtering, ...
-
View User Profile
-
Send Message
Posted Aug 17, 2011-
View User Profile
-
Send Message
Posted Jan 16, 2011-
View User Profile
-
Send Message
Posted Jan 13, 2011even though I was resulting in no bugs when it would load according to bugsack with !buggrabber...
it would sometimes result in a disconnect when loading the character or after it loads you wait 2-3 seconds and bam you get disconnected from the game... over and over and over.
I checked my errors folder. nothing
I checked the framesXML to see if it was spitting any errors... nope wasn't.
on a 5.9 rating scale if you want a hard number for windows rating system...
I'm not saying its a bad addon, I would of loved it if I was able to use it but at its current version state.. it simply doesn't support 4.0.3a patch.
-
View User Profile
-
Send Message
Posted Oct 18, 2010So I'll have to rewrite the scanning backend...
-
View User Profile
-
Send Message
Posted Oct 17, 2010-
View User Profile
-
Send Message
Posted Oct 13, 2010-
View User Profile
-
Send Message
Posted Apr 9, 2010-
View User Profile
-
Send Message
Posted Oct 12, 2009Thanks!