You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi guys, once again sorry for my absence. At work for this time of year there are always lots of pressing deadlines for the end of the year, and that combined with christmas-y stuff usually sucks up all my time.
Not forgotten about all this, I also have some issues with the integration as it stands on my own rig. I have been thinking about how best to tackle all your various issues in one sweeping change, and I have decided the best approach is to turn all the scraping code into a series of modules which can be reconfigured.
So basically, there would be a json file for say the Italian version or the US version. And this file will just say in what order to run each of the modules, and with what arguments. Such as checking things like the remember me box.
Taking this approach will mean separate documentation can be written for how to write your own configs, as no code would have to be touched. This documentation could also form the basis of a sort of setup wizard in the client. Whereby if no config is working for you, you could be guided through creating one with a series of questions.
This would also make it much easier for all of you too be able to merge in your configs for all of your various regions, and make them available to everyone else.
Which config to use would be automatically picked based on the domain of the regional instance the user is trying to access. So accessing amazon.it would use the italian one, etc. If no specific config exists, then it would fall back.
I will work on adding this over the weekend. It should be able to get the functional changes done and the documentation written, but no config wizard thing. But this should be enough to resolve most of the open issues at this point. I will add some configs to cover all the open issues.
Currently not sure if this is necessary. I did a bunch of checks on a few different versions of amazon around the world and only found one inconsistency so far.
I will keep this issue open in case anything new appears, but if not this will be closed off as redundant in time
Hi guys, once again sorry for my absence. At work for this time of year there are always lots of pressing deadlines for the end of the year, and that combined with christmas-y stuff usually sucks up all my time.
Not forgotten about all this, I also have some issues with the integration as it stands on my own rig. I have been thinking about how best to tackle all your various issues in one sweeping change, and I have decided the best approach is to turn all the scraping code into a series of modules which can be reconfigured.
So basically, there would be a json file for say the Italian version or the US version. And this file will just say in what order to run each of the modules, and with what arguments. Such as checking things like the remember me box.
Taking this approach will mean separate documentation can be written for how to write your own configs, as no code would have to be touched. This documentation could also form the basis of a sort of setup wizard in the client. Whereby if no config is working for you, you could be guided through creating one with a series of questions.
This would also make it much easier for all of you too be able to merge in your configs for all of your various regions, and make them available to everyone else.
Which config to use would be automatically picked based on the domain of the regional instance the user is trying to access. So accessing amazon.it would use the italian one, etc. If no specific config exists, then it would fall back.
I will work on adding this over the weekend. It should be able to get the functional changes done and the documentation written, but no config wizard thing. But this should be enough to resolve most of the open issues at this point. I will add some configs to cover all the open issues.
The text was updated successfully, but these errors were encountered: