cache geoip lookups #1071
cache geoip lookups #1071
Conversation
Seems like this is a bug -- caught by the dev server compiler/linter
geoip lookups for the same ip4 addr can be cached. this uses a similar cache to the peerid location cache (arguably peerid loc shouldnt be cached-- ip addr only?) the code is a bit messy, and probably should be refactored This reduced my load time significantly. lots of nodes have the same ip4 addr: - nodes under same physical machine - nodes under same VPN - nodes under same carrier grade nats - nodes behind city-wide firewalls eg lots of nodes in china show the same ip addr.
cleaned up the geoip cached resolver code some. put things into a class.
|
Note, i couldn't get the map to render properly (osx). it gives this error in the console, and stops rendering dots. Looks like a d3 error w/ the paths:
I dont think this is due to this changeset -- the same error exists for me in master. Note: looks like d3 currently renders one dot per peer. it should probably render one dot per city (massively deduped).
|
|
I also get these logs in the console -- ~6msgs every second
|
|
BTW, this is critical when peer count is large :) -- though it starts helping at ~100 peers
|
jbenet
force-pushed
the
cache-geoip-lookups
branch
from
d361921 to
70e843f
Compare
|
Wowsez! I need to create some tests with large numbers of peers. I've never seen a 26k before. Thanks for the heads up, will dig into it. |
|
Here is a first draft of all things we need to fix in the peers page: #1072 |
|
Am digging in to why the e2e test fails on CI. @jbenet I'm gonna adopt this PR. Let me know if you are still working on it. |
#1039 fixed a bunch of language issues, including upgrading a hyphen to and endash in the page title of the ipld explore section. The page title is used to by the e2e test to check that navigation has worked. This tweak should fix e2e tests for this PR. We have a more general issue in that we don't get CI tests for PRs from forks, as circle-ci (and all the CI systems) have no good way to protect users from exfiltration of secrets if you let anyone trigger a CI build, and the config for CI is stored in the repo...
|
thanks for adopting! go for it! 👍 |
License: MIT Signed-off-by: Henrique Dias <[email protected]>
|
Merged with master, looks good and seems to work well. |
geoip lookups for the same ip4 addr can be cached. this uses a similar
cache to the peerid location cache (arguably peerid loc shouldnt be
cached-- ip addr only?).
Requesting a pull to ipfs-shipyard:master from ipfs-shipyard:cache-geoip-lookups
Changes:
a1290cc (jbenet, 20 seconds ago)
bumped up concurrency for geoip lookups
28af4b6 (jbenet, 6 minutes ago)
cleaned up geoip cached resolver code
cleaned up the geoip cached resolver code some. put things into a class.
4f33563 (jbenet, 32 minutes ago)
cache geoip lookups
geoip lookups for the same ip4 addr can be cached. this uses a similar
cache to the peerid location cache
(arguably peerid loc shouldnt be cached-- ip addr only?) the code is a bit
messy, and probably should be refactored
This reduced my load time significantly.
lots of nodes have the same ip4 addr:
eg lots of nodes in china show the same ip addr.
e9ece8e (jbenet, 2 hours ago)
class should be className
Seems like this is a bug -- caught by the dev server compiler/linter