Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can we help? #14

Open
ZoeESummers opened this issue Oct 29, 2024 · 25 comments
Open

How can we help? #14

ZoeESummers opened this issue Oct 29, 2024 · 25 comments
Assignees

Comments

@ZoeESummers
Copy link

Now that's the official forums are (temporarily?) back up, how can we help you ensure that all that knowledge isn't lost?

@Lisias Lisias self-assigned this Oct 29, 2024
@Lisias
Copy link
Contributor

Lisias commented Oct 29, 2024

Hi!

Right now, these are my concerns in order of priority:

  1. Pro bono legal advising indications. I'm being dox'ed.
  2. A secure host for hosting a working mirror, so people would have where to go if Forum goes down.
    • You can bet your mouse this host will be harassed night and day.
  3. Some kind of URL trickery service that would monitor Forum and automatically redirect the request to Forum if they are alive, or the latest version of a page in the mirror if not.
    • So everybody would use this service on their documentation links.
  4. Tips for reliable Internet Archive alternatives
    • The buzzheavier service I'm using is working but...
  5. A small appliance to use for scraping other than my Steam Deck, so I can use it for playing again... :D
    • Not really a problem, to tell you the truth - as soon as I manage to finish archiving the Profiles, the workload will drop to a minimum and I will probably be able to use one of my raspberry pi 3 to do the job.
  6. Some funding to help me pay for the costs of some services I'm using. They are cheap, but the current exchange in my country is starting to be inconvenient.
    • As a matter of fact, the fees I would pay to get the funding and then use it could render the transaction unviable - sometimes is cheaper to just take the heat, so I'm not really concerned about this.

Thanks!

@ZoeESummers
Copy link
Author

I can't believe you're being doxxed. WTF is wrong with people?!

@towermom9
Copy link

  1. A secure host for hosting a working mirror, so people would have where to go if Forum goes down.

    • You can bet your mouse this host will be harassed night and day.
  2. Some kind of URL trickery service that would monitor Forum and automatically redirect the request to Forum if they are alive, or the latest version of a page in the mirror if not.

    • So everybody would use this service on their documentation links.

maybe cloudflare DDOS protection and cloudflare hosting?

@ZoeESummers
Copy link
Author

Wouldn't it be easier, cheaper to just build a raid storage NAS and host it locally via a public IP rather than paying someone to host it? That way you could clone/scrap the existing site directly onto the network storage, strip out external links etc and then it only costs the electricity to run it?

@towermom9
Copy link

Wouldn't it be easier, cheaper to just build a raid storage NAS and host it locally via a public IP rather than paying someone to host it? That way you could clone/scrap the existing site directly onto the network storage, strip out external links etc and then it only costs the electricity to run it?

personality, I think it needs to be community run and funded, cuz this couldn't be a one man job. unless you are Elon musk or some other rich gazillionares

@Lisias
Copy link
Contributor

Lisias commented Nov 5, 2024

I can't believe you're being doxxed. WTF is wrong with people?!

To tell you the truth, it's anyone guess. But:

  1. KSP is the only remotely possible controversy I'm involved at this point. For years
  2. The KSP ecosystem is being taken by people willing to explore it commercially. That's a fact.
  3. I had stepped against such exploitation, as well advocated strict adherence to the EULA (and some pertinent legislation). Also for years at this point.
  4. And I... well... had clashed with some famous modders in the past due technical issues...
  5. "DOXXing" is explicitly forbidden by Forum rules - so, yeah, this had happened before.

I'm being doxxed by doing this Archive? Or it was only the straw that broke some camel' back? Who knows.

But it was remarkable that a private monitoring service I pay alerted me about the leakage essentially at the same time Forum came back online.

Screenshot_20241105_005914_Messages

Are these events related? Who knows. But it's an interesting coincidence.

--- POST EDIT ---

Sometimes my dislexia really bites my arse. Where I had wrote "as well to strict", I should had write "as well to advocated". Crap, I passed the wrong message on that phrase. Fixed.

@Lisias
Copy link
Contributor

Lisias commented Nov 5, 2024

  1. A secure host for hosting a working mirror, so people would have where to go if Forum goes down.
    • You can bet your mouse this host will be harassed night and day.

maybe cloudflare DDOS protection and cloudflare hosting?

Probably, but how much this will cost? Money will always be an issue.

@Lisias
Copy link
Contributor

Lisias commented Nov 5, 2024

Wouldn't it be easier, cheaper to just build a raid storage NAS and host it locally via a public IP rather than paying someone to host it? That way you could clone/scrap the existing site directly onto the network storage, strip out external links etc and then it only costs the electricity to run it?

It's being considered. But, still, even if I manage personal funding for the service, I don't want to be the only one serving such data. Redundancy is the name of the game - and, assuming I'm being targeted (also, because it's a fact I'm being for other reasons) for this, I don't intent to concentrate all the fire on myself...

@Lisias
Copy link
Contributor

Lisias commented Nov 5, 2024

personality, I think it needs to be community run and funded, cuz this couldn't be a one man job. unless you are Elon musk or some other rich gazillionares

Agreed. But this doesn't meant that we could not built a kinda of "Federated" pool of data providers, sharing the load between us. This also would help with the redundancy.

@Lisias
Copy link
Contributor

Lisias commented Nov 17, 2024

I'm resurrecting another side project that demanded a Raspberry PI update (that Pi3 of mine is not cutting it anymore) that was already funded, and it ended up being cheap enough to buy the 8GB RAM one instead of the sufficient 2GB I originally intended to buy, and now I will have enough juice to shove this project on it too. No need for a new local dedicated appliance anymore.

I'm going to get my Steam Deck back! :)

@Lisias
Copy link
Contributor

Lisias commented Dec 1, 2024

Buzzheavier is definitively out - the new policies doesn't fit the project anymore.

I need suggestions for alternatives to distribute the content besides Archive.

@Lisias
Copy link
Contributor

Lisias commented Jan 22, 2025

Well... Apparently, this is it. As from 2025-01-22 01:37:41, forum.kerbalspaceprogram.com and wiki.kerbalspaceprogram.com are no more.

The Cloudflare DNS entries were deleted, so they are probably gone - unless it was a mishap, but... What would be the chances?

2025-01-22 01:37:34 <200 https://forum.kerbalspaceprogram.com/contact/>
2025-01-22 01:37:41 (530) <GET https://forum.kerbalspaceprogram.com/contact/> (referer: https://forum.kerbalspaceprogram.com/forum/130-ksp2-mod-releases/)

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

Forum is back! :)

2025-01-22 05:26:41,337: [WARNING]: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7fff89c8b910>: Failed to establish a new connection: 0x04: Host unreachable')'
127.0.0.1 - - [2025-01-22 05:27:00] "CONNECT forum.kerbalspaceprogram.com:443 HTTP/1.1" 200 75 26.523026

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

And it's gone again...

> dig forum.kerbalspaceprogram.com.cdn.cloudflare.com

; <<>> DiG 9.20.3 <<>> forum.kerbalspaceprogram.com.cdn.cloudflare.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35094
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: c6af3222aa2b26b665b8c3c26791877db236d633270e6392 (good)
;; QUESTION SECTION:
;forum.kerbalspaceprogram.com.cdn.cloudflare.com. IN A

;; AUTHORITY SECTION:
cloudflare.com.		300	IN	SOA	ns3.cloudflare.com. dns.cloudflare.com. 2363033056 10000 2400 604800 300

;; Query time: 25 msec
;; SERVER: 192.168.200.1#53(192.168.200.1) (UDP)
;; WHEN: Wed Jan 22 21:04:16 -03 2025
;; MSG SIZE  rcvd: 148

Someone is screwing with the DNS entries on CloudFlare.

I had an idea...

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

Yeah, I was right. Forum IS online, but they are changing DNS again.

Mangling with my /etc/hosts I was able to post on Forum.

Image

Image

Image

https://forum.kerbalspaceprogram.com/topic/226141-so-we-had-some-kind-of-technical-problem/?do=findComment&comment=4438588

---- EDIT

I had made a monstrous typo on a dig on Forum, just realized it today and fixed it. Pretty embarrassing, but by plain luck it didn't deviated me (too much) from the diagnosing. Anyway, that mistake were hurting a bit, so I fixed it.

@ColdJ-KSP
Copy link

Crossing everything.

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

More news:

Now forum.kerbalspaceprogram.com is being resolved on my cable-tv network too, But sp-forum-elb-2033387385.us-west-2.elb.amazonaws.com still don't - not on my home network, neither on my remote one, so this is the reason Forum is out of reach to the World.

But Forum itself is still alive and well - besides a bit alone...

Image

Image

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

While checking Forum to confirm it's still alive (for a redditor), I noticed this:

Image

Man... This is weird. End of the World kind of weird.

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

I built a socks proxy on an appliance of mine i AWS, and started to scrap forum again after adjusting the remote /etc/hosts.

Forum is essentially idle, there's no reason not to do it.

However, I'm lowering my footprint - I'm going to bed and leave the thing unattended, I don't have a clue about what's going to happen to Forum in the mean time. But since I locked the DNS entry for forum on that appliance, there's no risk of the server being moved to a smaller box somewhere where my current load would cause problems. Worst case scenario, they pull the plug and my spider just halts.

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

Forum still alive.

Image

But DNS still screwed.

The good part

> dig forum.kerbalspaceprogram.com

; <<>> DiG 9.20.3 <<>> forum.kerbalspaceprogram.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 50028
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;forum.kerbalspaceprogram.com.	IN	A

;; ANSWER SECTION:
forum.kerbalspaceprogram.com. 3600 IN	CNAME	sp-forum-elb-2033387385.us-west-2.elb.amazonaws.com.

;; AUTHORITY SECTION:
us-west-2.elb.amazonaws.com. 60	IN	SOA	ns-332.awsdns-41.com. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 60

;; Query time: 118 msec
;; SERVER: 192.168.200.1#53(192.168.200.1) (UDP)
;; WHEN: Thu Jan 23 11:39:33 -03 2025
;; MSG SIZE  rcvd: 197

The screwed part

> dig sp-forum-elb-2033387385.us-west-2.elb.amazonaws.com

; <<>> DiG 9.20.3 <<>> sp-forum-elb-2033387385.us-west-2.elb.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 27398
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1280
;; QUESTION SECTION:
;sp-forum-elb-2033387385.us-west-2.elb.amazonaws.com. IN	A

;; Query time: 0 msec
;; SERVER: 192.168.200.1#53(192.168.200.1) (UDP)
;; WHEN: Thu Jan 23 11:40:11 -03 2025
;; MSG SIZE  rcvd: 80

(there's no ANSWER SECTION)

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

NOPE! There was a typo! I typed kerbak and not kerbal!!!! Oh, crap. :D


News from the front

The DNS entry for Forum was deleted again:

> dig forum.kerbakspaceprogram.com

; <<>> DiG 9.20.3 <<>> forum.kerbakspaceprogram.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 53811
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;forum.kerbakspaceprogram.com.	IN	A

;; AUTHORITY SECTION:
com.			900	IN	SOA	a.gtld-servers.net. nstld.verisign-grs.com. 1737663218 1800 900 604800 900

;; Query time: 15 msec
;; SERVER: 192.168.200.1#53(192.168.200.1) (UDP)
;; WHEN: Thu Jan 23 17:14:06 -03 2025
;; MSG SIZE  rcvd: 130

(no ANSWER SECTION)

What means that they are working on the problem.

@Lisias
Copy link
Contributor

Lisias commented Jan 23, 2025

Note to my future self: at least on this moment, my spider does not reflects on the Guest List.

I doubled the concurrent connections (from my side, what's different from hitting Forum with all of them at the same time), then I halved them, then I just stopped it.

Absolutely no impact on the Guest List. And none of the URLs the spider were hitting were listed on that list neither.

So, at least for this spider I'm using, the hits on Forum DOES NOT reflects on the Guest List.

@Lisias
Copy link
Contributor

Lisias commented Jan 24, 2025

Right now, digging forum.kerbalspaceprogram.com and sp-forum-elb-2033387385.us-west-2.elb.amazonaws.com gave me exactly the same results as #14 (comment)

Note to my future self: avoid task switching on business hours. Something will be left behind.

@Lisias
Copy link
Contributor

Lisias commented Jan 24, 2025

Nothing changed for forum.kerbalspaceprogram.com , but privatedivision.com behaviour changed:

> curl -v https://www.privatedivision.com/
* Host www.privatedivision.com:443 was resolved.
* IPv6: (none)
* IPv4: 34.216.9.212, 44.226.249.193, 44.241.162.245
*   Trying 34.216.9.212:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /opt/local/share/curl/curl-ca-bundle.crt
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 / secp256r1 / rsaEncryption
* ALPN: server accepted h2
* Server certificate:
*  subject: C=US; ST=New York; L=New York; O=Take-Two Interactive Software, Inc.; CN=*.privatedivision.com
*  start date: Oct 12 00:00:00 2024 GMT
*  expire date: Oct 14 23:59:59 2025 GMT
*  subjectAltName: host "www.privatedivision.com" matched cert's "*.privatedivision.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert Global G2 TLS RSA SHA256 2020 CA1
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* Connected to www.privatedivision.com (34.216.9.212) port 443
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://www.privatedivision.com/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: www.privatedivision.com]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.10.1]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: www.privatedivision.com
> User-Agent: curl/8.10.1
> Accept: */*
>
* Request completely sent off
< HTTP/2 504
< server: awselb/2.0
< date: Fri, 24 Jan 2025 15:22:39 GMT
< content-type: text/html
< content-length: 132
< set-cookie: AWSALB=4qgE8akHOm+u8RSMsTpwmHQ17fOwJ26SNJbpjOKmMhHdr8GwctvsteGTXqii6v/m5fN8gycBMSI0uqbyf4uOzbVRh+Rjs+jHLcYnvekrDzdPkcxgpGh5z2+rPCiz; Expires=Fri, 31 Jan 2025 15:21:39 GMT; Path=/
< set-cookie: AWSALBCORS=4qgE8akHOm+u8RSMsTpwmHQ17fOwJ26SNJbpjOKmMhHdr8GwctvsteGTXqii6v/m5fN8gycBMSI0uqbyf4uOzbVRh+Rjs+jHLcYnvekrDzdPkcxgpGh5z2+rPCiz; Expires=Fri, 31 Jan 2025 15:21:39 GMT; Path=/; SameSite=None; Secure
<
<html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
</body>
</html>
* Connection #0 to host www.privatedivision.com left intact

and

> dig www.privatedivision.com

; <<>> DiG 9.20.3 <<>> www.privatedivision.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60602
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 7df3cdc671bd6049abd6bc286793b10c1d9e89719e0ec476 (good)
;; QUESTION SECTION:
;www.privatedivision.com.	IN	A

;; ANSWER SECTION:
www.privatedivision.com. 3600	IN	CNAME	privatedivision-wp-develop-ecs-1364482112.us-west-2.elb.amazonaws.com.
privatedivision-wp-develop-ecs-1364482112.us-west-2.elb.amazonaws.com. 60 IN A 44.241.162.245
privatedivision-wp-develop-ecs-1364482112.us-west-2.elb.amazonaws.com. 60 IN A 34.216.9.212
privatedivision-wp-develop-ecs-1364482112.us-west-2.elb.amazonaws.com. 60 IN A 44.226.249.193

;; Query time: 143 msec
;; SERVER: 192.168.200.1#53(192.168.200.1) (UDP)
;; WHEN: Fri Jan 24 12:26:07 -03 2025
;; MSG SIZE  rcvd: 208

@Lisias
Copy link
Contributor

Lisias commented Jan 24, 2025

Forum is back online.

> dig forum.kerbalspaceprogram.com

; <<>> DiG 9.20.3 <<>> forum.kerbalspaceprogram.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49764
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;forum.kerbalspaceprogram.com.	IN	A

;; ANSWER SECTION:
forum.kerbalspaceprogram.com. 3600 IN	CNAME	ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com.
ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com. 60 IN A 44.240.13.95
ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com. 60 IN A 35.155.205.25

;; Query time: 154 msec
;; SERVER: 192.168.200.1#53(192.168.200.1) (UDP)
;; WHEN: Fri Jan 24 13:06:41 -03 2025
;; MSG SIZE  rcvd: 152

and

> dig ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com

; <<>> DiG 9.20.3 <<>> ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 483ba604585cf5a8d71619f06793bae8b5f73d99084e4d41 (good)
;; QUESTION SECTION:
;ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com. IN A

;; ANSWER SECTION:
ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com. 60 IN A 44.240.13.95
ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com. 60 IN A 35.155.205.25

;; Query time: 14 msec
;; SERVER: 192.168.200.1#53(192.168.200.1) (UDP)
;; WHEN: Fri Jan 24 13:08:11 -03 2025
;; MSG SIZE  rcvd: 141

And just because:

> ping forum.kerbalspaceprogram.com
PING ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com (44.240.13.95): 56 data bytes
--- ksp-forum-elb-2033387385.us-west-2.elb.amazonaws.com ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

In time, I'm scaling down the spider again. Forum is back to business, it's not wise to hammer the guys the same as when they would be near empty.


And just for the sake of curiosity:

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants