[Discussion] Decentralisation

[ShalokShalom]

In order to succeed in the evaluation of possible decentralisation solutions:

https://blog.bluzelle.com/how-is-bluzelle-different-from-ipfs-filecoin-storj-sia-and-ethereums-swarm-1d1d792658f8

Here is the original article:
Human-Connection/lagacy-documentation#2

[HC-Team]

Thats an important topic, I was looking into Gluster FS for that reason. One thing with the Blockchain implementation of File Systems is always, how long will the data remain secure?

When the encrypted data (which is distrebuted on untrusted nodes) will be decoded in the future (as compution power for decryption increases)? All personal data would be in the wild! Do you have any knowledge on that regard?

[ShalokShalom]

I have zero knowledge on that and will ask someone who has.

You can also consider contacting the developers itself, very often is this the most reliable source.

[ogerly]

also in sachen ipfs ist es so das wir jetz drei testseiten mit verschiedenen datein laufen haben ..
das peertopeer benötigt anfangs einen node sobald eine relevanz entsteht wird es automatisch verteilt und steigt in der dezentralisierung.
daten die nicht zuordbar sind ... sind nicht gefährlich.
die kunst ist einen service anzubieten der dezentral liegt und dezentral arbeitet.
die schwachstelle wäre dann nur noch der der den service rausgibt.
wenn es vollkommen autonom läuft auf grund verschiedener smart contracts .. + denzentrale daten und DB ... dann ist es nur noch das verteilen und zusammenführen.
über den Key des users wäre er alleine verantwortlich und beitzer.
faire smart contracts wären unabdingbar..

[ghost]

decentral data storage like ipfs should be an option for a later point when the feature set is completed. For now it's not viable to integrate.

@HC-Team your concerns are good to think about, however if blockchain is easy to decrypt we will have a ton of other issues as it's one of the strongest encryption algorythms today and will likely require quantum computing power to crack.

[HC-Team]

@sebako86 sure I don’t think of today but it could be sooner then you think, I mean you can rent quantum computing time today at IBM and maybe others. So if data is public, It might become an issue.

But beside that, we can take a look at a later point in the project to that matter.

[ShalokShalom]

Quantum computing at IBM is even free (you might be able to rent it too) while I am quite sure they check for such things.

And yes: Care is of course appreciated ^-^

[frankgerhardt]

You all obviously know Human-Connection/lagacy-documentation#2

I haven't seen that repo earlier although I looked (hard) for the HC sources. Was this private? Then...what, why!?

[ShalokShalom]

It is public since the early beginning. Why wasnt the whole source code open?
Since Dennis Hack is a bit paranoid, to say the least.
Its also one reason, why I will never use HC on my own.
You can also see others worried all over the place, like in that exactly repo under the closed ones.

Legacy is spelled with an e, by the way.

To your issue: This repo was previously called documentation, which is probably why you havnt found it. Why start a thread like this in the documentation channel?
Well, since they had no other infrastructure for public communication.

[ionphractal]

@ShalokShalom
I'm feeling pretty confused about how bluzelle is advertised. In the blog post you linked they say it's a marriage of "best aspects of decentralization and the blockchain [...] together with decades-mature database science".
What I ask myself is how they tackle the unsolved problems[1] in "3rd gen distributed ledger ("DLT") systems"[2] aka current blockchain technology:

• proof-of-work system: Leads to transaction costs and latencies unacceptable for a database service. Energy consumption and required hardware aka "the hidden costs" (same with nuclear power plants) is also unacceptable.
• leader based consensus: Taking down the leader again and again and again also ends up in denial of service.
• immutability: You need to know the whole transaction history in order to verify the integrity. Correct me if I'm wrong but I haven't seen a DLT that is capable of truncating parts of its data. It is actually what they remark as bad with filesystems, but I also didn't see an explanation how they do it differently except that they're limiting the data by the factor of a swarm size (whitepaper [4] section 3.10).

Other thoughts:

• How do you share data between users? Fully decentralized should mean that every user owns its data. Otherwise it wouldn't make much a difference from traditional networks except that you have an awkwardly complicated backbone. A glimpse into the whitepaper[4] under section 3.5 it seems like every user has its own data encrypted with a single password-seeded key, that means sharing inbetween users is impossible without giving full write permissions.
• How does one secure data so that only eligible parties are able to view it? What happens when one of the parties looses their keys (bad luck I guess?) or even has it stolen from them (-> the attacker may see all their data)?
• Data retrieval latency: If you query a swarm, it is highly dependant on the participants / nodes which makes it not so suitable for high performance applications. The KV[3] part of bluzelle also adds to this as you probably have to query different nodes multiple times to get all your data (depending how complex your data is) which adds (not only) network round trip times. Though it seems this is somewhat mitigated by the partial replication feature although I don't know to which extend.

I must say that I have not studied the whitepaper extensively, so please forgive me if some information may be not 100% accurate. And I don't want to diregard this topic in any way as I find this vital in the near future. The "hashgraph" described in the videos sounds also promising for solutions to emerge based on it. Though, an aspect in me is not happy with the proprietary aspect of it. But because of the complexity of distibuted storage and databases stated above it has to be thought out properly and I would see it as a "next generation" feature.

@ALL
Regarding the open source topic there are a lot of reasons why to have the code private for a while. And it's been open sourced now, so I don't understand why this should be a discussion topic, especially since there were a lot of gatherings and (i.e. gitter and skype) chats in the past where anyone could get in contact and participate. I'm not affiliated with the project and I haven't attended one personally but I subscribed to the publicly announced feeds and got this information. Just try to be nice and forgiving to eachother, ok? ;)

[1] https://youtu.be/pOc23lJw7ls?t=1021
[2] https://www.youtube.com/embed/ZrFrXFdRW4k
[3] https://blog.bluzelle.com/technical-concepts-behind-bluzelle-ed53709ed6cf
[4] https://bluzelle.com/wp-content/uploads/2017/10/Bluzelle-Technical-Paper.pdf

[ShalokShalom]

You may be right with your thought about Bluezelle, you have obviously a lot of understand of these concepts and its surely possible that this is simply yet another wanna be project. The intention behind the initial post which I linked as well was more like something as a collection of available solutions and a place to discuss them. Exactly as you did. :)

[ionphractal]

@HC-Team I personally don't recommend GlusterFS because of I know of past bad experiences with it in production use cases. I'd also consider IPFS is still experimental and it doesn't solve some things in the equation, e.g. there is afaik no authorization mechanism for neither the connecting peers (needs more research, can filters be used? https://ipfs.io/docs/commands/#ipfs-swarm-filters) nor the http gateway for accessing content and its administration looks rather complex compared to others. Not sure what kind of content/files you want to replicate but for static user content (e.g. photos, PDFs, etc.) I would say that an s3 or swift compatible object store is (still) best practice. E.g. Ceph or Minio for self-hosting.

[ShalokShalom]

The authorization issue in ipfs is heavily tackled and there are possible solutions, see the mentioned link in the initial post here.

Thanks a lot for that links, it makes pretty obvious about what this stuff is around.
I get more and more confirmed, by how dangerous this can be, since it shows how much people will succeed to use the money system which is broken by design.

About which sane reasons you speak about, to keeping the source code unpublished?

[ShalokShalom]

I just saw this thread is mostly in German:
https://github.com/solid/solid-signup
Then there is uPort, while I have no idea if ipfs supports that.

Anyway how it is: Akasha is a social media platform which uses ipfs, so you can see how they solve it.

[ShalokShalom]

Here are a couple of "awesome" lists, you might simply open all the links in the initial post as well, we collected a couple of interesting projects there.
Hnn, we could simply care them in this lists (and them into one)

https://github.com/retrohacker/awesome-p2p
https://github.com/dennismartensson/awesome-peer-to-peer
https://github.com/hackerkid/Awesome-P2P
https://github.com/steve-vincent/awesome-decentralized

[ShalokShalom]

If you think about Kubernetes, I recommend Kelsey Hightower as contact.

[ionphractal]

[bluzelle is...] possible that this is simply yet another wanna be project.

I don't know, just stated my concerns.

About which sane reasons you speak about, to keeping the source code unpublished?

Ask the devs, I'm not one of them ;) Though I can understand your frustration about how some things progress in this world (seems to me like it's not going fast enough for you Human-Connection/lagacy-documentation#2 (comment)). But we're all in some way or the other affected by a highly dynamical ecosystem that has some broken elements (depending on the information at hand). Whatever incentive may have led to a decision, does it help blaming anyone within the field of influence of such a system for having made decisions like that? Therefore I don't see a point in arguing about that.

money system which is broken by design

+1 But that's a topic with no solution at hand. I wonder if any other money(-based solution) at all would do the trick. If the incentive is money, can there be a solution that doesn't perpetuate money as an incentive?

Anyway, thanks for sharing the links!

@sebako86 Could you please explain what problem you're trying to solve exactly? K8s is a workload scheduler framework, not a file storage. Persistant volumes do not neccessarily solve high availability requirements (they are dependent on the IaaS/storage layer they reside on and binding may be limited).

[frankgerhardt]

And as awesome info for the others and because it hasn't been mentioned yet, there is the Facebook group https://www.facebook.com/groups/distributedinternet/ that @ShalokShalom and @ogerly manage. Sorry about mentioning FB here 🤦‍

[frankgerhardt]

http://matrix.org is an encrypted p2p messaging protocol with decentralized servers, think email. It is mostly used for chat but targets other domain like IoT and M2M as well. I can imagine a data model being defined for the HC use cases that sits on top of matrix and syncs "topics" or "rooms" between peers as needed. Not the whole dataset but only what you are interested in.

[ShalokShalom]

Whatever incentive may have led to a decision

While my comment above might look like a pure rant, is it based on the statement of one of the potential developers.

+1 But that's a topic with no solution at hand.

Surely there is.

You recall the meaning of fighting fire with fire?

This is due the technique to burn a small ring around your farm/house/whatsoever, in order to gain protection from huge firestorms, as they happen in the dry areas on this planet.

So, its the same with money. The most logical investment into something is imho obviously into infrastructure which allows us to completely remove the need for money?

I am doing this with a project which offers a life as everyone wants, while the most of them are simply grown up in a culture which does avoid to show them how.

We build a bridge between the paradies and capitalism. ^-^

http://matrix.org is an encrypted p2p messaging protocol

When it comes to adoption, is Matrix the most widespread p2p technique asides from BitTorrent, which I am currently aware.

I love F# with Apache Strom plus Elchemy:
https://github.com/Prolucid/FsShelter
https://github.com/wende/elchemy

While those are simply my personal favorites.

I think self-healing is an obvious bonus in so many ways.
Concise syntax and functional programming as well.

My 2 cents

[ionphractal]

@ShalokShalom

You recall the meaning of fighting fire with fire?

Of course you have a point there. Though it may've sounded like it, I didn't say 'never use money'. Using what 'is' is the most efficient way. However, if you don't play your cards well, the fire you lay in well intent may turn against you. Being cautious and double checking if one is not just swallowed by one's emotional resistance eventually leading strengthen what you fight sounds right to me. For example Bitcoin, as well intended it may be (or have been) and as some beneficial points it may have, is pretty much useless, imho worsened the situation by far. I do not wish to drag this topic to the length, and it is just my opinion that is based on my thoughts and informations which may be flawed as well, but just let me say that it did not reach down to the fundamental problem that brought forward the 'broken money system'. And so it became what it tried to fight: a tool for speculation and burning money (and on top even centralizing it, see the whales). The money that has been and is being poored into it would be better invested in projects like you do (or HC or the others).
But the point I'm trying to make is that I think we as a species should calm down first to regain our composure (aka not jumping to early conclusions just because of the emotional state we're in, rushing against our 'enemies' when we are blind to see that we're our own devils). That said...

While my comment above might look like a pure rant, is it based on the statement of one of the potential developers.

Don't get me wrong, I don't blame you nor anyone else for being impatient. I'm neither emotionless nor in a sense perfect. But as stated above I don't see how impatience and straining human relationship, which goes hand in hand, does any good let alone lead to a healthy human society. How could we possibly apply it at scale when we can't even maintain peace inside a micro society like a team?

The most logical investment into something is imho obviously into infrastructure which allows us to completely remove the need for money?

While it can ease tensions, infrastructure does not necessarily remove the need for money, for it is trust that does. Of course one may argue whether one has to remove trust completely (like Bitcoin or the likes?... means also giving up your own power to decide for yourself, behind it is coersion by an artificial 'law'? Who makes the laws? Can it adopt to the environment fast enough?... a deep and complex topic) or maybe one wants to control (centralization of power that we already have...). But neither of them is an organic approach if I look at how my own body works...

I am doing this with a project which offers a life as everyone wants, while the most of them are simply grown up in a culture which does avoid to show them how.

We build a bridge between the paradies and capitalism. ^-^

❤️

[ionphractal]

@sebako86 The bad experiences include bad performance with large file sets, lost synchronicity (meaning some data is on one but missing on other nodes), problems with self-healing functionality and in rare cases data losses occured. The cluster was a bare-metal setup with Ubuntu 14.04 (Trusty) and a pretty old 3.4 glusterfs, so I can't speak for present versions.

But from my past experiences and also the cloud background of the company I work for, directly mounted file systems like glusterfs or nfs are a bad idea in general. The benefits (i.e. performance) usually do not outweigh the (debugging/maintenance) complexity and efforts. Regardless of the scheduler you use, my recommendation would be to adhere to microservice and cloud-native best practices (i.e. https://12factor.net/). For this case it means the web app (/container) should be totally stateless consuming dedicated, fault-tolerant backing services (i.e. s3-ish services for files) via standard protocols like http. For files, open source projects with high reputations are Ceph and Openstack Swift (or recently Minio) but I personally do have not helpful experience with them.

[ShalokShalom]

As hinted above, do I share your thought about Bitcoin and so on.
Greed is its driving force. And it can appear quite frightening that this is so less obvious to the people. As said, I see direct investments into grounds the way to go.

In general, there is also the promising IOTA and Faircoin.
Probably some more as well. :)

[ShalokShalom]

Here is a solution to implement peer to peer authentication:
https://hub.libranet.de/help/en/developer/zot_protocol

Human-Connection/Human-Connection#2

[ShalokShalom]

Here is another solution: https://blockstack.org/blog/serverless-sign-in-with-blockstack-auth

[appinteractive]

we will add more auth providers in the future

[ShalokShalom]

👍

[ionphractal]

When I worked on getting Firefox Sync to run on my server, I found something that might be interesting for this decentralization topic later on: https://www.kinto-storage.org/
It's a pluggable JSON storage / cache / API put forward by Mozilla, I guess it'll replace syncserver eventually.
But why I thought it might be interesting here is because I see potential in storing user data encrypted on arbitrary or user self-hosted storage. From the looks it also supports different auth mechanisms, premissions and storage backends https://kinto.readthedocs.io/en/latest/overview.html .

[ShalokShalom]

Yes and I remind that the Akasha community is already in public testing, so they might solve this already: http://akasha.world/