Implement GridFS

[ramazanpolat]

Since ArangoDB competes with MongoDB, that would be nice to have something like GridFS implemented in driver.

In GridFS documentation, GridFS is defined as:

GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16 MB

I know ArangoDB doesn't have 16 MB limit but GridFS is not just for exceeding document size limits. It is used to store files in chunks. This will have some benefits for some use cases. For example putting profile picture to ArangoDB collection will make developers life much more easier. In past, I've used to store TBs of data in MongoDB that is accessed occasionally and it saved me lots of time to look for a distributed file system. I'd like to see ArangoDB drivers also implement GridFS. In fact, I may try to implement it if this gets accepted.

[dothebart]

probably related to arangodb/arangodb#107

[rashtao]

I think this is not driver related: if the db will support in the future binary fields/objects the driver will support them as well.

Currently you can achieve something similar from the driver encoding/decoding binary data to/from base64 encoded strings, so saving strings in the db.
Although in this case the stored data would be bigger than its binary representation, network transfers would be proportionally slower and the client would have to consume cpu for encoding/decoding.
Also pay attention to these limitations: arangodb/arangodb#10754

[ramazanpolat]

I think this is not driver related: if the db will support in the future binary fields/objects the driver will support them as well.

In case of MongoDB, GridFS is just a driver implementation of splitting files into chunks and inserting into a collection. MongoDB itself doesn't have any specific part for GridFS, it is all about the driver which does the job.

Currently you can achieve something similar from the driver encoding/decoding binary data to/from base64 encoded strings, so saving strings in the db.

This is actually what GridFS does.

Although in this case the stored data would be bigger than its binary representation, network transfers would be proportionally slower and the client would have to consume cpu for encoding/decoding.

Yes, you are right. The file will be bigger. But this is a trade-off which in some cases users will favor using it, especially for small sized files like profile pictures. Same trade-off also applies for GridFS.

Also pay attention to these limitations: arangodb/arangodb#10754

There are similar limitaions in MongoDB and this is why they come up with GridFS specification in first place.

[rashtao]

Thanks for clarifying, in general I like the idea. I would proceed defining a specification which will be then implemented by all the drivers.