No Browserify

[leetreveil]

Node 4, cross platform buffers without browserify!

[ondras]

Hi @leetreveil,

I would love to have a browser-based library that reads (music) metadata from an ArrayBuffer instance (I already have all data in memory, because I feed it to AudioContext.prototype.decodeAudioData).

This particular branch seems to be the best fit I could find. What is the current status regarding the non-browserify arraybuffer-accepting build? May I somehow help with the development?

[leetreveil]

Hi @ondras,

I would love some help with this!

Getting this codebase to a state where it can accept an ArrayBuffer as input would require us to:

1) Fix the last failing test on this branch (running node v4.2.3)

This requires us to update strtok2 (https://github.com/andrewrk/node-strtok) to use Uint8Array instead of Buffer like what's been done in this branch.

Fix all diffs like this:

-  var value = this.data.toString('utf8', this.offset, this.offset + len)
+  var value = String.fromCharCode.apply(null, this.data.slice(this.offset, this.offset + len))

To properly decode utf8 encoded array buffers. Would need to implement our own ala:

https://github.com/leetreveil/musicmetadata/blob/master/lib/windows1252decoder.js

Or find one on the net.

2) Wrap ArrayBuffer to expose a streaming API for input to the parsers:

musicmetadata/lib/ogg.js

Line 23 in e064112

strtok.parse(stream, function (v, cb) {

Might just be able to push the bytes to a stream:

musicmetadata/lib/browser.js

Line 12 in e064112

function wrapFileWithStream (file) {

This doesn't get us off browserify but it should allow us to pass an ArrayBuffer as input. Let me know if you have anymore questions.

[ondras]

Hi @leetreveil,

thanks for your prompt response!

As far as the decoding goes, what about this (standard) interface: https://developer.mozilla.org/en-US/docs/Web/API/TextDecoder ?

I assume that TextDecoder is not present in node (yet), but Uint8Array already is? It might be possible to create s small TextDecoder node-only module that wraps the current Buffer.prototype.toString() behavior...

[leetreveil]

I've looked at TextDecoder before, it's a monster dependency. So including it in the browserify output would bloat the output massively. We only need to support a couple decodings anyway.

This could work: https://github.com/nfroidure/UTF8.js

[leetreveil]

Also, if you want ArrayBuffer support you may just be able to work on 2) without the other step and let browserify handle the buffer stuff for you.

[ondras]

I've looked at TextDecoder before, it's a monster dependency. So including it in the browserify output would bloat the output massively. We only need to support a couple decodings anyway.

My point is that TextDecoder is already implemented in modern browsers, so there really is no need to bundle its implementation.

But if the strtok2 is to use TextDecoder internally, we must make sure this API exists in node as well.

[leetreveil]

Yeah I think that's reasonable. I would be happy to take a PR that uses TextDecoder. We may be able to use TextDecoder instead of node's own decoders but I would like to see some perf comparisons first, if TextDecoder is any slower we might want to fallback to node's decoders in non browser environments.

We don't have to worry about strtok either, we can just use BufferType instead of StringType and do the text decoding ourselves:

https://github.com/leetreveil/musicmetadata/search?utf8=%E2%9C%93&q=StringType