Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EMFILE when using StaticAssets with a lot of files. #94

Open
chakrit opened this issue Aug 31, 2013 · 3 comments
Open

EMFILE when using StaticAssets with a lot of files. #94

chakrit opened this issue Aug 31, 2013 · 3 comments

Comments

@chakrit
Copy link

chakrit commented Aug 31, 2013

I have made a test repository for this: https://github.com/chakrit/rack-emfile-issue It basically includes:

  • A minified distribution of ace.c9.io with all files.
  • A test file that creates a new Rack with a StaticAssets pointing to the ace files.

Basically, StaticAssets cannot seem to handle EMFILE well. Running node test.js result in this:

/private/tmp/test/node_modules/asset-rack/compiled/rack.js:42
          throw error;
                ^
Error: EMFILE, open '/private/tmp/test/public/ace/snippets/velocity.js'

A bit of googling leads me to https://github.com/isaacs/node-graceful-fs but not sure it'd be as simple as replacing all fs with graceful-fs. This is blocking me from working on one of my websites.

Will check up on the codebase and make a PR if I have time but if you can take a look at this I'd greatly appreciate it.


node --version is v0.10.17 and I'm on OS X ML.

@techpines
Copy link
Owner

@chakrit I just ran your test, and I didn't get the error. I would try upping the number of open file handles on your system.

graceful-js seems like a good option to me.

In general, asset-rack is not designed for "large" numbers of static assets as it holds everything in memory.

@chakrit
Copy link
Author

chakrit commented Sep 1, 2013

@techpines would you accept a graceful-fs patch then?

I think the problem might be that it is opening too many files too quickly during initialization to load things in memory.

@techpines
Copy link
Owner

I think so, as long as it didn't break anything.

chakrit pushed a commit to chakrit/asset-rack that referenced this issue Oct 31, 2013
Since this module always deal with lots and lots of static files
I believe it is a good idea to add a central point where all fs
calls can be traced/inspect/modifed/patched/whatever.

In this commit I'm simply replacing all `require 'fs'` found in
the codebase with require `require './fs.coffee' which now
simply exports the `graceful-fs` module.

This is to combat the EMFILE issue:
techpines#94
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants