Skip to content
This repository has been archived by the owner on Jan 31, 2020. It is now read-only.

App works fine locally and appears to deploy but "silently" fails to actually start #10

Open
sjmcdowall opened this issue Feb 24, 2015 · 6 comments
Open

App works fine locally and appears to deploy but "silently" fails to actually start #10

sjmcdowall opened this issue Feb 24, 2015 · 6 comments

Comments

@sjmcdowall
Copy link

I have a meteor app that works fine locally -- but when I deploy to bluemix everything appears proper yet the app just silently fails to launch (or so it appears from the logs -- or more precisely the lack of them). Here are the logs from the latest cf push attempt:

2015-02-24T09:12:33.75-0500 [API/0] OUT Updated app with guid 6710b001-87ee-469a-b7fd-7fb8730042b6 ({"name"=>"RTSDash", "command"=>"PRIVATE DATA HIDDEN", "instances"=>1, "memory"=>256, "disk_quota"=>512, "buildpack"=>"https://github.com/ind1go/bluemix-buildpack-meteor.git", "environment_json"=>"PRIVATE DATA HIDDEN", "health_check_timeout"=>80})
2015-02-24T09:12:44.69-0500 [API/10] OUT Updated app with guid 6710b001-87ee-469a-b7fd-7fb8730042b6 ({"state"=>"STOPPED"})
2015-02-24T09:12:46.71-0500 [DEA/102] OUT Got staging request for app with id 6710b001-87ee-469a-b7fd-7fb8730042b6
2015-02-24T09:13:03.28-0500 [STG/102] OUT -----> Downloaded app package (12K)
2015-02-24T09:13:03.45-0500 [STG/102] OUT -----> Downloaded app buildpack cache (4.0K)
2015-02-24T09:13:03.66-0500 [STG/0] ERR Cloning into '/tmp/buildpacks/bluemix-buildpack-meteor'...
2015-02-24T09:13:04.28-0500 [STG/0] OUT -----> Resolving engine versions
2015-02-24T09:13:04.29-0500 [STG/0] OUT No version of Node.js specified in nodeversion, using '0.10.33'
2015-02-24T09:13:04.60-0500 [STG/0] OUT Using Node.js version: 0.10.33
2015-02-24T09:13:16.99-0500 [STG/0] OUT Installing meteor
2015-02-24T09:13:17.34-0500 [STG/0] ERR % Total % Received % Xferd Average Speed Time Time Time Current
2015-02-24T09:13:17.34-0500 [STG/0] ERR Dload Upload Total Spent Left Speed
100 6121 0 6121 0 0 13377 0 --:--:-- --:--:-- --:--:-- 46371
2015-02-24T09:13:18.00-0500 [STG/0] ERR Downloading Meteor distribution
2015-02-24T09:13:21.53-0500 [STG/0] ERR Dropped log message: message too long (>64K without a newline)
2015-02-24T09:13:24.26-0500 [STG/0] ERR Dropped log message: message too long (>64K without a newline)
2015-02-24T09:13:26.35-0500 [STG/0] ERR Dropped log message: message too long (>64K without a newline)
2015-02-24T09:13:28.08-0500 [STG/0] ERR Dropped log message: message too long (>64K without a newline)
2015-02-24T09:13:31.80-0500 [STG/0] ERR Dropped log message: message too long (>64K without a newline)

################################################################## 100.0% 88.5%

2015-02-24T09:13:35.05-0500 [STG/0] ERR Meteor 1.0.3.1 has been installed in your home directory (~/.meteor).
2015-02-24T09:13:35.06-0500 [STG/0] ERR Writing a launcher script to /usr/local/bin/meteor for your convenience.
2015-02-24T09:13:35.06-0500 [STG/0] ERR This may prompt for your password.
2015-02-24T09:13:35.11-0500 [STG/0] ERR sudo: no tty present and no askpass program specified
2015-02-24T09:13:35.12-0500 [STG/0] ERR Couldn't write the launcher script. Please either:
2015-02-24T09:13:35.12-0500 [STG/0] ERR (1) Run the following as root:
2015-02-24T09:13:35.12-0500 [STG/0] ERR cp "/tmp/node-meteor-ahFZ/.meteor/packages/meteor-tool/1.0.40/meteor-tool-os.linux.x86_64/scripts/admin/launch-meteor" /usr/bin/meteor
2015-02-24T09:13:35.12-0500 [STG/0] ERR (2) Add "$HOME/.meteor" to your path, or
2015-02-24T09:13:35.12-0500 [STG/0] ERR (3) Rerun this command to try again.
2015-02-24T09:13:35.12-0500 [STG/0] ERR Then to get started, take a look at 'meteor --help' or see the docs at
2015-02-24T09:13:35.12-0500 [STG/0] ERR docs.meteor.com.
2015-02-24T09:13:35.13-0500 [STG/0] OUT Meteor installed
2015-02-24T09:13:40.78-0500 [STG/0] OUT WARNING: The output directory is under your source tree.
2015-02-24T09:13:54.24-0500 [STG/0] OUT Installing meteor bundle
2015-02-24T09:13:54.90-0500 [STG/0] OUT npm WARN package.json [email protected] No description
2015-02-24T09:13:54.90-0500 [STG/0] OUT npm WARN package.json [email protected] No repository field.
2015-02-24T09:13:54.90-0500 [STG/0] OUT npm WARN package.json [email protected] No README data
2015-02-24T09:13:56.64-0500 [STG/0] OUT > [email protected] install /tmp/staged/app/.meteor/meteor_build/app/bundle/programs/server/node_modules/fibers
2015-02-24T09:13:56.64-0500 [STG/0] OUT > node ./build.js
2015-02-24T09:13:56.86-0500 [STG/0] OUT Binary is fine; exiting
2015-02-24T09:13:56.91-0500 [STG/0] OUT [email protected] node_modules/underscore
2015-02-24T09:13:56.91-0500 [STG/0] OUT [email protected] node_modules/semver
2015-02-24T09:13:56.91-0500 [STG/0] OUT [email protected] node_modules/eachline
2015-02-24T09:13:56.91-0500 [STG/0] OUT [email protected] node_modules/chalk
2015-02-24T09:13:56.91-0500 [STG/0] OUT ├── [email protected]
2015-02-24T09:13:56.91-0500 [STG/0] OUT ├── [email protected]
2015-02-24T09:13:56.91-0500 [STG/0] OUT ├── [email protected]
2015-02-24T09:13:56.91-0500 [STG/0] OUT ├── [email protected] ([email protected])
2015-02-24T09:13:56.91-0500 [STG/0] OUT └── [email protected] ([email protected])
2015-02-24T09:13:56.91-0500 [STG/0] OUT [email protected] node_modules/source-map-support
2015-02-24T09:13:56.91-0500 [STG/0] OUT └── [email protected] ([email protected])
2015-02-24T09:13:56.91-0500 [STG/0] OUT [email protected] node_modules/fibers
2015-02-24T09:13:56.97-0500 [STG/0] OUT Setting up MONGO_URL in .profile.d
2015-02-24T09:13:56.97-0500 [STG/0] OUT Looking for a service labelled "mongolab" in the following $VCAP_SERVICES...
2015-02-24T09:13:56.97-0500 [STG/0] OUT {"mongolab":[{"name":"mongo100","label":"mongolab","tags":["Data Stores","Development Tools","Infrastructure","Service","data_management","ibm_third_party"],"plan":"sandbox","credentials":{"uri":"mongodb://IbmCloud_t7vtsm40_icer83ci_84k4gkco:[email protected]:35760/IbmCloud_t7vtsm40_icer83ci"}}]}
2015-02-24T09:13:57.06-0500 [STG/0] OUT #!/bin/sh
2015-02-24T09:13:57.06-0500 [STG/0] OUT export ROOT_URL=${ROOT_URL:-http://gxs-rtsdash.mybluemix.net}
2015-02-24T09:13:57.06-0500 [STG/0] OUT export MONGO_URL=${MONGO_URL:-mongodb://IbmCloud_t7vtsm40_icer83ci_84k4gkco:[email protected]:35760/IbmCloud_t7vtsm40_icer83ci}
2015-02-24T09:13:57.06-0500 [STG/0] OUT export
2015-02-24T09:13:57.06-0500 [STG/0] OUT -----> Vendoring node into slug
2015-02-24T09:13:59.82-0500 [STG/102] OUT -----> Uploading droplet (7.7M)
2015-02-24T09:14:03.95-0500 [DEA/102] OUT Starting app instance (index 0) with guid 6710b001-87ee-469a-b7fd-7fb8730042b6
2015-02-24T09:14:21.84-0500 [API/4] OUT App instance exited with guid 6710b001-87ee-469a-b7fd-7fb8730042b6 payload: {"cc_partition"=>"default", "droplet"=>"6710b001-87ee-469a-b7fd-7fb8730042b6", "version"=>"5b972dfb-f4ef-4ec0-bd72-52944f2e3a01", "instance"=>"ac0c4837bd944da3ba73ccbf5a909fa1", "index"=>0, "reason"=>"CRASHED", "exit_status"=>127, "exit_description"=>"failed to accept connections within health check timeout", "crash_timestamp"=>1424787261}
2015-02-24T09:14:21.85-0500 [API/2] OUT App instance exited with guid 6710b001-87ee-469a-b7fd-7fb8730042b6 payload: {"cc_partition"=>"default", "droplet"=>"6710b001-87ee-469a-b7fd-7fb8730042b6", "version"=>"5b972dfb-f4ef-4ec0-bd72-52944f2e3a01", "instance"=>"ac0c4837bd944da3ba73ccbf5a909fa1", "index"=>0, "reason"=>"CRASHED", "exit_status"=>127, "exit_description"=>"failed to accept connections within health check timeout", "crash_timestamp"=>1424787261}
2015-02-24T09:15:00.64-0500 [DEA/93] OUT Starting app instance (index 0) with guid 6710b001-87ee-469a-b7fd-7fb8730042b6
2015-02-24T09:15:14.77-0500 [DEA/93] ERR Instance (index 0) failed to start accepting connections
2015-02-24T09:15:14.78-0500 [API/2] OUT App instance exited with guid 6710b001-87ee-469a-b7fd-7fb8730042b6 payload: {"cc_partition"=>"default", "droplet"=>"6710b001-87ee-469a-b7fd-7fb8730042b6", "version"=>"5b972dfb-f4ef-4ec0-bd72-52944f2e3a01", "instance"=>"bc768ea8308245f1baf2675093fabbe2", "index"=>0, "reason"=>"CRASHED", "exit_status"=>127, "exit_description"=>"failed to accept connections within health check timeout", "crash_timestamp"=>1424787314}
2015-02-24T09:15:14.79-0500 [API/2] OUT App instance exited with guid 6710b001-87ee-469a-b7fd-7fb8730042b6 payload: {"cc_partition"=>"default", "droplet"=>"6710b001-87ee-469a-b7fd-7fb8730042b6", "version"=>"5b972dfb-f4ef-4ec0-bd72-52944f2e3a01", "instance"=>"bc768ea8308245f1baf2675093fabbe2", "index"=>0, "reason"=>"CRASHED", "exit_status"=>127, "exit_description"=>"failed to accept connections within health check timeout", "crash_timestamp"=>1424787314}
2015-02-24T09:16:00.80-0500 [DEA/56] OUT Starting app instance (index 0) with guid 6710b001-87ee-469a-b7fd-7fb8730042b6
2015-02-24T09:16:18.28-0500 [API/2] OUT App instance exited with guid 6710b001-87ee-469a-b7fd-7fb8730042b6 payload: {"cc_partition"=>"default", "droplet"=>"6710b001-87ee-469a-b7fd-7fb8730042b6", "version"=>"5b972dfb-f4ef-4ec0-bd72-52944f2e3a01", "instance"=>"5aaea7a3b71845d691b8cfcfa56aa363", "index"=>0, "reason"=>"CRASHED", "exit_status"=>127, "exit_description"=>"app instance exited", "crash_timestamp"=>1424787378}
@ind1go
Copy link
Owner

ind1go commented Feb 24, 2015

Thanks for moving this! I've found that one fairly big difficulty of Node on CloudFoundry is that it's really difficult to debug startup problems, and then wrapping Node in Meteor compounds the problem.

Could you, to attempt to try to narrow it down, go through the same process that the buildpack does? i.e., from the root of the project, do the following (I think... I may have missed something out here):

meteor build ../built-app --directory
cd ../built-app/bundle/programs/server
npm install
node boot.js program.json

@sjmcdowall
Copy link
Author

OK — took a bit of work, but I got it to run ok following the below EXCEPT I had to change the version of fibers to 1.0.5, before that it failed horribly ..

I changed the npm-shrinkwrap.json AND packages.json before doing the npm install. Not sure if this is helpful or not.

Oh in the instructions below — it’s important to remember to set ROOT_URL and MONGO_URL before trying to launch.. :)

/Steve

On Feb 24, 2015, at 9:59 AM, Ben Cox [email protected] wrote:

Thanks for moving this! I've found that one fairly big difficulty of Node on CloudFoundry is that it's really difficult to debug startup problems, and then wrapping Node in Meteor compounds the problem.

Could you, to attempt to try to narrow it down, go through the same process that the buildpack does? i.e., from the root of the project, do the following (I think... I may have missed something out here):

meteor build ../built-app --directory
cd ../built-app/bundle/programs/server
npm install
node boot.js program.json
—
Reply to this email directly or view it on GitHub #10 (comment).

@ind1go
Copy link
Owner

ind1go commented Feb 24, 2015

OK then, I think it might be a case of trying to reduce your application to the minimal application that fails vs maximal application that works, I'm afraid, to try to work out what's going on - I'm not sure of a better way of diagnosing.

@ind1go
Copy link
Owner

ind1go commented Feb 24, 2015

Perhaps also see whether cf events <APP_NAME> gives you anything more.

@sjmcdowall
Copy link
Author

Nothing interesting on cf events. Just the fact it “crashed” (or crashes).

You don’t think it’s the use of [email protected] instead of @1.0.5 ? Hmm.. probably not as it appears [email protected] is for node 0.10.XX and [email protected] is for later node versions (like 0.12 which I am running locally). Sigh ..

Nevermind — fibers is a red-herring. Just downgraded by node back to 0.10.33 and it runs fine with [email protected]

Damn I wish there was a way to peer into the what’s going inside when it actually tries to run the thing.. :)

On Feb 24, 2015, at 12:25 PM, Ben Cox [email protected] wrote:

Perhaps also see whether cf events gives you anything more.

—
Reply to this email directly or view it on GitHub #10 (comment).

@ind1go
Copy link
Owner

ind1go commented Feb 24, 2015

I guess another area of research might be the memory usage - is it a particularly large app? Increasing memory to 1G would allow us to write off that possibility.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ind1go @sjmcdowall