Glide update fails for dependency in detached HEAD state

[dnathe4th]

I'm not entirely sure the root cause of this yet, but hopefully you can shed some light on what is going on, and whether this is a bug or not (it certainly feels like it).

My example project is as follows. Is it silly, but it is a combination of dependencies that experience this issue so bear with me:

main.go

package main

import (
        "fmt"

        _ "github.com/Sirupsen/logrus"
        _ "github.com/cactus/go-statsd-client"
        _ "github.com/onsi/ginkgo"
        _ "github.com/onsi/gomega"
        _ "github.com/opentracing/opentracing-go"
        "github.com/pborman/uuid"
        _ "github.com/uber/tchannel-go"
)

func main() {
        fmt.Println("my favorite uuid is", uuid.NewUUID())
}

glide.yaml

package: example
import:
- package: github.com/pborman/uuid
- package: github.com/uber/tchannel-go
- package: github.com/onsi/ginkgo
- package: github.com/onsi/gomega
- package: github.com/Sirupsen/logrus
- package: github.com/cactus/go-statsd-client
  subpackages:
  - statsd

With the project in this state, I can call glide update no problem, and the dependencies are installed:

~/gocode/src/example$ glide update
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/pborman/uuid.
[INFO] Fetching updates for github.com/onsi/ginkgo.
[INFO] Fetching updates for github.com/cactus/go-statsd-client.
[INFO] Fetching updates for github.com/onsi/gomega.
[INFO] Fetching updates for github.com/dnathe4th/tchannel-go.
[INFO] Fetching updates for github.com/Sirupsen/logrus.
[INFO] Resolving imports
[INFO] Fetching golang.org/x/sys/unix into /home/dnathe4th/gocode/src/example/vendor
[INFO] Fetching github.com/opentracing/opentracing-go into /home/dnathe4th/gocode/src/example/vendor
[INFO] Found Godeps.json file.
[INFO] Fetching golang.org/x/net/context into /home/dnathe4th/gocode/src/example/vendor
[INFO] Setting version for golang.org/x/net to 6c89489cafabcbc76df9dbf84ebf07204673fecf.
[INFO] Downloading dependencies. Please wait...
[INFO] Setting references for remaining imports
[INFO] Project relies on 9 dependencies.

However, if I immediately run glide update again, it errors:

~/gocode/src/example$ glide update
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/cactus/go-statsd-client.
[INFO] Fetching updates for github.com/dnathe4th/tchannel-go.
[INFO] Fetching updates for github.com/pborman/uuid.
[INFO] Fetching updates for github.com/onsi/gomega.
[INFO] Fetching updates for github.com/onsi/ginkgo.
[INFO] Fetching updates for github.com/Sirupsen/logrus.
[INFO] Resolving imports
[INFO] Fetching updates for golang.org/x/sys.
[INFO] Fetching updates for golang.org/x/net.
[WARN] Download failed.
[WARN] Error updating golang.org/x/net/context: exit status 1
[INFO] Found Godeps.json file.
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/opentracing/opentracing-go.
[INFO] Setting references for remaining imports
[INFO] Project relies on 9 dependencies.

Now, 'Download Failed' is not a terribly helpful message, so I did a bit of digging into glide leading me to vcs to try to get a bit more out. I had to (1) patch a proper logger to stdout onto vcs.Logger, and (2) update runFromDir to call b.log(out) just like run does after the CombinedOutput call, but that's a complaint for another repo.

With these fixes in place, and a recompiled glide, I can get a proper error!

~/gocode/src/example$ ./go-build/glide update
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/cactus/go-statsd-client.
[INFO] Fetching updates for github.com/uber/tchannel-go.
[INFO] Fetching updates for github.com/pborman/uuid.
[INFO] Fetching updates for github.com/onsi/gomega.
TRACE: 2016/03/30 01:01:30 repo.go:170:
[INFO] Fetching updates for github.com/Sirupsen/logrus.
[INFO] Fetching updates for github.com/onsi/ginkgo.
TRACE: 2016/03/30 01:01:30 repo.go:170:
TRACE: 2016/03/30 01:01:30 repo.go:170:
TRACE: 2016/03/30 01:01:35 repo.go:170:
TRACE: 2016/03/30 01:01:35 repo.go:170:
TRACE: 2016/03/30 01:01:35 repo.go:170:
TRACE: 2016/03/30 01:01:36 repo.go:170: Already up-to-date.
TRACE: 2016/03/30 01:01:36 repo.go:170: Already up-to-date.
TRACE: 2016/03/30 01:01:36 repo.go:170: Already up-to-date.
TRACE: 2016/03/30 01:01:36 repo.go:170: 4b6ea7319e214d98c938f12692336f7ca9348d6b
TRACE: 2016/03/30 01:01:36 repo.go:170: 7ce781ea776b2fd506491011353bded2e40c8467
TRACE: 2016/03/30 01:01:36 repo.go:170: c55201b036063326c5b1b89ccfe45a184973d073
TRACE: 2016/03/30 01:01:36 repo.go:170: 8f1ed1aa5bb22b4e9f2ced0e736ff771b34dcb39
TRACE: 2016/03/30 01:01:36 repo.go:170: 5517f304319b9a066354b84860cb264c89655f69
TRACE: 2016/03/30 01:01:36 repo.go:170: c3a655f9b37a2207fa39721b078b3173ee9218d6
[INFO] Resolving imports
[INFO] Fetching updates for golang.org/x/sys.
TRACE: 2016/03/30 01:01:37 repo.go:170:
TRACE: 2016/03/30 01:01:37 repo.go:170:
TRACE: 2016/03/30 01:01:37 repo.go:170: Already up-to-date.
TRACE: 2016/03/30 01:01:37 repo.go:170: 320cb01ddbbf0473674c2585f9b6e245721de355
[INFO] Fetching updates for golang.org/x/net.
TRACE: 2016/03/30 01:01:38 repo.go:170:
TRACE: 2016/03/30 01:01:38 repo.go:170:
TRACE: 2016/03/30 01:01:39 repo.go:170: You are not currently on a branch, so I cannot use any
'branch.<branchname>.merge' in your configuration file.
Please specify which branch you want to merge with on the command
line and try again (e.g. 'git pull <repository> <refspec>').
See git-pull(1) for details.
[WARN] Download failed.
[WARN] Error updating golang.org/x/net/context: exit status 1
[INFO] Found Godeps.json file in vendor/github.com/dnathe4th/tchannel-go
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/opentracing/opentracing-go.
TRACE: 2016/03/30 01:01:39 repo.go:170:
TRACE: 2016/03/30 01:01:40 repo.go:170:
TRACE: 2016/03/30 01:01:41 repo.go:170: Already up-to-date.
[INFO] Setting references for remaining imports
TRACE: 2016/03/30 01:01:41 repo.go:170: 01498abd158dfdbe8e251856bc7d14cb0f046fa3
TRACE: 2016/03/30 01:01:41 repo.go:170: 6c89489cafabcbc76df9dbf84ebf07204673fecf
[INFO] Project relies on 9 dependencies.

So, somehow I ended up with a repo that was checked out to a ref which no longer allowed glide update to update it.

Note: I came across #356 earlier, and with that in mind tested adding version: master to every package listed in glide.yaml. After deleting the vendor directory and doing a fresh install from glide, this seemed to fix this particular issue.

I still can't figure out exactly why that combination of dependencies results in an unrecoverable detached HEAD state, since in my testing the vendor directory always has its git repos in a detached state to the ref in glide.lock, but maybe I am missing something.

Thoughts?

[technosophos]

Can you drop your glide.lock in here, too?

[technosophos]

Also, just to rule out one possibility... did you commit your vendor/ directory to VCS? The error above is sometimes related to trying to check one git repo into another git repo.

[dnathe4th]

~/gocode/src/example$ cat glide.lock
hash: 508d7f512759bda6eb412a18ab30d992fbfdcd05fd0ded5d50511345ca065181
updated: 2016-03-30T01:01:41.170110135Z
imports:
- name: github.com/cactus/go-statsd-client
  version: 5517f304319b9a066354b84860cb264c89655f69
  subpackages:
  - statsd
- name: github.com/onsi/ginkgo
  version: c3a655f9b37a2207fa39721b078b3173ee9218d6
  subpackages:
  - config
  - internal/codelocation
  - internal/failer
  - internal/remote
  - internal/suite
  - internal/testingtproxy
  - internal/writer
  - reporters
  - reporters/stenographer
  - types
  - internal/containernode
  - internal/leafnodes
  - internal/spec
  - internal/specrunner
- name: github.com/onsi/gomega
  version: 7ce781ea776b2fd506491011353bded2e40c8467
  subpackages:
  - internal/assertion
  - internal/asyncassertion
  - internal/testingtsupport
  - matchers
  - types
  - internal/oraclematcher
  - format
  - matchers/support/goraph/bipartitegraph
  - matchers/support/goraph/edge
  - matchers/support/goraph/node
  - matchers/support/goraph/util
- name: github.com/opentracing/opentracing-go
  version: 01498abd158dfdbe8e251856bc7d14cb0f046fa3
- name: github.com/pborman/uuid
  version: c55201b036063326c5b1b89ccfe45a184973d073
- name: github.com/Sirupsen/logrus
  version: 4b6ea7319e214d98c938f12692336f7ca9348d6b
- name: github.com/uber/tchannel-go
  version: 8f1ed1aa5bb22b4e9f2ced0e736ff771b34dcb39
  subpackages:
  - atomic
  - tnet
  - typed
- name: golang.org/x/net
  version: 6c89489cafabcbc76df9dbf84ebf07204673fecf
  subpackages:
  - context
- name: golang.org/x/sys
  version: 320cb01ddbbf0473674c2585f9b6e245721de355
  subpackages:
  - unix
devImports: []

No committed vendor/, in this example I haven't actually even run git init in ~/gocode/src/example so the only repos are the git dependencies.

[technosophos]

Hmm... I tried to reproduce and can't get that result.

My glide.yaml file:

package: technosophos.com/x/example
import:
- package: github.com/pborman/uuid
- package: github.com/uber/tchannel-go
- package: github.com/onsi/ginkgo
- package: github.com/onsi/gomega
- package: github.com/Sirupsen/logrus
- package: github.com/cactus/go-statsd-client
  subpackages:
  - statsd

My console:

⇒  glide up
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/uber/tchannel-go.
[INFO] Fetching updates for github.com/pborman/uuid.
[INFO] Fetching updates for github.com/onsi/ginkgo.
[INFO] Fetching updates for github.com/cactus/go-statsd-client.
[INFO] Fetching updates for github.com/Sirupsen/logrus.
[INFO] Fetching updates for github.com/onsi/gomega.
[INFO] Resolving imports
[INFO] Found Godeps.json file.
[INFO] Fetching golang.org/x/net/context into /Users/mattbutcher/Code/Go/src/technosophos/x/example/vendor
[INFO] Setting version for golang.org/x/net to 6c89489cafabcbc76df9dbf84ebf07204673fecf.
[INFO] Downloading dependencies. Please wait...
[INFO] Setting references for remaining imports
[INFO] Project relies on 7 dependencies.
☃ @ENG001590.local ~/Code/Go/src/technosophos/x/example «/Users/mattbutcher/Code»
⇒  glide up
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/onsi/gomega.
[INFO] Fetching updates for github.com/uber/tchannel-go.
[INFO] Fetching updates for github.com/onsi/ginkgo.
[INFO] Fetching updates for github.com/Sirupsen/logrus.
[INFO] Fetching updates for github.com/cactus/go-statsd-client.
[INFO] Fetching updates for github.com/pborman/uuid.
[INFO] Resolving imports
[INFO] Found Godeps.json file.
[INFO] Fetching updates for golang.org/x/net.
[INFO] golang.org/x/net is already set to version 6c89489cafabcbc76df9dbf84ebf07204673fecf. Skipping update.
[INFO] Setting version for golang.org/x/net to 6c89489cafabcbc76df9dbf84ebf07204673fecf.
[INFO] Downloading dependencies. Please wait...
[INFO] Setting references for remaining imports
[INFO] Project relies on 7 dependencies.

Go env:

go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mattbutcher/Code/Go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.6/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.6/libexec/pkg/tool/darwin_amd64"
GO15VENDOREXPERIMENT="1"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"

I'm on Glide 0.10.1

[technosophos]

I'm going to tag @mattfarina on this one, since he's the owner of the VCS code, too. I took a look again this morning, and am not sure why a detached head state should cause a problem in this case.

[dnathe4th]

Strange, thanks for investigating so far. I'm building glide from a1e937b, which looks like it's 0.9.3 and then some.

[technosophos]

Any chance you could try with 0.10.1?

[dnathe4th]

No dice on 0.10.1-1-g844a771, or 0.9.3 proper.

Something strange though, why does your output say [INFO] Project relies on 7 dependencies. but I get [INFO] Project relies on 9 dependencies.? Something is still inconsistent between our setups.

[mattfarina]

Now that I'm back from vacation I'll take a look at this.

[mattfarina]

To help with investigating... there are a few things at play here.

1. There are two phases. First is the update and then there is the version setting.
2. When in a detached head state Git does a fetch but doesn't pull (because it would cause an error).
3. When version setting happens if there is no version set it skips setting the version.

If this is the case there is a possibility to get into a detached head state and updates stop. This is a bug. What we need to do if try to make the default (when no version is set) smarter so this detached head state isn't such an issue.

[ax-nathan]

Just to help those who (like me) have been repeatedly bitten by this, adding version: master for every package in your glide.yaml file seems to be an acceptable workaround for the moment...

[mattfarina]

@ax-nathan I'd actually like to automate some of those things. I recently made changes to the vcs package to allow me to do that more easily. So, I agree completely.

[ax-nathan]

@mattfarina, sounds like a good idea to me!

[mattfarina]

A couple things....

• Can anyone who had this try to reproduce. I think the latest codebase fixes this problem.
• Using the --debug flag will now display the output to a failed vcs command if there was any. No need to path in a logger now.

[ippy04]

This seems to work now for me with v0.11.1_1 installed from homebrew if I specify version: master in my glide.yaml. Previously setting version: master still mostly didn't work, and I'd have to delete the vendor/github.com/repo directory as a workaround before running glide up to ensure the last version made it into glide.lock.

Without version: master though glide.lock still isn't being updated properly.

[mattfarina]

@ippy04 did you try the tip of master? The latest code as of today?

[ippy04]

@mattfarina Just tried with latest tip of master and everything seems to work without version: master. Didn't realize how many of my vendored packages were out of date!

[mattfarina]

@ippy04 thanks for checking. Glad this is fixed.