Document & optimize git output (index)

- Improves point 1 of issue #3121 - the index is not rebuilded anymore
every time when we fetch the last config of a node
- Documents that the git output creates an index and how to manipulate
the git repo (don't!), documenting the problems encountered in #1805
(not solving the issue itself)

docs/Troubleshooting.md

@@ -3,6 +3,7 @@
 1. [Connects but no/partial configuration collected](d#oxidized-connects-to-a-supported-device-but-no-or-partial-configuration-is-collected)
 2. [No push to remote git repository](#oxidized-does-not-push-to-a-remote-git-repository-hook-githubrepo)
 3. [Git performance issues with large device counts](#git-performance-issues-with-large-device-counts)
+4. [Oxidized ignores the changes I made to its git repository](#oxidized-ignores-the-changes-i-made-to-its-git-repository)
 
 ## Oxidized connects to a supported device but no (or partial) configuration is collected
 
@@ -95,3 +96,22 @@ Follow these steps to do so:
 3. Change directory your oxidized git repository (as configured in oxidized configuration file)
 4. Execute the command `git gc` to run the garbage collection
 5. Restart oxidized - you're done!
+
+## Oxidized ignores the changes I made to its git repository
+First of all: you shouldn't manipulate the git repository of oxidized. Don't
+create it, don't modify it, leave it alone. You can break things. You have
+been warned.
+
+In some situations, you may need to make changes to the git repository of
+oxidized. Stop oxidized, make backups, and be sure you know exactly what you
+are doing. You have been warned.
+
+If you simply clone the git repository, make changes and push them, oxidized
+will ignore these modifications. This is because oxidized caches the HEAD tree
+in the index and `git push` does not update the index because the repository is
+a bare repo and not a working directory repository.
+
+So, you have to update the index manually. For this, go into oxidized repo, and
+run `git ls-tree -r HEAD | git update-index --index-info`. While you're at it,
+consider running `git gc`, as oxidized cannot garbage collect the repo (this
+is not supported in [Rugged](https://github.com/libgit2/rugged)).

lib/oxidized/nodes.rb

@@ -61,6 +61,9 @@ def show(node)
       end
     end
 
+    # Returns the configuration of group/node_name
+    #
+    # #fetch is called by oxidzed-web
     def fetch(node_name, group)
       yield_node_output(node_name) do |node, output|
         output.fetch node, group
@@ -98,6 +101,9 @@ def find_node_index(node)
       find_index(node) || raise(NodeNotFound, "unable to find '#{node}'")
     end
 
+    # Returns all stored versions of group/node_name
+    #
+    # Called by oxidized-web
     def version(node_name, group)
       yield_node_output(node_name) do |node, output|
         output.version node, group

lib/oxidized/output/git.rb

@@ -60,11 +60,19 @@ def store(file, outputs, opt = {})
       update repo, file, outputs.to_cfg
     end
 
+    # Returns the configuration of group/node_name
+    #
+    # #fetch is called by Nodes#fetch
+    # Nodes#fetch creates a new Output object each time, so we cannot
+    # store the repo index in memory. But as we keep the repo index up
+    # to date on disk in #update_repo, we can read it from disk instead of
+    # rebuilding it each time.
     def fetch(node, group)
       repo, path = yield_repo_and_path(node, group)
       repo = Rugged::Repository.new repo
+      # Read the index from disk
       index = repo.index
-      index.read_tree repo.head.target.tree unless repo.empty?
+
       repo.read(index.get(path)[:oid]).data
     rescue StandardError
       'node not found'
@@ -172,11 +180,26 @@ def update(repo, file, data)
       end
     end
 
+    # Uploads data into file in the repo
+    #
+    # @param [String] file: the file to save the configuration to
+    # @param [String] data: the configuration to save
+    # @param [Rugged::Repository] repo: the git repository to use
+    #
+    # If Oxidized.config.output.git.single_repo = false (which is the default),
+    # there will one repository for each group.
+    #
+    # update_repo caches the index on disk. An index is usually used in a
+    # working directory and not in a bare repository, which confuses users.
+    # The alternative would be to rebuild the index each time, which a little
+    # time consuming. Caching the index in memory is difficult because a new
+    # Output object is created each time #store is called.
     def update_repo(repo, file, data)
       oid_old = repo.blob_at(repo.head.target_id, file) rescue nil
       return false if oid_old && (oid_old.content.b == data.b)
 
       oid = repo.write data, :blob
+      # Read the index from disk
       index = repo.index
       index.add path: file, oid: oid, mode: 0o100644