The crate index is a git repository where metadata about all the crates managed by a registry is stored.
+In a way, it is a snapshot of a registry's knowledge of every crates.
Alexandrie will create the directories needed to store a crate automatically, so it suffices to only manually create the config.json file to get going.
+
+
The way the crate index is accessed is called a crate index management strategy (a bit of a mouthful, sorry about that 😅).
+
Strategies will allow Alexandrie to interact with crate indices in a variety of ways, not only locally but potentially remotely (using a litte server on another machine to perform the operation) where the registry itself doesn't have full access to the underlying git repository.
A crate store is the place where all the contents of the published crates (the actual code and assets) are stored.
+
The contents of crates are stored as TAR archives and compressed using Gzip (basically a .tar.gz blob).
+The store takes that blob, stores it and make it available for download later on.
+
Currently, the store is also responsible for storing rendered README pages (which are simple HTML files).
+
Because these can amount to a lot of storage space, it can be desirable to separate the crates' metadata (modelled by the crate index) and their actual contents (handled by the crate stores).
+
A crate store may be local (as files on disk, for instance) or remote (as blobs in AWS S3, for instance).
+For usage and configuration see Available crate stores.
This sections covers the different concepts that Alexandrie defines and uses.
+
The goal of these concepts is to formally define how the different parts of Alexandrie interacts.
+Alexandrie can operate with both its crate index and storage on different remote machines and this brings in the need for a well-defined definition of what each of these parts are responsible for.
+
Alexandrie is built around two principal concepts:
+
+
Crate index: manages metadata about published crates.
+
Crate store: stores actual contents of the published crates (code, assets, etc...).
+
+
A crate index is a git repository layed out as specified in Cargo's Alternative Crate Registries RFC.
+It stores metadata about each stored crate and their versions, but not the contents of the crates themselves.
+
A crate store is what takes care of storing the crate contents themselves and then making them available for download.
This page describes the database tables that Alexandrie uses.
+
The types of the tables' fields are chosen to be compatible with all the database vendors supported by Alexandrie.
+
Here is a high-level picture of the different tables and their relationships:
+
+
The crates table stores metadata about the latest version of each stored crate.
+It is meant to store the most useful metadata for fast access.
+If more complete metadata is required or metadata about an older version is needed, the crate-index can be queried for this.
+
The authors table stores the list of every registered crate authors of the registry.
+It stores details like emails, fullnames and a signature of the author's password.
+Details about what this signature really is, and how to compute it from the users' password is available at:
+https://polomack.eu/alexandrie-security
+
The keywords and categories tables store the keywords and categories used by the registry's crates.
+The keywords table can gain new entries as crates makes use of new distinct keywords.
+The categories table, on the other hand, is fixed and not expandable by crates.
+
The salts table stores the salts generated for each registered author and used to compute their passwords' signatures.
+The sessions table stores details about the frontend sessions for registered authors.
+Sessions are not automatically deleted when expired, so a cleanup periodic clean may be required to prune expired sessions.
+The author_tokens tables stores the generated Cargo authentication tokens for registered users (used for cargo login).
+
The crate_categories, crate_keywords and crate_authors are all one-to-many relationship tables.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/elasticlunr.min.js b/elasticlunr.min.js
new file mode 100644
index 00000000..94b20dd2
--- /dev/null
+++ b/elasticlunr.min.js
@@ -0,0 +1,10 @@
+/**
+ * elasticlunr - http://weixsong.github.io
+ * Lightweight full-text search engine in Javascript for browser search and offline search. - 0.9.5
+ *
+ * Copyright (C) 2017 Oliver Nightingale
+ * Copyright (C) 2017 Wei Song
+ * MIT Licensed
+ * @license
+ */
+!function(){function e(e){if(null===e||"object"!=typeof e)return e;var t=e.constructor();for(var n in e)e.hasOwnProperty(n)&&(t[n]=e[n]);return t}var t=function(e){var n=new t.Index;return n.pipeline.add(t.trimmer,t.stopWordFilter,t.stemmer),e&&e.call(n,n),n};t.version="0.9.5",lunr=t,t.utils={},t.utils.warn=function(e){return function(t){e.console&&console.warn&&console.warn(t)}}(this),t.utils.toString=function(e){return void 0===e||null===e?"":e.toString()},t.EventEmitter=function(){this.events={}},t.EventEmitter.prototype.addListener=function(){var e=Array.prototype.slice.call(arguments),t=e.pop(),n=e;if("function"!=typeof t)throw new TypeError("last argument must be a function");n.forEach(function(e){this.hasHandler(e)||(this.events[e]=[]),this.events[e].push(t)},this)},t.EventEmitter.prototype.removeListener=function(e,t){if(this.hasHandler(e)){var n=this.events[e].indexOf(t);-1!==n&&(this.events[e].splice(n,1),0==this.events[e].length&&delete this.events[e])}},t.EventEmitter.prototype.emit=function(e){if(this.hasHandler(e)){var t=Array.prototype.slice.call(arguments,1);this.events[e].forEach(function(e){e.apply(void 0,t)},this)}},t.EventEmitter.prototype.hasHandler=function(e){return e in this.events},t.tokenizer=function(e){if(!arguments.length||null===e||void 0===e)return[];if(Array.isArray(e)){var n=e.filter(function(e){return null===e||void 0===e?!1:!0});n=n.map(function(e){return t.utils.toString(e).toLowerCase()});var i=[];return n.forEach(function(e){var n=e.split(t.tokenizer.seperator);i=i.concat(n)},this),i}return e.toString().trim().toLowerCase().split(t.tokenizer.seperator)},t.tokenizer.defaultSeperator=/[\s\-]+/,t.tokenizer.seperator=t.tokenizer.defaultSeperator,t.tokenizer.setSeperator=function(e){null!==e&&void 0!==e&&"object"==typeof e&&(t.tokenizer.seperator=e)},t.tokenizer.resetSeperator=function(){t.tokenizer.seperator=t.tokenizer.defaultSeperator},t.tokenizer.getSeperator=function(){return t.tokenizer.seperator},t.Pipeline=function(){this._queue=[]},t.Pipeline.registeredFunctions={},t.Pipeline.registerFunction=function(e,n){n in t.Pipeline.registeredFunctions&&t.utils.warn("Overwriting existing registered function: "+n),e.label=n,t.Pipeline.registeredFunctions[n]=e},t.Pipeline.getRegisteredFunction=function(e){return e in t.Pipeline.registeredFunctions!=!0?null:t.Pipeline.registeredFunctions[e]},t.Pipeline.warnIfFunctionNotRegistered=function(e){var n=e.label&&e.label in this.registeredFunctions;n||t.utils.warn("Function is not registered with pipeline. This may cause problems when serialising the index.\n",e)},t.Pipeline.load=function(e){var n=new t.Pipeline;return e.forEach(function(e){var i=t.Pipeline.getRegisteredFunction(e);if(!i)throw new Error("Cannot load un-registered function: "+e);n.add(i)}),n},t.Pipeline.prototype.add=function(){var e=Array.prototype.slice.call(arguments);e.forEach(function(e){t.Pipeline.warnIfFunctionNotRegistered(e),this._queue.push(e)},this)},t.Pipeline.prototype.after=function(e,n){t.Pipeline.warnIfFunctionNotRegistered(n);var i=this._queue.indexOf(e);if(-1===i)throw new Error("Cannot find existingFn");this._queue.splice(i+1,0,n)},t.Pipeline.prototype.before=function(e,n){t.Pipeline.warnIfFunctionNotRegistered(n);var i=this._queue.indexOf(e);if(-1===i)throw new Error("Cannot find existingFn");this._queue.splice(i,0,n)},t.Pipeline.prototype.remove=function(e){var t=this._queue.indexOf(e);-1!==t&&this._queue.splice(t,1)},t.Pipeline.prototype.run=function(e){for(var t=[],n=e.length,i=this._queue.length,o=0;n>o;o++){for(var r=e[o],s=0;i>s&&(r=this._queue[s](r,o,e),void 0!==r&&null!==r);s++);void 0!==r&&null!==r&&t.push(r)}return t},t.Pipeline.prototype.reset=function(){this._queue=[]},t.Pipeline.prototype.get=function(){return this._queue},t.Pipeline.prototype.toJSON=function(){return this._queue.map(function(e){return t.Pipeline.warnIfFunctionNotRegistered(e),e.label})},t.Index=function(){this._fields=[],this._ref="id",this.pipeline=new t.Pipeline,this.documentStore=new t.DocumentStore,this.index={},this.eventEmitter=new t.EventEmitter,this._idfCache={},this.on("add","remove","update",function(){this._idfCache={}}.bind(this))},t.Index.prototype.on=function(){var e=Array.prototype.slice.call(arguments);return this.eventEmitter.addListener.apply(this.eventEmitter,e)},t.Index.prototype.off=function(e,t){return this.eventEmitter.removeListener(e,t)},t.Index.load=function(e){e.version!==t.version&&t.utils.warn("version mismatch: current "+t.version+" importing "+e.version);var n=new this;n._fields=e.fields,n._ref=e.ref,n.documentStore=t.DocumentStore.load(e.documentStore),n.pipeline=t.Pipeline.load(e.pipeline),n.index={};for(var i in e.index)n.index[i]=t.InvertedIndex.load(e.index[i]);return n},t.Index.prototype.addField=function(e){return this._fields.push(e),this.index[e]=new t.InvertedIndex,this},t.Index.prototype.setRef=function(e){return this._ref=e,this},t.Index.prototype.saveDocument=function(e){return this.documentStore=new t.DocumentStore(e),this},t.Index.prototype.addDoc=function(e,n){if(e){var n=void 0===n?!0:n,i=e[this._ref];this.documentStore.addDoc(i,e),this._fields.forEach(function(n){var o=this.pipeline.run(t.tokenizer(e[n]));this.documentStore.addFieldLength(i,n,o.length);var r={};o.forEach(function(e){e in r?r[e]+=1:r[e]=1},this);for(var s in r){var u=r[s];u=Math.sqrt(u),this.index[n].addToken(s,{ref:i,tf:u})}},this),n&&this.eventEmitter.emit("add",e,this)}},t.Index.prototype.removeDocByRef=function(e){if(e&&this.documentStore.isDocStored()!==!1&&this.documentStore.hasDoc(e)){var t=this.documentStore.getDoc(e);this.removeDoc(t,!1)}},t.Index.prototype.removeDoc=function(e,n){if(e){var n=void 0===n?!0:n,i=e[this._ref];this.documentStore.hasDoc(i)&&(this.documentStore.removeDoc(i),this._fields.forEach(function(n){var o=this.pipeline.run(t.tokenizer(e[n]));o.forEach(function(e){this.index[n].removeToken(e,i)},this)},this),n&&this.eventEmitter.emit("remove",e,this))}},t.Index.prototype.updateDoc=function(e,t){var t=void 0===t?!0:t;this.removeDocByRef(e[this._ref],!1),this.addDoc(e,!1),t&&this.eventEmitter.emit("update",e,this)},t.Index.prototype.idf=function(e,t){var n="@"+t+"/"+e;if(Object.prototype.hasOwnProperty.call(this._idfCache,n))return this._idfCache[n];var i=this.index[t].getDocFreq(e),o=1+Math.log(this.documentStore.length/(i+1));return this._idfCache[n]=o,o},t.Index.prototype.getFields=function(){return this._fields.slice()},t.Index.prototype.search=function(e,n){if(!e)return[];e="string"==typeof e?{any:e}:JSON.parse(JSON.stringify(e));var i=null;null!=n&&(i=JSON.stringify(n));for(var o=new t.Configuration(i,this.getFields()).get(),r={},s=Object.keys(e),u=0;u0&&t.push(e);for(var i in n)"docs"!==i&&"df"!==i&&this.expandToken(e+i,t,n[i]);return t},t.InvertedIndex.prototype.toJSON=function(){return{root:this.root}},t.Configuration=function(e,n){var e=e||"";if(void 0==n||null==n)throw new Error("fields should not be null");this.config={};var i;try{i=JSON.parse(e),this.buildUserConfig(i,n)}catch(o){t.utils.warn("user configuration parse failed, will use default configuration"),this.buildDefaultConfig(n)}},t.Configuration.prototype.buildDefaultConfig=function(e){this.reset(),e.forEach(function(e){this.config[e]={boost:1,bool:"OR",expand:!1}},this)},t.Configuration.prototype.buildUserConfig=function(e,n){var i="OR",o=!1;if(this.reset(),"bool"in e&&(i=e.bool||i),"expand"in e&&(o=e.expand||o),"fields"in e)for(var r in e.fields)if(n.indexOf(r)>-1){var s=e.fields[r],u=o;void 0!=s.expand&&(u=s.expand),this.config[r]={boost:s.boost||0===s.boost?s.boost:1,bool:s.bool||i,expand:u}}else t.utils.warn("field name in user configuration not found in index instance fields");else this.addAllFields2UserConfig(i,o,n)},t.Configuration.prototype.addAllFields2UserConfig=function(e,t,n){n.forEach(function(n){this.config[n]={boost:1,bool:e,expand:t}},this)},t.Configuration.prototype.get=function(){return this.config},t.Configuration.prototype.reset=function(){this.config={}},lunr.SortedSet=function(){this.length=0,this.elements=[]},lunr.SortedSet.load=function(e){var t=new this;return t.elements=e,t.length=e.length,t},lunr.SortedSet.prototype.add=function(){var e,t;for(e=0;e1;){if(r===e)return o;e>r&&(t=o),r>e&&(n=o),i=n-t,o=t+Math.floor(i/2),r=this.elements[o]}return r===e?o:-1},lunr.SortedSet.prototype.locationFor=function(e){for(var t=0,n=this.elements.length,i=n-t,o=t+Math.floor(i/2),r=this.elements[o];i>1;)e>r&&(t=o),r>e&&(n=o),i=n-t,o=t+Math.floor(i/2),r=this.elements[o];return r>e?o:e>r?o+1:void 0},lunr.SortedSet.prototype.intersect=function(e){for(var t=new lunr.SortedSet,n=0,i=0,o=this.length,r=e.length,s=this.elements,u=e.elements;;){if(n>o-1||i>r-1)break;s[n]!==u[i]?s[n]u[i]&&i++:(t.add(s[n]),n++,i++)}return t},lunr.SortedSet.prototype.clone=function(){var e=new lunr.SortedSet;return e.elements=this.toArray(),e.length=e.elements.length,e},lunr.SortedSet.prototype.union=function(e){var t,n,i;this.length>=e.length?(t=this,n=e):(t=e,n=this),i=t.clone();for(var o=0,r=n.toArray();o
+
+
+
+
+ Getting started - Alexandrie
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Before anything, keep in mind that the current minimum supported Rust version for Alexandrie is 1.68 (on stable), make sure to check if your local Rust version is adequate by running rustc -V.
+
Alexandrie offers multiple options to be used as its database, so before building Alexandrie, you'll need to pick which supported database vendor you wish to use.
+The currently supported vendors are:
+
+
sqlite: for SQLite
+
mysql: for MySQL (including MariaDB)
+
postgres: for PostgreSQL
+
+
To build, you can then run:
+
# navigate to the `crates/alexandrie/` subfolder
+cd crates/alexandrie
+
+# build the binary
+# (replace `(foo|bar)` occurences by one of `foo` or `bar`)
+# (replace `[foo]` occurences by either `foo` or nothing)
+cargo build [--release] \
+ --no-default-features \
+ --features "[frontend] (sqlite|mysql|postgres)"
+
+
Before running Alexandrie, you'll need to configure your instance in the alexandrie.toml file.
+
The database is configured through the [database] table:
+
[database]
+# Replace the '<...>' placeholders by their real actual values.
+
+# For MySQL
+url = "mysql://<user>:<password>@<hostname>:<port>/<database>"
+
+# For PostgreSQL
+url = "postgresql://<user>:<password>@<hostname>:<port>/<database>"
+
+# For SQLite
+url = "<path-to-sqlite-database-file>"
+# or:
+url = ":memory:" # ephemeral in-memory database, doesn't persists between restarts
+
+
Then, you can configure the crates' tarballs storage strategy and the crate index management strategy that you want to use.
You can find more information about crate index management and crate stores in the 'What's available' section of this book.
+
You can also configure things like the address and port of the server:
+
[general]
+bind_address = "127.0.0.1:3000" # The endpoint and port on which to serve the service.
+
+
Then, you need to configure a crate index.
+A crate index is simply a git repository that the registry uses to keep metadata information about each crate and their individual versions.
+The repository can be created on any machine you want, as long as it is reachable from the current machine as a git remote in a clone of that repository.
+The remote can be specified using either an HTTPS or SSH link.
+
If you're using SSH for the remote link, Cargo might have an issue where it can't fetch from the registry's index when doing cargo search or cargo build.
+This is because Cargo uses libgit2 to fetch from remotes and fails to find the SSH credentials needed to do so.
+To work around this issue, you may need to set the CARGO_NET_GIT_FETCH_WITH_CLI environment variable to true, so that Cargo will offload the fetching from remotes operation to the git command-line utility.
+See issue #44 for a previous occurence of this exact issue.
+
To run the registry with the configuration above, be sure to clone your crate index at the location designated by the path key in [index].
+In this case, it is ./crate-index.
+To clone an existing crate index, you can run:
+
# Replace the '<...>' placeholders by their real actual values.
+git clone <url-of-the-crate-index> <path-from-config>
+
+# <url-of-the-crate-index>: URL to the git repository serving as the registry's crate index.
+# <path-from-config>: Path to the same directory as the one specified as `index.path` in the `alexandrie.toml`.
+
+# Example:
+git clone 'https://github.com/Hirevo/alexandrie-index' 'crate-index'
+
Once everything is configured, you can run with: cargo run [--release].
+
Then, if you want to use this index with Cargo, you can follow these steps:
+
+
Edit or create the ~/.cargo/config file, and add the following code:
+
# Replace the '<...>' placeholders by their real actual values.
+[registries.<name-of-your-registry>]
+index = "<url-of-the-crate-index>"
+
+# <name-of-your-registry>: A name of your choosing, that you'll be using to refer to it in `cargo` commands.
+# <url-of-the-crate-index>: URL to the git repository serving as the registry's crate index.
+# BE CAREFUL: this is not the URL to the registry's API or frontend.
+
+
+
Then, run cargo login --registry <name-of-your-registry> and enter your author token.
+To generate a token, you need to register as an author first.
+You can do this using the frontend by:
+
+
Registering at /account/register.
+
Generating a token at /account/manage.
+
+
+
You can now use the registry using cargo [search|publish] --registry <name-of-your-registry>
If you wish to have a more concrete resource to learn how to setup an Alexandrie instance, like a shell script, you may refer to an example installation script which can help you get started:
Alexandrie is a modular alternative crate registry for Rust crates.
+
+
It implements the Rust's Alternative Crate Registries RFC in order to be usable with Cargo.
+It seeks to be modular, extensible and configurable to be usable in many different environments, from a openly-accessible crate store to a private registry in a company's toolchain.
+
It includes both a programmatic API and an optional server-rendered frontend.