Skip to content

Commit

Permalink
docs: restore and typos
Browse files Browse the repository at this point in the history
  • Loading branch information
topless committed Mar 13, 2020
1 parent dd19fe6 commit 54a397f
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 14 deletions.
55 changes: 44 additions & 11 deletions docs/operations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
Operations
==========

Since our only copy of statts is stored in the indices of Elasticsearch in case
of a cluster error or failure we will lose our stats data. Thus it is adviced
Since our only copy of stats is stored in the indices of Elasticsearch in case
of a cluster error or failure we will lose our stats data. Thus it is advised
to setup a backup/restore mechanism for projects in production.

We have several options when it comes down to tooling and methods for preserving
Expand Down Expand Up @@ -42,7 +42,9 @@ Backup with elasticdump

.. note::
Apart from the data, you will also have to backup the mappings, so you are
able to restore data properly.
able to restore data properly. The following example will backup only stats
for record-views (not the events), you can go through your indices and
select which ones make sense to backup.


Save our mappings and our index data to record_view_mapping_backup.json and
Expand All @@ -51,7 +53,7 @@ record_view_index_backup.json files respectively.
.. code-block:: console
$ elasticdump \
> --input=http://production.es.com:9200/stats-record-view \
> --input=http://localhost:9200/stats-record-view-2020-03 \
> --output=record_view_mapping_backup.json \
> --type=mapping
Expand All @@ -63,7 +65,7 @@ record_view_index_backup.json files respectively.
Fri, 13 Mar 2020 13:13:01 GMT | dump complete
$ elasticdump \
> --input=http://production.es.com:9200/stats-record-view \
> --input=http://localhost:9200/stats-record-view-2020-03 \
> --output=record_view_index_backup.json \
> --type=data
Expand All @@ -74,15 +76,46 @@ record_view_index_backup.json files respectively.
Fri, 13 Mar 2020 13:13:13 GMT | Total Writes: 5
Fri, 13 Mar 2020 13:13:13 GMT | dump complete
In order to test restore functionality below I will delete on purpose the
index we backed up, from my instance.

.. code-block:: console
$ curl -XDELETE http://localhost:9200/stats-record-view-2020-03
{"acknowledged":true}
Restore with elasticdump
~~~~~~~~~~~~~~~~~~~~~~~~

There is a saying that goes "A backup worked only when it got restored." This
section will take us through the restore process of the previous step. We will
have to bring our application close to the state it was before the ES cluster
failure.
As we are all aware a backup did not work until it gets restored. Note that
before importing our data, we need to import the mappings to re-create the index.
The process is identical with the backup with just reversed sources --input and
--output.


.. code-block:: console
Some data loss is possible, from the time we notice the issue and restore
our cluster and its data to the last valid backed up dataset.
$ elasticdump \
> --input=record_view_mapping_backup.json \
> --output=http://localhost:9200/stats-record-view-2020-03 \
> --type=mapping
Fri, 13 Mar 2020 15:22:17 GMT | starting dump
Fri, 13 Mar 2020 15:22:17 GMT | got 1 objects from source file (offset: 0)
Fri, 13 Mar 2020 15:22:17 GMT | sent 1 objects to destination elasticsearch, wrote 4
Fri, 13 Mar 2020 15:22:17 GMT | got 0 objects from source file (offset: 1)
Fri, 13 Mar 2020 15:22:17 GMT | Total Writes: 4
Fri, 13 Mar 2020 15:22:17 GMT | dump complete
$ elasticdump \
> --input=record_view_mapping_backup.json \
> --output=http://localhost:9200/stats-record-view-2020-03 \
> --type=mapping
Fri, 13 Mar 2020 15:23:01 GMT | starting dump
Fri, 13 Mar 2020 15:23:01 GMT | got 5 objects from source file (offset: 0)
Fri, 13 Mar 2020 15:23:01 GMT | sent 5 objects to destination elasticsearch, wrote 5
Fri, 13 Mar 2020 15:23:01 GMT | got 0 objects from source file (offset: 5)
Fri, 13 Mar 2020 15:23:01 GMT | Total Writes: 5
Fri, 13 Mar 2020 15:23:01 GMT | dump complete
6 changes: 3 additions & 3 deletions invenio_stats/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ def register_events():
cluster. Thus Invenio-Stats provides a way to *compress* those events by
pre-aggregating them into meaningful statistics.
*Example: individual file downoalds events can be aggregated into the number of
*Example: individual file downloads events can be aggregated into the number of
file download per day and per file.*
Aggregations are registered in the same way as events, under the entrypoint
Expand Down Expand Up @@ -270,7 +270,7 @@ def register_aggregations():
]
An aggregator class must be specified. The dictionary ``params``
contains all the arguments given to its construtor. An Aggregator class is
contains all the arguments given to its constructor. An Aggregator class is
just required to have a ``run()`` method.
The default one is :py:class:`~invenio_stats.aggregations.StatAggregator`
Expand Down Expand Up @@ -300,7 +300,7 @@ def register_aggregations():
]
}
Again the registering function returns the configuraton for the query:
Again the registering function returns the configuration for the query:
.. code-block:: python
Expand Down

0 comments on commit 54a397f

Please sign in to comment.