ISMIR-2017-Discogs: Dataset, code for analysis and results

This repository contains:

Pre-processed dataset of release metadata from the Discogs database
Useful code for analysis of release metadata using this dataset
Additional materials for the paper "Quantifying music trends and facts using editorial metadata from the Discogs database" (Bogdanov & Serra, ISMIR-2017).

Please, cite this paper if you are using our dataset and code.

See examples of metadata analysis that can be done using metadata from Discogs.

Pre-processed dataset of release metadata from Discogs

Pre-processed release metadata dump in hdf format (pandas DataFrame).
Pre-processed release metadata dump in json format (every line in the file is a json dictionary).
Original data dumps in XML format

Code for data pre-processing and analysis

This is the code that we used to create our release dataset and for our example studies presented in the ISMIR-2017's paper.

Dependencies

Run pip install -r requirements.txt to install required dependencies.

Configuration

config.py: basic configuration script, contains some global variables (like filenames) used by other scripts

Dataset creation and analysis

preprocess_releases_xml_to_json.py: downloads the original XML dump archive and converts a subset of its metadata fields to a json dump.
preprocess_releases_json_to_hdf_pandas.py: further simplifies the metadata removing and recoding some fields, and outputs a HDF file with a pandas DataFrame.
analyze.py: a collection of useful functions for analysis of the dataset.