grimoirelab-metrics

Client to generate GrimoireLab metrics for Project Health using the software analytics platform GrimoireLab.

Installation

Prerequisites

Instances of GrimoireLab 2.x and OpenSearch must be running before launching this tool. Please check the GrimoireLab documentation in order to deploy the platform.

To get this tool running, we also recommend using poetry. This package manager will install the tool and all its dependencies. You can install poetry by following this guide.

Once you have poetry running, move to the next section.

Steps

1. Clone the repository:

   git clone [email protected]:Bitergia/grimoirelab-metrics.git
   cd grimoirelab_metrics

2. Install dependencies and tool:

   poetry update
   poetry install

   This will install the tool inside of a virtual environment managed by poetry. To use the tool you will have to activate it first with the command eval $(poetry env activate) (poetry >= 2.x) or poetry shell (poetry < 2.x).

   For development mode, install the tool with: poetry install --with dev

Usage

Given a SPDX SBOM file with git repositories as input, this tool will generate a set of Project Health metrics. These metrics are calculated using the data stored on GrimoireLab about those repositories. If any of the repositories is not available on GrimoireLab, the tool will add it to GrimoireLab to have it analyzed.

grimoirelab-metrics spdx.xml \
  --grimoirelab-url http://localhost:8000 \
  --grimoirelab-user user --grimoirelab-password password \
  --opensearch-url https://admin:[email protected]:9200 \
  --opensearch-index events \
  --from-date 2024-01-01 --to-date 2025-01-01 \
  --repository-timeout 3600
  --code-file-pattern "\.py$|\.js$" \
  --binary-file-pattern "\.exe$|\.tar$" \
  --output metrics.json

The parameters needed to run the tool are:

• A valid SPDX file
• GrimoireLab instance address
• OpenSearch instance address
• OpenSearch index name, where GrimoireLab events data are stored
• Output filename, where metrics will be written.

This is an example of a valid SPDX file:

<?xml version="1.0" encoding="utf-8"?>
<Document>
    <SPDXID>SPDXRef-DOCUMENT</SPDXID>
    <creationInfo>
        <created>2025-02-07T00:00:01Z</created>
        <creators>Organization: Bitergia</creators>
    </creationInfo>
    <dataLicense>CC0-1.0</dataLicense>
    <name>GrimoireLab</name>
    <spdxVersion>SPDX-2.3</spdxVersion>
    <documentNamespace>mynamespace</documentNamespace>
    <packages>
        <SPDXID>SPDXRef-bootstrap-gnu-config.bst-0</SPDXID>
        <comment>Product: gnu-config</comment>
        <downloadLocation>https://github.com/chaoss/grimoirelab-perceval.git</downloadLocation>
        <filesAnalyzed>false</filesAnalyzed>
        <name>bootstrap/gnu-config.bst</name>
        <sourceInfo>git</sourceInfo>
    </packages>
    <packages>
        <SPDXID>SPDXRef-bootstrap-gnu-config.bst-0</SPDXID>
        <comment>Product: gnu-config</comment>
        <downloadLocation>https://github.com/chaoss/grimoirelab-core.git</downloadLocation>
        <filesAnalyzed>false</filesAnalyzed>
        <name>bootstrap/gnu-config.bst</name>
        <sourceInfo>git</sourceInfo>
    </packages>
</Document>

Project Health Metrics

This is the list of the metrics generated by this tool:

• Number of commits per repository
• Number of developers per repository
• Number of developers producing up to 50% of the total contributions
• Number of companies producing up to 50% of the total number of code contributions
• File type metrics (code, binaries or other)
• Commit side metrics (added lines and removed lines)
• Message size metrics (total, mean and median)
• Frequency metrics for commits (week, month and year)
• Developer categories (core, regular and casual)