Add a new Solr search facet

Faceting is the arrangement of search results into categories based on indexed terms.

Searchers are presented with the indexed terms, along with numerical counts of how many matching documents were found for each term. Faceting makes it easy for users to explore search results, narrowing in on exactly the results they are looking for.

In this example, we add a new boolean processed field, that will allow users to filter processed / non-processed dataset (see PR #2146)

Solr configuration file

Add a new <field> element into docker-compose/solr/core/schema.xml (starting from line 441)

...
<field name="processed" type="boolean" indexed="true" stored="true" required="true" />
...

📖 Check Solr documentation for properties description

Datasets micro-service

Models

ShanoirMetadata.java

Add the new property into the ShanoirMetadata class

• new property
• accessors
• constructor parameter
• new @ColumnResult annotation

@Entity
@SqlResultSetMapping(name = "SolrResult", classes = {@ConstructorResult(targetClass = ShanoirMetadata.class,
		columns = {...,
			@ColumnResult(name="processed", type = Boolean.class)
		})
})

...

	private boolean processed;

	...

	public ShanoirMetadata (..., boolean processed) {
		...
		this.processed = processed;
	}

	...

	public boolean isProcessed() {
		return processed;
	}

	public void setProcessed(boolean processed) {
		this.processed = processed;
	}
...
}

ShanoirSolrDocument.java

Add the new property into the ShanoirSolrDocument class

• new @Field property
• accessors
• constructor parameter

...
@Field
private boolean processed;

...

public ShanoirSolrDocument (..., boolean processed) {
	...
	this.processed = processed;
}

...

public boolean isProcessed() {
	return processed;
}

public void setProcessed(boolean processed) {
	this.processed = processed;
}
...

ShanoirSolrQuery.java

Add the new Collection property into the ShanoirSolrQuery class

• new property
• accessors

...
private Collection<Boolean> processed;
...
public Collection<Boolean> getProcessed() {
	return processed;
}

public void setProcessed(Collection<Boolean> processed) {
	this.processed = processed;
}

Repository

ShanoirMetadataRepositoryImpl.java

Add the new property into the SQL SELECT clause of the String constants of the ShanoirMetadataRepositoryImpl class

• MR_QUERY
• PET_QUERY
• CT_QUERY
• GENERIC_QUERY
• EEG_QUERY
• BIDS_QUERY
• PROCESSED_QUERY
• MEASUREMENT_QUERY
• XA_QUERY

...
public static final String MR_QUERY = "SELECT d.id as datasetId, " +
	...
	"0 as processed"
	+ " FROM dataset d"
	+ " LEFT JOIN dataset_acquisition da on da.id = d.dataset_acquisition_id"
	...
	+ " WHERE d.updated_metadata_id = dm.id AND md.id = d.id";
...

Service

SolrServiceImpl.java

• Add the new property as parameter of the ShanoirSolrDocument class constructor in the getShanoirSolrDocument() method of the SolrServiceImpl class

...
private ShanoirSolrDocument getShanoirSolrDocument(ShanoirMetadata shanoirMetadata) {
		return new ShanoirSolrDocument(..., shanoirMetadata.isProcessed());
	}
...

• Add a new OR condition on the new property to the prepareTextFields() method of the SolrServiceImpl class

private Pageable prepareTextFields(Pageable pageable) {
	for (Sort.Order order : pageable.getSort()) {
		if (...
				|| order.getProperty().equals("processed")
		) {
...

SolrJWrapperImpl.java

• Add a new [property]_FACET constant to SolrJWrapperImpl class

...
private static final String PROCESSED_FACET = "processed";
...

• Add the new constant to the DOCUMENT_FACET_LIST and TEXTUAL_FACET_LIST String arrays in the SolrJWrapperImpl class

...
private static final String[] DOCUMENT_FACET_LIST = {
			...
			PROCESSED_FACET
	};

	private static final String[] TEXTUAL_FACET_LIST = {
			...
			PROCESSED_FACET
	};
...

• Add a call to the relevant addFilterQuery*(...) method in the addUserFiltering() method in the SolrJWrapperImpl class, based on the new property type

...
private void addUserFiltering(SolrQuery query, ShanoirSolrQuery shanoirQuery) {
	/* add user's filtering */
	...
	addFilterQueryFromBoolean(query, PROCESSED_FACET, shanoirQuery.getProcessed());
	...
}
...

Database

Create a new SQL script into docker-compose/database/db-changes/datasets to add the new field into datasets.shanoir_metadata table

alter table shanoir_metadata add column processed bit(1);

Frontend

solr.document.model.ts

Add the new property into SolrDocument and SolrQuery classes

export class SolrDocument {
    ...
    processed: boolean;
    ...
}

export class SolrRequest {
    ...
    processed: boolean[];
    ...
}
...

solr.search.component.html

Add a new solr-paging-criterion element to the Solr view

...
        <solr-paging-criterion
            label="Processed Dataset"
            awesome="fa-solid fa-gears"
            facetName="processed"
            [getPage]="getFacetFieldPage.bind(this)"
            [(ngModel)]="solrRequest.processed"
            [ngModelOptions]="{standalone: true}"
            (onChange)="updateSelections(); refreshTable('processed')">
       </solr-paging-criterion>
...

solr.search.component.ts

• Add the new property to the TextualFacetNames (or RangeFacetNames ) array

const TextualFacetNames: string[] = [..., 'processed'];
const RangeFacetNames: string[] = ['sliceThickness', 'pixelBandwidth', 'magneticFieldStrength'];

• If you want to display the new property into the Solr search result table, add a new ColumnDefinition into the columnDefs array

let columnDefs: ColumnDefinition[] = [...
            {headerName: "", type: "boolean", cellRenderer: row => row.data.processed, awesome: "fa-solid fa-gears", color: "dimgrey", disableSorting: true, tip: item => { return item.processed ? "processed dataset" : "" }},...];

solr.text-search-mode.component.html

Add the new field to the expert info block

...
        <p>The present field names are :
           <code>centerName, datasetCreationDate, studyName, subjectName, subjectType, acquisitionEquipmentName, datasetId, datasetName, datasetNature, datasetType, processed, examinationComment, examinationDate, tags, magneticFieldStrength, pixelBandwidth, sliceThickness, studyId, </code>
....

Docker volumes

For the change to be taken into account, Solr docker volume need to be recreated.

Local instance

docker exec solr rm -rf /var/solr/data

Then, as administrator, launch a manual indexation through the Shanoir webpage (Administration / Index to Solr)

Or

⚠️ This operation will reset your Shanoir local environment

On your local environment, you can recreate said volume either by executing

./bootstrap.sh --clean

Qualif instances

On qualif server, after having deployed your branch, you need to remove the volume with

dk exec ofsep-qualif-solr rm -rf /var/solr/data

Then restart the Solr docker (and its dependences)

dk stop ofsep-qualif-nginx ofsep-qualif-nifti-conversion ofsep-qualif-datasets ofsep-qualif-solr
...
dk start ofsep-qualif-nifti-conversion ofsep-qualif-datasets ofsep-qualif-solr ofsep-qualif-nginx
...

Then, as administrator, launch a manual indexation through the Shanoir webpage (Administration / Index to Solr)