CKAN 2.9 solr9: SolrCore not found on hub release (Dec 13, 2024 at 7:38 pm)

[duttonw]

related: #13

Moving back from 2.9-solr8 to 2.9-solr9 image, It now fails as there is no ckan SolrCore.

Digest: sha256:2bd00276373721c856c70a527257ef6fb0a155a1708a962917c6facaf5462b38
  Status: Downloaded newer image for ckan/ckan-solr:2.9-solr9
  docker.io/ckan/ckan-solr:2.9-solr9
...
2024-12-16 04:08:02,309 ERROR [ckan.lib.search.common] Solr responded with an error (HTTP 500): [Reason: SolrCore 'ckan' is not available due to init failure: Could not load conf for core ckan: Error loading parsing schema]
Traceback (most recent call last):
  File "/srv/app/src/ckan/ckan/lib/search/common.py", line 61, in is_available
    conn.search(q="*:*", rows=1)
  File "/usr/local/lib/python3.9/site-packages/pysolr.py", line 720, in search
    response = self._select(params, handler=search_handler)
  File "/usr/local/lib/python3.9/site-packages/pysolr.py", line 418, in _select
    return self._send_request('get', path)
  File "/usr/local/lib/python3.9/site-packages/pysolr.py", line 393, in _send_request
    raise SolrError(error_message % (resp.status_code, solr_message))
pysolr.SolrError: Solr responded with an error (HTTP 500): [Reason: SolrCore 'ckan' is not available due to init failure: Could not load conf for core ckan: Error loading parsing schema]
2024-12-16 04:08:02,312 WARNI [ckan.lib.search] Problems were found while connecting to the SOLR server
2024-12-16 04:08:02,3[13](https://github.com/ckan/ckanext-validation/actions/runs/12345721388/job/34450245494#step:7:14) INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
2024-12-16 04:08:02,801 CRITI [ckanext.validation.plugin] 
The validation extension requires a database setup.
Validation pages will not be enabled.
Please run the following to create the database tables:
    ckan validation init-db
2024-12-[16](https://github.com/ckan/ckanext-validation/actions/runs/12345721388/job/34450245494#step:7:17) 04:08:02,927 ERROR [ckan.lib.search.common] Solr responded with an error (HTTP 500): [Reason: SolrCore 'ckan' is not available due to init failure: Could not load conf for core ckan: Error loading parsing schema]
Traceback (most recent call last):
  File "/srv/app/src/ckan/ckan/lib/search/common.py", line 61, in is_available
    conn.search(q="*:*", rows=1)
  File "/usr/local/lib/python3.9/site-packages/pysolr.py", line 720, in search
    response = self._select(params, handler=search_handler)
  File "/usr/local/lib/python3.9/site-packages/pysolr.py", line 4[18](https://github.com/ckan/ckanext-validation/actions/runs/12345721388/job/34450245494#step:7:19), in _select
    return self._send_request('get', path)
  File "/usr/local/lib/python3.9/site-packages/pysolr.py", line 393, in _send_request
    raise SolrError(error_message % (resp.status_code, solr_message))
pysolr.SolrError: Solr responded with an error (HTTP 500): [Reason: SolrCore 'ckan' is not available due to init failure: Could not load conf for core ckan: Error loading parsing schema]
[20](https://github.com/ckan/ckanext-validation/actions/runs/12345721388/job/34450245494#step:7:21)24-12-16 04:08:02,928 WARNI [ckan.lib.search] Problems were found while connecting to the SOLR server

[amercader]

I now finally managed to find out what is going on here.

• The schema included in CKAN 2.9 (this one in dev-v2.9) is not compatible with Solr 9 because it has <defaultSearchField>text</defaultSearchField> in it. That's the error that we see if we check the docker container logs:

  [...]
  Caused by: org.apache.solr.common.SolrException: Setting defaultSearchField in schema not supported since Solr 7
  	at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:571) ~[?:?]
  	at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:193) ~[?:?]
  	at org.apache.solr.schema.ManagedIndexSchema.<init>(ManagedIndexSchema.java:113) ~[?:?]
  	at org.apache.solr.schema.ManagedIndexSchemaFactory.create(ManagedIndexSchemaFactory.java:288) ~[?:?]
  	at org.apache.solr.schema.ManagedIndexSchemaFactory.create(ManagedIndexSchemaFactory.java:54) ~[?:?]
  [...]
  
  

• This Solr schema is compatible with Solr 8 (no deprecation warnings for defaultSearchField being here)

• How did the ckan/ckan-solr:2.9-solr9 images ever worked in the first place? As @themowski correctly identified in CKAN 2.9 images embed CKAN 2.10 schema #9 there was a bug in the images that meant that regardless of the CKAN version the Solr schema included always came from the dev-v2.10 branch. That branch included the changes needed to support Solr 8/9 (Switch to Solr v8.0 (with the minimal code changes and no new features) ckan#6530)

• When we fixed the bug in the images and the 2.9 images started including the actual 2.9 schema file Solr 9 stopped working.

Where do we go from here?

Although CKAN 2.9 is no longer supported and we won't be supporting the 2.9 Solr images going forward, we broke something that was working (even if by accident) and is widely used in tests so I'm keen to get the ckan/ckan-solr:2.9-solr9 images working again.

Note

For extension maintainers, a workaround for now is to use ckan/ckan-solr:2.9-solr8 image for the Solr service in your workflows

We could include the updated schema.xml file present in dev-v2.10 in the 2.9 images, but I think it will be safer to use it alongside changes in CKAN itself specially in the query code. Perhaps backport these in the long term support 2.9 fork that Derilinx maintains.
I'll raise this tomorrow in the dev meeting.

[duttonw]

Nice sleuthing @amercader.

I don't mind if cicd is stuck on solr8, just need a reference point in cicd for upgrading and ensuring i'm still backwards compatible for a little while longer.

Unsure how hard it would be to re-instate the previous tagged container which no one complained about for years and leave it 'deprecated' / as is going forward.