MIME type of netCDF (.nc) files with direct upload

[stevenferey]

entrepot.recherche.data.gouv.fr

What steps does it take to reproduce the issue?

Have direct upload enabled
Have a netCDF file with a .nc file extension

• When does this issue occur?

A user downloads a netCDF (.nc) file with direct upload active

• Which page(s) does it occurs on?

the dataset page

• What happens?

Having this configuration in the MimeTypeDetectionByFileExtension.properties file: nc=application/netcdf
I expect to get this MimeType in BDD when adding a .nc file to my dataverse.
This is not the case :)

I get the MimeType "application/x-netcdf" and I can't see the previewer for this file because it is configured for the MimeType "application/netcdf"

Note that when I apply a redetect with the API, the result is OK:
{"status":"OK","data":{"dryRun":true,"oldContentType":"application/x-netcdf","newContentType":"application/netcdf"}}

• To whom does it occur (all users, curators, superusers)?

All the users

• What did you expect to happen?

1. For Java and Apache, the MimeType for .nc and .cdf files appears to be "application/x-netcdf".
   Perhaps we need to adapt MimeTypeDetectionByFileExtension.properties and the previewers in order to maintain consistency?

2. Add "application/x-netcdf" to previewers to enable it for this MimeType

Which version of Dataverse are you using?

5.14

Any related open or closed issues to this bug report?

[pdurbin]

@stevenferey just to restate what you're saying to make sure I understand...

• When using direct upload the MIME type becomes "application/x-netcdf".
• When not using direct upload the MIME type becomes "application/netcdf".

Is that right? Have you tested non-direct upload? There are caveats associated with direct upload: https://guides.dataverse.org/en/5.14/developers/big-data-support.html#features-that-are-disabled-if-s3-direct-upload-is-enabled

Maybe we could add more tests as part of this "test S3" issue:

• Add S3 tests to the regular integration test suite #6783

In FileUtilTest we have the following...

@Test
public void testNetcdfFile() throws IOException {
    // We got madis-raob.nc from https://www.unidata.ucar.edu/software/netcdf/examples/files.html
    // and named it "madis-raob" with no file extension for this test.
    String path = "src/test/resources/netcdf/";
    String pathAndFile = path + "madis-raob";
    File file = new File(pathAndFile);
    String contentType = FileUtil.determineFileType(file, pathAndFile);
    assertEquals("application/netcdf", contentType);
}

... which asserts that the MIME type should be "application/netcdf" when the determineFileType method is used. The possible MIME types it can assign are these:

case "NetCDF":
    return "application/netcdf";
case "NetCDF-4":
    return "application/netcdf";
case "HDF5":
    return "application/x-hdf5";

Yes, as a workaround, you could add the previewer twice for both MIME types. This is already the strategy for the image previewer (for PNG, GIF, JPG, etc.).

[stevenferey]

Hello Philip,

Yes that's fine, the observed behavior is :

• When using direct upload the MIME type becomes "application/x-netcdf".

• When not using direct upload the MIME type becomes "application/netcdf".

Yes, we will use the recommended workaround.

Thank you
Steven.