ANFOG_DM harvester failing on 10aws

[bpasquer]

ANFOG_DM harvester is failing on 10aws.
The error is the same as for AODN_RAN_CTD #501

See the following log for full details IMOS_ANFOG_BCEOPSTUV_20170109T100117Z_SG540_FV01_timeseries_END-20170131T204838Z.nc.EbYY92.20170310-112353.log

Going to process a total of '1' files
Processing slice with '1' files
Preparing file '/mnt/ebs/tmp/IMOS_ANFOG_BCEOPSTUV_20160228T225753Z_SL239_FV01_timeseries_END-20160330T073251Z.nc.xpGyCy,IMOS/ANFOG/slocum_glider/Portland20160229/IMOS_ANFOG_BCEOPSTUV_20160228T225753Z_SL239_FV01_t
imeseries_END-20160330T073251Z.nc' in temp dir '/tmp/d20170307-27783-1tjbpem'
ADD '/mnt/ebs/tmp/IMOS_ANFOG_BCEOPSTUV_20160228T225753Z_SL239_FV01_timeseries_END-20160330T073251Z.nc.xpGyCy' => 'IMOS/ANFOG/slocum_glider/Portland20160229/IMOS_ANFOG_BCEOPSTUV_20160228T225753Z_SL239_FV01_timeser
ies_END-20160330T073251Z.nc'
'anfog_dm-anfog_dm': ["IMOS/ANFOG/slocum_glider/Portland20160229/IMOS_ANFOG_BCEOPSTUV_20160228T225753Z_SL239_FV01_timeseries_END-20160330T073251Z.nc"]
Executing '/mnt/ebs/talend/jobs/anfog_dm-anfog_dm/java/ANFOG_DM_harvester/ANFOG_DM_harvester_run.sh --context_param paramFile="/mnt/ebs/talend/jobs/anfog_dm-anfog_dm/etc/anfog_dm-anfog_dm.conf" --context_param ba
se=/tmp/d20170307-27783-1tjbpem --context_param fileList=/tmp/harvester_file_list20170307-27783-s6auuw --context_param logDir=/tmp/d20170307-27783-phi06f' for '["IMOS/ANFOG/slocum_glider/Portland20160229/IMOS_ANF
OG_BCEOPSTUV_20160228T225753Z_SL239_FV01_timeseries_END-20160330T073251Z.nc"]'
<...>
SLF4J: The requested version 1.5.6 by your slf4j binding is not compatible with [1.6, 1.7]
SLF4J: See http://www.slf4j.org/codes.html#version_mismatch for further details.
<...>
log4j:WARN No appenders could be found for logger (it.geosolutions.geonetwork.op.GNMetadataGet).
log4j:WARN Please initialize the log4j system properly.
Exception in thread "main" java.lang.Error: java.lang.Error: java.lang.Error: java.lang.Error: java.lang.AbstractMethodError: org.slf4j.impl.JDK14LoggerAdapter.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V

[jonescc]

Looks like two different versions of the netcdf java library are being included in the harvester build causing incompatible versions of classes to be attempted to be used. Talend sources these jars from

/home/jenkins/talend/talend-components/iNetCDFInput/netcdfAll-4.6.4.jar
/home/jenkins/talend/lib/java/netcdfAll-4.2.jar

The first location is the jar specified in the talend components and is correct, the second jar should not be included. The second location is an include all these jars when building the zip file and should not include this file. Thought I removed this from the source TOS install file for jenkins a long time ago and removed it from jenkins master and slave instances but it looks like its come back somehow.

@anguss00 - any ideas on how this got here?

I'll remove them from the jenkins instances and the talend jobs will need to be rebuilt, but if we installed this from something that includes this incorrect version of the netcdf file we probably need to remove it there as well.

[jonescc]

Looks like the old netcdf jar was included in some Talend Spatial components I wrote a long time ago (sNetCDFInput, sOPeNDAPInput and iThreddsIterator) - they may have been picked up from there. I've removed these components from the installation zip file on emiisheryl (//researchdata.its.utas.edu.au/research/IMOS/emiiSheryl/Talend%20Software/TOS-Spatial-5.4.1.zip), aws s3 (s3://imos-binary/static/talend/TOS-Spatial-5.4.1.zip) and from 13-aws-syd.

There are still some references in old talend harvesters which I'll remove in a separate pull request.

@bpasquer , ANFOG_DM has been rebuilt in jenkins and promoted to rc. Try it out there and if it runs OK we can promote to prod.

[bpasquer]

@jonescc Unfortunately the issue is not resolved. Tested today. The error listed in the following log is same as before:
/mnt/ebs/log/data-services/ANFOG_DM/IMOS_ANFOG_BCEOPSTUV_20160127T231611Z_SL239_FV01_timeseries_END-20160226T031401Z.nc.Gf5RrI.20170328-143235.log

[jonescc]

anfog_dm changes haven't been promoted to prod yet. Was testing in RC OK?

[bpasquer]

Yes, sorry i thought i had commented about the test i did on RC last week. It's working fine.

[jonescc]

OK, I've promoted it to prod. Should be provisioned after 5 sometime I think.

[bpasquer]

The ANFOG_DM harvester is now processing files as before. Thanks @jonescc