2.16.1

This is a relatively small release which includes bug fixes and some new features. Users should upgrade from 2.16.0.

New Feature: Write VCF to Paths

You can now write vcfs to NIO Paths including gcs Paths.
4902ee9 VCFWriter accepts Path (#1134)

Compatibility:

This release is binary compatible with 2.16.0. There are some new overloads of methods which may require minor source changes where they were previously being called with untyped nulls.

Deprecations

Some poorly named methods are now deprecated.
e0bf651 locatable all the things (#1159)
c4dbf4a fixing spelling Indeces -> Indices (#1157)

Other updates and bugfixes:

fb36a36 adding back deleted method(#1178)
e37d8de extending use of IOUtil.toPath() (#967)
85580fc restore no-args VariantContextWriterBuilder.build() (#1177)
16704e6 Use quality header and SAMTag.CO in FastqEncoder (#912)
e637281 Expose VCFCodec version and header fields. (#1176)
57442bb Fix a bug where invalid bytes in BufferedInputStream's buffer were being accessed. (#1175)
5168c79 Fix CRAM container offset calculation (#1167)
e6d5c29 fix an copy-paste error in VCFHeader (#1160)
8767e64 adding position to SAMRecord.toString() (#1158)
c4912e9 update to gradle 4.8 (#1154)
c380053 Fixes for CachingBAMFileIndex (#1156, #1127)
38a24d5 Additional unit tests for IOUtil (#1149)

2.16.0

This release includes significant new features and bug fixes. It is not technically backwards compatible with 2.15 releases, but most downstream users should not encounter major difficulties when updating.

New Features:

Block compressed indexed FASTA files:

Htsjdk can now read block compressed (.bgz / .gz) fasta files if they have a .gzi and a .fai index. No more need to unzip the reference before reading it!
Load reference files with ReferenceSequenceFileFactory.getReferenceSequenceFile() and it will automatically discover the correct indexes.

A gzi index can be generated with GZIIndex.createIndex() and related methods. Alternatively it can be created with the bgzip program. Note that gzip and bgzip are different formats and gzipped files cannot be index with a gzi. The fai index is used alongside the .gzi for random access to fasta and is unchanged for zipped vs unzipped files.

People directly instantiating the concrete ReferenceSequenceFile classes may want to move to using the factory in order to get access to this feature.
b2bfb32 Support for block-compressed indexed FASTA references (#1014)
c19f9f7 removing new finals in AbstractIndexedFastaSequenceFile (#1135)
#996, #864

SamLocusAndBaseIterator:

New iterator which traverses a bam file and reference together and returns pileup information along with the associated reference bases.
336fa5c Yf sam locus and reference base iterator (#1137)

Expanded support for non File inputs

Added new methods to allow using Path in more places.
ccc9259 Fix reading a CRAM with an index from a non-File Path. (#1125)
3a20218 Add SAMFileWriterFactory methods that take referenceFile as a Path(#1005)
96ce8fb deprecating overloads makeCRAMWriter with mixed Path/File (#1150)

Added additional methods to read/write directly from Streams in cases where a Path or File is not available.
72818a0 Allow ReferenceSequenceFileFactory to load from streams (#1123)
34dbda7 Make BAMFileWriter public to expose code to write a BAM header to a stream. (#1119)
71c5a69 Add method to load tribble index from a stream. (#1145)

Restore SRA support

Support for reading SRA was broken because of changes in the SRA API. A new version of their Java API adapter has been published and is now integrated into htsjdk. Support should be restored.
0aff433 restoring SRA support by updating the backing library (#1142)

Methods moved to IOUtils

Several methods for identifying if a file is block compressed moved to IOUtils. The original methods have been deprecated.
85d2b8d Add utilities for checking block-compressed files in IOUtils #1130 (#1132)
55debe2 adding test for IOUtil.isBlockCompressed (#1141)

Other improvements and bugfixes

9ea0a03 new method CRAMFileReader.createIndexIterator (#1120)
8b55de6 Add new class VCFHeaderReader #1122 (#1148)
52ec082 making public static fields in SamReader.Type final (#1146)
5eb8ee3 Make SamReader.Type methods public (#1144)
efefdd0 update FastqRecord.toString javadoc to warn users of future behavior change (#952)
0d8e1c6 Fixed bug where GL field overrides PL field #1097 (#1098)
497a0f3 made VariantContextWriterBuilder.determineOutputTypeFromFile public (#1066)
f007ed3 Add an optimized version of CramContainerIterator (#1129)
d3d7a6e SAMFileWriterImpl extensibility improvements (#913)
442029c seeking within SeekableBufferedStream's buffer reuses the existing buffer (#1121)

2.15.1

This is a special narrowly focused bug fix release to fix an issue introduced in 2.14.2. All projects using 2.14.2+ should upgrade.

Before 2.14.2 you could open a pipe using the Path API in some cases. There was a regression in 2.14.2 which broke this. This is now fixed in 2.15.1. There was a related issue around opening pipes using the File api, which was fixed in 2.14.3.

This release does not include all commits since 2.15.0, it only includes the fix for opening pipes using the Path API.

See #1084, #1083 for more information.

2.15.0

This release includes new features and bug fixes. There are no known compatibility issues with previous releases. Users are recommended to upgrade, particularly if they require support for very long reads.

Support for long CIGARs:

htsjdk can now read and write bam files with long CIGAR strings. This allows very long reads that have complicated alignments to be represented in bam. Previously the limit was 65535 CIGAR elements per read which is insufficient for new long read sequencing technology. See for more details about the internal representation samtools/hts-specs#227. (#1086)

Improvements to SortingCollection

SortingCollection now uses java parallel for some speedup and does a better job estimating available memory availability. #1111, #1109

Other updates:
adf428f Make BlockCompressedFilePointerUtil.makeFilePointer public (#1116)
1bbd3cf Fix typos and add defaults to javadoc in Defaults (#1114)
8b63854 renaming IoUtilTest and minor refactoring (#990)
1aaba3d reduce garbage creation when writing VCF (#1110)
73bf89c Add support for long reference sequences (#980)
447f82e fix compilation error caused by typo (#1106)
e266acc improve documentation in SAMRecord (#1036)
a5be9f2 upgrade jacoco version to 0.8.0 (#1102)
4245d9d FTP tests moved to separate test target (#1104)
673eaf7 Adding check on validator on VCFHeadLineCount (#1093)
ae57c10 prevent SamLocusIterator.accumulateIndels printing unecessary newlines (#1095)
7a38277 throw a clear exception if writing vcf without a header set #1008 (#1090)
d482e78 aesthetic changes only (#1087)

2.14.3

This is a bugfix release to fix #1083 that was broken in 2.14.3.

2.14.2

This release includes several important bug fixes. This release is backwards compatible with 2.14.1.

Known Regression:

This release introduced a regression (#1083) that prevents opening named pipes with SamReaderFactory.
Prefer 2.14.3 over this version.

Important fixes:

A longstanding issue that could cause reading block compressed vc.gz files to be truncated is fixed.
The issue was caused by a known bug in the java GZipInputStream implementation that make inappropriate use of the InputStream.available() method to detect end of stream.
The work-around was to implement available() on SeekableStream
See https://bugs.java.com/bugdatabase/view_bug.do?bug_id=7036144# for more information (and maybe prod the java team to fix this one...)

Fixed shutdown hooks that automatically delete Path to use a single shutdown hook instead of registering one for each path.

All changes:

5c8de55 Add DeleteOnExitPathHook to register java.nio.Path for deletion (#1037)
8b8d961 Implement SeekableStream.available #898 (#1077)
9d724c3 Performing CIGAR and sequence length only when both the read sequence and the CIGAR exists. (#1075)
88edbf3 Added filters to output of VariantContext.toString() #833 (#1074)
6e51261 move FastaSequenceIndexEntry to a separate file and make it public #1072

2.14.1

This is a minor release which includes a few new methods and bug fixes.

Upgrading from 2.14.0 is recommended and there should be no compatibility issues.

e051a56 make SortingCollection.spillToDisk() public(#1061)
c791472 Add isQueryable to FeatureReader interface and expose it in VcfFileReader (#1041)
5029bde Add mark support for SeekableStream (#1016)
ad0f4c1 Make the TestSuite be more deterministic (#1059)
a9f0181 toString method added to LineReader / LineIteratorImpl (#1057)
b65ac83 Prevent NPE in BlockCompressedInputStream exception messages(#1067)
369dd40 improve error message in AbstractFastaSequenceFile.java (#1060)
3f183fe Use SeekableStreamFactory in TabixFeatureReader #1042 (#1064)

2.14.0

This small release includes bug fixes and additional support for reading VCF's from java.nio.Path using VcfFileReader.

It was already possible to read VCFs from a Path with a more convoluted method, but VcfFileReader is a convenient way to do so.

BREAKING CHANGE:

• new abstract method SamReaderFactory.getFileHeader(Path samFile)
  This should have no impact for most users unless you're providing custom implementations of SamReaderFactory. If you are, your subclasses will have to implement this new method.

Deprecations:

• SAMSequenceDictionaryExtractor.extractDictionary(File) has been deprecated, use SAMSequenceDictionaryExtractor.extractDictionary(Path) instead

80122b9 Enable VcfFileReader and other classes to use Path (#1018)
335cd06 Modify FilterTag to Pass or Filter reads (#1052)
7ee6d55 Add checks on Histogram::getPercentile to protect against negative and zero values(#1043)
3308baf Strip query parameters from url before testing for block compressed extension #1021 (#1033)

2.13.2

This is a minor update that includes several bug fixes

Anyone using the EBI reference service should upgrade because the hardcoded link changed.
There should be no compatibility issues with previous 2.13 releases.

f45c654 Fix MergingIterator.close() to avoid ConcurrentModificationException #1055 (#1056)
8296466 Fixed BCF Decoding and Writing to exclude marker bit on first GT field value (#1048)
46ac4e2 replaced an instance of '>=' with 'more than' to allow use of javadoc '@value' pragma. (#1053)
3c6dd65 update EBI_REFERENCE_SERVICE_URL_MASK to use https. (#1051)
e1978ef new ReferenceSequenceFileWalker constructor for Paths (#1050)
8794b1c removing un-used crud (#1039)

2.13.1

Fixes problems with SortingCollections introduced in 2.13.0
All users should prefer this over 2.13.0

eb364a4 fixing crash on shutdown hook #1029 (#1032)
e2a03d6 revert change to tempdir in SortingCollection introduced in #910 #1030 (#1031)
d1fd8d6 Add method to log the last record recorded. (#1027)
0ca4b47 Fix performance bug in AbstractLocusIterator for gapped reads #838 (#1020)