Ability to store Reader / InputStream fields [LUCENE-1206]

[asfimport]

In some situations we would like to store the whole text, but the whole text won't always fit in memory so we can't create a String. Likewise for storing binary, it would sometimes be better if we didn't have to read into a byte[] up-front (even when it doesn't use much memory, it increases the number of copies made and adds burden to GC.)

FieldsWriter currently writes the length at the start of the chunks though, so I don't know whether it would be possible to seek back and write the length after writing the data.

It would also be useful to use this in conjunction with compression, both for Reader and InputStream types. And when retrieving the field, it should be possible to create a Reader without reading the entire String into memory up-front.

---

Migrated from LUCENE-1206 by Trejkaz, updated Mar 02 2013

[asfimport]

Jan Høydahl (@janhoy) (migrated from JIRA)

This issue has been inactive for more than 4 years. Please close if it's no longer relevant/needed, or bring it up to date if you intend to work on it. SPRING_CLEANING_2013

[asfimport]

Trejkaz (migrated from JIRA)

I think this would still be useful. The workaround of using a separate database to store larger text and binary stuff has never really sat with me terribly well.