bleve seems to keep deleted files open

[elektito]

I had this weird situation on my server where df reported 29G of disk space in use, but du only found 12G. Doing an lsof +L1 reported lots of deleted zap files kept open by my program (which uses bleve). Sure enough, when I restarted the service, the reports of df and du matched.

Is there any reason for those files being kept open, or is that a bug? Can I do something about it through configuration?

[abhinavdangeti]

@elektito This could likely do with the merger's policy not being strict enough for your use-case. As can be seen here, this is the default merger policy ..

// DefaultMergePlanOptions suggests the default options.
var DefaultMergePlanOptions = MergePlanOptions{
	MaxSegmentsPerTier:   10,
	MaxSegmentSize:       5000000,
	TierGrowth:           10.0,
	SegmentsPerMergeTask: 10,
	FloorSegmentSize:     2000,
	ReclaimDeletesWeight: 2.0,
}

You could try defining a more aggressive merger policy to achieve faster deletion of overwritten index content by passing scorchMergePlanOptions within kvConfig while creating/opening a bleve index. Here's an example ..

map[string]interface{}{
    "scorchMergePlanOptions" : {
        "maxSegmentsPerTier": 5,
        "maxSegmentSize": 2000000,
        "tierGrowth": 10.0,
        "segmentsPerMergeTask": 10,
        "floorSegmentSize": 2000,
        "reclaimDeletesWeight": 2.0
    }  
}