Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

java.io.IOException: The current offset on block-info isn't update #511

Open
nannan111 opened this issue Jul 1, 2024 · 9 comments
Open

Comments

@nannan111
Copy link

nannan111 commented Jul 1, 2024

catch unknown error
java.io.IOException: The current offset on block-info isn't update correct, 48176155 != 53291524 on 7
at com.liulishuo.okdownload.core.file.MultiPointOutputStream.inspectComplete(MultiPointOutputStream.java:264)
at com.liulishuo.okdownload.core.interceptor.BreakpointInterceptor.interceptFetch(BreakpointInterceptor.java:123)
at com.liulishuo.okdownload.core.download.DownloadChain.processFetch(DownloadChain.java:241)
at com.liulishuo.okdownload.core.interceptor.RetryInterceptor.interceptFetch(RetryInterceptor.java:57)
at com.liulishuo.okdownload.core.download.DownloadChain.processFetch(DownloadChain.java:241)
at com.liulishuo.okdownload.core.download.DownloadChain.start(DownloadChain.java:211)
at com.liulishuo.okdownload.core.download.DownloadChain.run(DownloadChain.java:276)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487)
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at java.lang.Thread.run(Thread.java:1012)

下载的是你提供的500m那个zip,很容易出现这个异常,用的是你另一个工程https://github.com/tianshaokai/okdownload?tab=readme-ov-file,我设置了多线程下载,google手机

@yangfeng1994
Copy link

我下载的是102兆的,也非常容易出现这种情况。是必现的

@axel8888
Copy link

axel8888 commented Jul 24, 2024

按其他issue里的方法继承MultiPointOutputStream在close里加上noSyncLengthMap.remove(blockIndex);但是依旧会出现,用16进制编辑器查看出错的位置发现实际上下载到了东西,并不是00占位,所以直接在这个继承类里重写write把统计已下载长度计数用其他方式承载,比如

private long[] totalSizes = new long[breakpointCount];

public synchronized void write(int blockIndex, byte[] bytes, int length) throws IOException {
        ...
        totalSizes[blockIndex] += length;
        inspectAndPersist();
    }

判断长度也改成用这个

public void inspectComplete(int blockIndex) throws IOException {
      ...
            if (!Util.isCorrectFull(totalSizes[blockIndex], blockInfo.getContentLength())) {
                throw new IOException("The current offset on block-info isn't update correct, "
                        + blockInfo.getCurrentOffset() + " != " + blockInfo.getContentLength()
                        + " on " + blockIndex);
            }
}

我发现每次totalSize统计出来都是完整的,暂时没去细究为什么CurrentOffset会有问题,感觉像同步数据时候在多线程里搞丢了
write方法是synchronized的照理也不会出什么问题,有遇到的可以试试

@axel8888
Copy link

我觉得罪魁祸首应该是那个done不是同步方法。。继承下来改成同步就一切正常了

@nannan111
Copy link
Author

@axel8888 你测试了吗?可以具体说下修改那个方法吗

@axel8888
Copy link

测试了,就是其他帖子里继承ProcessFileStrategy和MultiPointOutputStream的方法,然后把done方法改成synchorized就行了

@nannan111
Copy link
Author

😅,你这消息太老了。okdownload 代码量最大的贡献者已经单独开工程了,把你上面说的那两个问题已经修改代码了

@axel8888
Copy link

axel8888 commented Aug 5, 2024

改完我这里都很正常从没再出现过那个报错

@nannan111
Copy link
Author

你用的那个帖子的修改方式,给贴个链接

@cuitzzh
Copy link

cuitzzh commented Jan 4, 2025

为了跟大家分享我这几天的成果,我特意去把我的github账户验证了
我用的是一楼所用的那个工程 [https://github.com/tianshaokai/okdownload]
上面所说的把done方法改成synchorized 这个工程里面已经改好了,所以不是这个原因
我的这边发现的原因是作者把LockSupport.park()这个方法用错了,这个方法的注释里面有第三种情况

  • {@code park} method may also return at any other time, for "no
  • reason", so in general must be invoked within a loop that rechecks
  • conditions upon return
    当调用done后 LockSupport.park() 作者认为他会等待,直到同步线程把数据处理好后再来unpark done结束,想法是好的,大多数情况是这样,但是
    LockSupport.park() 根本不能这样用,就像wait 一样,需要在条件循环里面调用这个接口,而不是单独使用,我发现park偶发根本不会等,会立刻返回,这时候同步线程根本没把数据同步,导致了此issue
    不知道你们和我是不是同一个问题,可以在MultiPointOutputStream 的ensureSync 方法最后一个parkThread() 醒来后
    打印一下这个值 noSyncLengthMap.get(blockIndex).get(); 这个值一定要是0 否则就会触发这个issue
    当然如果和我问题一样,解决起来就简单了 醒来发现不是0 就再次park

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants