Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] System.OutOfMemoryException - LiteDatabase in memory #2541

Open
ednsinf opened this issue Sep 9, 2024 · 5 comments
Open

[BUG] System.OutOfMemoryException - LiteDatabase in memory #2541

ednsinf opened this issue Sep 9, 2024 · 5 comments
Labels

Comments

@ednsinf
Copy link

ednsinf commented Sep 9, 2024

Bug not resolved. When I use LiteDatabase in memory and use bulkinsert and insert in 1k to 1k rows (using transaction our not) and shows memory erros always. Please verify thanks.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at LiteDB.Engine.ConcurrentStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at LiteDB.Engine.DiskService.WriteLogDisk(IEnumerable`1 pages)
at LiteDB.Engine.TransactionService.PersistDirtyPages(Boolean commit)
at LiteDB.Engine.TransactionService.Commit()
at LiteDB.Engine.LiteEngine.CommitAndReleaseTransaction(TransactionService transaction)
at LiteDB.Engine.LiteEngine.Commit()

@ednsinf ednsinf added the bug label Sep 9, 2024
@JKamsker
Copy link
Collaborator

JKamsker commented Sep 9, 2024

Can you provide a repro?

@ednsinf
Copy link
Author

ednsinf commented Sep 9, 2024

This error occurs here
LiteDB\Engine\Engine\Transaction.cs -> AutoTransaction line 93
image

@ednsinf
Copy link
Author

ednsinf commented Sep 9, 2024

More details:
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at LiteDB.Engine.ConcurrentStream.Write(Byte[] buffer, Int32 offset, Int32 count) in C:\LiteDB\Engine\Disk\Streams\ConcurrentStream.cs:line 75
at LiteDB.Engine.DiskService.WriteDataDisk(IEnumerable1 pages) in C:\LiteDB\Engine\Disk\DiskService.cs:line 304 at LiteDB.Engine.WalIndexService.CheckpointInternal() in C:\LiteDB\Engine\Services\WalIndexService.cs:line 349 at LiteDB.Engine.WalIndexService.TryCheckpoint() in C:\LiteDB\Engine\Services\WalIndexService.cs:line 293 at LiteDB.Engine.LiteEngine.CommitAndReleaseTransaction(TransactionService transaction) in C:\Users\e_dns\Downloads\LiteDB\LiteDB\Engine\Engine\Transaction.cs:line 118 at LiteDB.Engine.LiteEngine.AutoTransaction[T](Func2 fn) in C:\LiteDB\Engine\Engine\Transaction.cs:line 91

@ednsinf
Copy link
Author

ednsinf commented Sep 10, 2024

Sugested solution to fix Memory Alocation error and inprove performance:
File: LiteDB\Engine\Disk\StreamFactory\StreamFactory.cs - line 33

public Stream GetStream(bool canWrite, bool sequencial)
 {
     const int bufferSize = 81920; // 80 KB    

     if (_password == null)
     {
         using (var bs = new BufferedStream(_stream, bufferSize))
         {
             return new ConcurrentStream(bs, canWrite);
         }
     }
     else
     {
         using (var bs = new BufferedStream(_stream, bufferSize))
         {
             return new AesStream(_password, new ConcurrentStream(bs, canWrite));
         }
     }
 }

@JKamsker
Copy link
Collaborator

You are always more than welcome to open a pr including the fix and tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants