About custom caching implementation.

[Muratsavkli]

Hello everyone. I'm Murat. I've been learning programming for a year and a half.
Nowadays I'm developing a ASP.NET core Web API for mobile apps which can be used like chatbots.

First of all, many thanks @marcominerva for the contribution, it's so helpful to practice AI techs by using this library.

My question:

• I want to cache conversations to my database to prevent data loss.

• For this reason I created a new custom class named "DatabaseMessageCache" and implemented if from IChatGPTCache.

• In appsettings.json MessageLimit : 20 & MessageExpiration : 240:00:00

• The problem is only the message in SetupAsync() method comes from db but the other messages does not. So chatbot can not recognize the user and not collect the informations for the last 20 messages.

• I'm sharing Set() and Get() methods in the pictures. Thanks for your support.

[marcominerva]

Have you tried setting breakpoints in the DatabaseMessageCache class to verify if you correctly save and retrieve from the database?

[Muratsavkli]

Hello Marco. Thanks for your quick replay. Finally I found a solution for my case and it works properly with your library. I'm sharing related code blocks below to help any other rookie players. :) I would be pleased to hear any other advices about your experiences.

public async Task SetAsync(Guid conversationId, IEnumerable<ChatGptMessage> messages, TimeSpan expiration, CancellationToken cancellationToken = default)
{
	using var scope = _serviceScopedFactory.CreateScope();
	using var context = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();

	var existingCacheEntry = await context.MessageCacheEntries
		.Include(x => x.Messages)
		.FirstOrDefaultAsync(x => x.ConversationId == conversationId, cancellationToken);

	if (existingCacheEntry != null)
	{
		var messagesToAdd = messages
			.Select(x => new {x.Name, x.Content, x.Role})
			.Except(existingCacheEntry.Messages
					.Select(x => new { x.Name, x.Content, x.Role }));

		var messageList = messagesToAdd.Select(x => new CustomChatGptMessage()
		{
			Role = x.Role,
			Content = x.Content,
			Name = x.Name
		});
		
		existingCacheEntry.Messages.AddRange(messageList);
	}
	else
	{
		var messageList = messages.Select(x => new CustomChatGptMessage()
		{
			Role = x.Role,
			Content = x.Content,
			Name = x.Name
		});

		await context.MessageCacheEntries.AddAsync(new DatabaseMessageCacheEntry
		{
			ConversationId = conversationId,
			Messages = messageList.ToList()
		}, cancellationToken);
	}

	await context.SaveChangesAsync(cancellationToken);
}

public async Task<IEnumerable<ChatGptMessage>?> GetAsync(Guid conversationId, CancellationToken cancellationToken = default)
{
	using var scope = _serviceScopedFactory.CreateScope();
	using var context = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();

	var cacheEntries = await context.MessageCacheEntries
		.Include(x => x.Messages)
		.FirstOrDefaultAsync(x => x.ConversationId == conversationId, cancellationToken);

	return cacheEntries?.Messages?.Select(x => new ChatGptMessage()
	{
		Role = x.Role,
		Content = x.Content,
		Name = x.Name,
	});
}

[marcominerva]

Instead of explicitly using a ServiceProvider and creating a scope, you you try registering the cache as Scoped with the proper overload of the WithCache extension method (see https://github.com/marcominerva/ChatGptNet/blob/master/src/ChatGptNet/IChatGptBuilderExtensions.cs#L20).

[Muratsavkli]

Ohh thank you again. I missed this point. Refactored the registering of AddChatGpt service with the same lifetime of the dbContext. By doing this I got rid of code duplication. As it seems below 👍
Program.cs :

builder.Services.AddChatGpt(builder.Configuration).WithCache<DatabaseMessageCache>(ServiceLifetime.Scoped);

DatabaseMessageCache.cs :

public class DatabaseMessageCache : IChatGptCache
{
	private readonly ApplicationDbContext _context;

	public DatabaseMessageCache(ApplicationDbContext context)
	{
		_context = context;
	}

	public async Task SetAsync(Guid conversationId, IEnumerable<ChatGptMessage> messages, TimeSpan expiration, CancellationToken cancellationToken = default)
	{
		var existingCacheEntry = await _context.MessageCacheEntries
			.Include(x => x.Messages)
			.FirstOrDefaultAsync(x => x.ConversationId == conversationId, cancellationToken);

		if (existingCacheEntry != null)
		{
			var messagesToAdd = messages
				.Select(x => new {x.Name, x.Content, x.Role})
				.Except(existingCacheEntry.Messages
						.Select(x => new { x.Name, x.Content, x.Role }));

			var messageList = messagesToAdd.Select(x => new CustomChatGptMessage()
			{
				Role = x.Role,
				Content = x.Content,
				Name = x.Name
			});
			
			existingCacheEntry.Messages.AddRange(messageList);
		}
		else
		{
			var messageList = messages.Select(x => new CustomChatGptMessage()
			{
				Role = x.Role,
				Content = x.Content,
				Name = x.Name
			});

			await _context.MessageCacheEntries.AddAsync(new DatabaseMessageCacheEntry
			{
				ConversationId = conversationId,
				Messages = messageList.ToList()
			}, cancellationToken);
		}

		await _context.SaveChangesAsync(cancellationToken);
	}

	public async Task<IEnumerable<ChatGptMessage>?> GetAsync(Guid conversationId, CancellationToken cancellationToken = default)
	{
		var cacheEntries = await _context.MessageCacheEntries
			.Include(x => x.Messages)
			.FirstOrDefaultAsync(x => x.ConversationId == conversationId, cancellationToken);

		return cacheEntries?.Messages?.Select(x => new ChatGptMessage()
		{
			Role = x.Role,
			Content = x.Content,
			Name = x.Name,
		});
	}

	public async Task RemoveAsync(Guid conversationId, CancellationToken cancellationToken = default)
	{
		var cacheEntry = await _context.MessageCacheEntries
			.FirstOrDefaultAsync(x => x.ConversationId == conversationId, cancellationToken);

		if (cacheEntry != null)
		{
			_context.MessageCacheEntries.Remove(cacheEntry);
			await _context.SaveChangesAsync(cancellationToken);
		}
	}

	public async Task<bool> ExistsAsync(Guid conversationId, CancellationToken cancellationToken = default)
	{
		return await _context.MessageCacheEntries
			.AnyAsync(x => x.ConversationId == conversationId, cancellationToken);
	}
}