Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit tests that aim to verify the behavior and correctness of the sampling pipeline under various conditions #1107

Draft
wants to merge 30 commits into
base: master
Choose a base branch
from

Conversation

csharpconsultant
Copy link

@csharpconsultant csharpconsultant commented Feb 21, 2025

This pull request adds several unit tests to the LlamaSharp library's SamplingTests.cs file. The added tests aim to verify the behavior and correctness of the sampling pipeline under various conditions. The tests include:

SamplingWithTopK:

This test checks the behavior of the LLamaSamplerChainHandle when using different TopK values during sampling.

Ensures the sampled tokens and resulting text differ when using TopK=5 and TopK=50.

FrequencyPenalty_ThrowsException_WhenValueIsLessThanMinusTwo:

Verifies that an ArgumentOutOfRangeException is thrown when the frequency penalty is less than -2.

FrequencyPenalty_ThrowsException_WhenValueIsGreaterThanTwo:

Verifies that an ArgumentOutOfRangeException is thrown when the frequency penalty exceeds 2.

PresencePenalty_ThrowsException_WhenValueIsLessThanMinusTwo:

Verifies that an ArgumentOutOfRangeException is thrown when the presence penalty is less than -2.

PresencePenalty_ThrowsException_WhenValueIsGreaterThanTwo:

Verifies that an ArgumentOutOfRangeException is thrown when the presence penalty exceeds 2.

DefaultValues_AreSetCorrectly:

Verifies that the default values in the DefaultSamplingPipeline are set correctly.

Seed_HasLowProbabilityOfCollision:

This test verifies that the DefaultSamplingPipeline generates seeds with a low collision probability.

Seed_IsInitializedWithSpecificValue:

Verifies that the pipeline seed can be initialized with a specific value.

SetMinKeep:

Tests setting the MinKeep property to a specific value.

GetMinKeepDefault:

Verifies that the default value of the MinKeep property is set correctly.

@csharpconsultant
Copy link
Author

I made the following updates:

Removed the non-deterministic sampling with temperature test.
Made the TopK test deterministic.
Removed the uint seed range test that would always pass.
Added a test to verify that the DefaultSamplingPipeline generates seeds with a low collision probability.

Adding LLama.Rag project to solution for implementation of RAG
@csharpconsultant csharpconsultant marked this pull request as draft February 24, 2025 02:58
Created an interface IWebScraper
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants