Skip to content

Commit f38814c

Browse files
committed
Split change key
1 parent c8bff72 commit f38814c

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

src/pre_tokenizer.cpp

+2-1
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,8 @@ PreTokenizerConfig& PreTokenizerConfig::parse_json(const json& json_config) {
7878
type = json_config.at("type");
7979
if (type == "Split") {
8080
try {
81-
pattern = json_config.at("pattern");
81+
pattern = json_config.at("pattern").at("Regex");
82+
std::cout << "pattern: " << pattern.value() << std::endl;
8283
} catch (json::out_of_range&) {
8384
}
8485
} else if (type == "Digits") {

0 commit comments

Comments
 (0)