-
Notifications
You must be signed in to change notification settings - Fork 1
/
example.txt
220 lines (152 loc) · 12.1 KB
/
example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
Delete database
Creating database and tables
Reading KB content from scraped web
Scraping: https://mariadb.com/kb/en/vector-overview/
Scraped content: Vector Overview
Scraping: https://mariadb.com/kb/en/what-is-mariadb-galera-cluster/
Scraped content: What is MariaDB Galera Cluster?
Inserting KB content to MariaDB table kb_articles
Getting content from MariaDB kb_articles
Retrieved items: 2
Embedded chunk 1/2, page 1/2 'Vector Overview', lines 0-51 of 108, length 2599 of article 4421. Embed 0.76 s, MariaDB insert 0.01 s
Embedded chunk 2/2, page 1/2 'Vector Overview', lines 52-107 of 108, length 1812 of article 4421. Embed 0.74 s, MariaDB insert 0.01 s
Inserting 2 chunks from page 0 'Vector Overview' to MariaDB kb_chunks
Article processing time: 1.53 seconds. So far total embedding time: 0.00 seconds, vector insert time: 0.02 seconds
Embedded chunk 1/4, page 2/2 'What is MariaDB Galera Cluster?', lines 0-47 of 144, length 2126 of article 14331. Embed 0.60 s, MariaDB insert 0.01 s
Embedded chunk 2/4, page 2/2 'What is MariaDB Galera Cluster?', lines 48-90 of 144, length 5367 of article 14331. Embed 0.41 s, MariaDB insert 0.01 s
Embedded chunk 3/4, page 2/2 'What is MariaDB Galera Cluster?', lines 91-131 of 144, length 6213 of article 14331. Embed 0.56 s, MariaDB insert 0.01 s
Embedded chunk 4/4, page 2/2 'What is MariaDB Galera Cluster?', lines 132-143 of 144, length 608 of article 14331. Embed 0.52 s, MariaDB insert 0.01 s
Inserting 4 chunks from page 1 'What is MariaDB Galera Cluster?' to MariaDB kb_chunks
Article processing time: 2.13 seconds. So far total embedding time: 0.00 seconds, vector insert time: 0.06 seconds
Embedding Statistics:
Total processing time: 3.66 seconds
Total articles processed: 2
Average time per article: 1.83 seconds
Total chunks processed: 6
Average time per chunk: 0.61 seconds
Total embedding time: 0.00 seconds
Average embedding time per chunk: 0.00 seconds
Total vector insert time: 0.06 seconds
Average vector insert time per chunk: 0.01 seconds
Total chunks in MariaDB kb_chunks:
Total articles: 2
Total chunks: 6
Searching for 5 closest chunks to: 'Is MariaDB compatible with MySQL?'
Prompting LLM without closest chunks for question: 'Is MariaDB compatible with MySQL?'
Response from LLM:
Yes, MariaDB is designed to be compatible with MySQL. It was developed as a fork of MySQL, and its goal is to maintain compatibility with MySQL's features, APIs, and libraries. This means that many MySQL applications and tools can work seamlessly with MariaDB without modification. However, as both databases have evolved, there may be some differences in features and performance optimizations. It's always a good idea to test your specific application with MariaDB to ensure compatibility.
Prompting LLM with 5 closest chunks and question: 'Is MariaDB compatible with MySQL?'
Response from LLM:
The content provided does not answer the question of whether MariaDB is compatible with MySQL.
Total time to answer question: 3.66 seconds
Time to embed question and vector search for closest chunks: 0.51 seconds
Vector search time: 0.01 seconds
Time to prompt LLM: 3.15 seconds
Searching for 5 closest chunks to: 'tell me more about MariaDB Oracle mode'
Prompting LLM without closest chunks for question: 'tell me more about MariaDB Oracle mode'
Response from LLM:
MariaDB Oracle Mode is a compatibility feature in MariaDB designed to enhance the compatibility with Oracle Database. This mode allows users who are transitioning from Oracle to MariaDB to have a smoother experience by providing functionalities and syntactic similarities that resemble Oracle's SQL dialect. Here are some key aspects of MariaDB's Oracle Mode:
1. **SQL Syntax Compatibility**: Oracle Mode aims to mimic the SQL syntax used in Oracle databases. This includes support for specific functions, data types, and constructs that are common in Oracle.
2. **Data Types**: In Oracle Mode, MariaDB can support Oracle-specific data types, such as `NUMBER`, `VARCHAR2`, and `DATE`, among others, making it easier for developers accustomed to Oracle to work with MariaDB.
3. **PL/SQL Compatibility**: While MariaDB uses its own procedural language called MariaDB SQL, Oracle Mode can help in adapting PL/SQL code, facilitating the gradual migration of stored procedures and functions.
4. **Reserved Keywords**: Oracle Mode changes the interpretation of certain reserved keywords to align with Oracle's usage. This helps avoid conflicts when migrating applications that may use these keywords as identifiers.
5. **Compatibility Modes**: Users can switch to Oracle Mode through the MariaDB configuration, permitting finer control over the compatibility features being utilized. This is typically done through session variables.
6. **Functionality Mapping**: Many built-in functions in Oracle have their equivalents in MariaDB. Oracle Mode allows these functions to operate similarly, helping developers write code that can work in both environments with minimal changes.
7. **Migration Ease**: For organizations migrating from Oracle to MariaDB, enabling Oracle Mode can significantly reduce the costs associated with the transition by limiting the needed code modifications and retraining.
8. **Documentation and Examples**: The MariaDB documentation provides detailed insights and examples on how to use Oracle Mode effectively, helping developers understand the nuances involved.
Overall, MariaDB Oracle Mode is a valuable feature for those looking to leverage MariaDB's capabilities while maintaining a level of compatibility with traditional Oracle systems. It eases the migration process and allows users to take advantage of MariaDB's performance and features with less friction.
Prompting LLM with 5 closest chunks and question: 'tell me more about MariaDB Oracle mode'
Response from LLM:
I'm sorry, but the provided content does not contain any information about MariaDB Oracle mode.
Total time to answer question: 8.70 seconds
Time to embed question and vector search for closest chunks: 0.43 seconds
Vector search time: 0.01 seconds
Time to prompt LLM: 8.27 seconds
Searching for 5 closest chunks to: 'where can MariaDB Galera Cluster be downloaded?'
Prompting LLM without closest chunks for question: 'where can MariaDB Galera Cluster be downloaded?'
Response from LLM:
MariaDB Galera Cluster can be downloaded from the official MariaDB website. You can find it at the following link:
- [MariaDB Downloads](https://mariadb.org/download/)
On this page, you can choose the version of MariaDB that includes Galera Cluster and download the appropriate binaries for your operating system. Additionally, you can also find installation instructions and documentation related to setting up and using MariaDB Galera Cluster.
Prompting LLM with 5 closest chunks and question: 'where can MariaDB Galera Cluster be downloaded?'
Response from LLM:
MariaDB Galera Cluster can be downloaded from the following links:
- [Download Now](https://mariadb.com/downloads/)
- [Alternate download from mariadb.org](https://downloads.mariadb.org/mariadb/11.4.4/)
Total time to answer question: 5.12 seconds
Time to embed question and vector search for closest chunks: 0.64 seconds
Vector search time: 0.02 seconds
Time to prompt LLM: 4.49 seconds
Searching for 5 closest chunks to: 'What data types should be used for vector data?'
Prompting LLM without closest chunks for question: 'What data types should be used for vector data?'
Response from LLM:
In MariaDB, when dealing with vector data, a suitable approach would be to utilize the following data types:
1. **BLOB (Binary Large Object)**: If you are storing a raw binary representation of your vectors, BLOBs can handle large amounts of binary data.
2. **VARCHAR or TEXT**: If your vectors can be represented as a string (e.g., comma-separated values), you might consider using VARCHAR or TEXT. However, this may not be suitable for high-dimensional vectors due to size limitations.
3. **JSON**: If your vector data can be structured in a way that benefits from an array-like representation, you can use the JSON data type. This allows for easy querying and manipulation of vector data in a more organized format.
4. **Decimal or Float**: If the vectors are of a fixed length and composed of numerical values, you can use DECIMAL or FLOAT data types for each dimension of the vector. Create separate columns for each dimension.
Here's an example of how you might structure a table for 2D vectors:
```sql
CREATE TABLE vectors (
id INT AUTO_INCREMENT PRIMARY KEY,
x FLOAT,
y FLOAT
);
```
If you're dealing with higher dimensions, you might choose to have more columns or store the data as JSON.
Choosing the right type will depend on how you intend to use and query this vector data.
Prompting LLM with 5 closest chunks and question: 'What data types should be used for vector data?'
Response from LLM:
For vector data in MariaDB, the appropriate data type to use is `VECTOR`. Vectors store 32-bit IEEE 754 floating point numbers. Here’s an example of how to create a table with a vector column:
```sql
CREATE TABLE v (
id INT PRIMARY KEY,
v VECTOR(5) NOT NULL,
VECTOR INDEX (v)
);
```
In this example, the vector column `v` can store vectors with 5 dimensions.
Total time to answer question: 7.19 seconds
Time to embed question and vector search for closest chunks: 0.40 seconds
Vector search time: 0.01 seconds
Time to prompt LLM: 6.79 seconds
Searching for 5 closest chunks to: 'create vector index'
Prompting LLM without closest chunks for question: 'create vector index'
Response from LLM:
To create a vector index in MariaDB, you typically use the `CREATE INDEX` statement in conjunction with a data type that supports vector similarity, such as `FLOAT` or `BLOB`. However, as of my last update, MariaDB does not have built-in support for vector indices like some other databases (e.g., Postgres with its vector extension).
You can, however, create a basic index on a column that contains vector-like data by using multi-column indexing or other methods. Here’s a general example of how you might structure your data and create an index:
1. **Structure your table**: Define the table to store vectors, for instance, as an array of floats.
```sql
CREATE TABLE vectors (
id INT PRIMARY KEY,
vector_data FLOAT(5) ARRAY[3] -- assuming 3-dimensional vector for example
);
```
2. **Insert data**: Populate your table with vector values.
```sql
INSERT INTO vectors (id, vector_data) VALUES (1, ARRAY[1.0, 2.0, 3.0]);
INSERT INTO vectors (id, vector_data) VALUES (2, ARRAY[4.0, 5.0, 6.0]);
```
3. **Create an index**: While direct vector indexing isn't available, you can still create an index on each component of the vector separately (if you're storing them in separate columns):
```sql
CREATE INDEX idx_vector_component_x ON vectors (vector_data[1]);
CREATE INDEX idx_vector_component_y ON vectors (vector_data[2]);
CREATE INDEX idx_vector_component_z ON vectors (vector_data[3]);
```
Remember that while this can improve performance on queries filtering by these vector components, it doesn't provide the full capabilities of vector similarity search you might find in specialized databases.
For advanced vector operations like nearest neighbor search, you may need to handle this logic in your application layer or use additional libraries that can integrate with MariaDB. Always check the latest MariaDB documentation or version updates for new features, as capabilities may have changed after my last training cut-off.
Prompting LLM with 5 closest chunks and question: 'create vector index'
Response from LLM:
To create a vector index in MariaDB, you can define vectors using the `VECTOR INDEX` syntax within the `CREATE TABLE` statement. Here is an example:
```sql
CREATE TABLE v (
id INT PRIMARY KEY,
v VECTOR(5) NOT NULL,
VECTOR INDEX (v)
);
```
This defines a table with an `id` column and a `v` column that stores vectors, along with a vector index on the `v` column. Note that the `CREATE VECTOR INDEX` statement is not fully implemented yet, so for testing purposes, it is recommended to use the method shown above.
Total time to answer question: 10.11 seconds
Time to embed question and vector search for closest chunks: 0.56 seconds
Vector search time: 0.01 seconds
Time to prompt LLM: 9.56 seconds