Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Загрузка базы-эталона из DT или 1CD #124

Merged
merged 22 commits into from
Aug 30, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
60e184c
Имплементация загрузки из dt или 1cd
Segate-ekb Aug 24, 2024
3f12889
Описал Readme, Добавил vrunner settings во все места
Segate-ekb Aug 24, 2024
4108f98
Добавил имя стейджа в pipeline1C.groovy
Segate-ekb Aug 24, 2024
632fc87
fix у initInfobase опции не в lowercase
Segate-ekb Aug 24, 2024
eefbb89
Добавил степ createInfobase.groovy, убрал дублирование установки зави…
Segate-ekb Aug 24, 2024
b503ae6
Убрал лишнее обновление. После загрузки оно не нужно.
Segate-ekb Aug 24, 2024
d43d6bd
Выделил стейдж создания базы.
Segate-ekb Aug 26, 2024
5abf419
Update vars/pipeline1C.groovy
Segate-ekb Aug 26, 2024
6b90045
Update src/ru/pulsar/jenkins/library/configuration/TimeoutOptions.groovy
Segate-ekb Aug 26, 2024
55ff0d2
Заменил baseDBpath на templateDBPath
Segate-ekb Aug 26, 2024
9f07d3e
gradlew 7.6.1
nixel2007 Aug 26, 2024
be50b03
deps bump
nixel2007 Aug 26, 2024
5fb93d2
Additionally check toUri
nixel2007 Aug 26, 2024
258af0e
loadSources -> loadConfiguration
nixel2007 Aug 26, 2024
9c0ba58
JDK 17 on CI
nixel2007 Aug 26, 2024
50e73ea
Значение настроек vrunnerSettings "По умолчанию" теперь равно ./tools…
Segate-ekb Aug 27, 2024
f158b48
Фикс ошибки с загрузкой расширения
Segate-ekb Aug 27, 2024
a9fbf86
Исправил описание в схеме
Segate-ekb Aug 28, 2024
5c5b284
Небольшой рефакторинг
nixel2007 Aug 29, 2024
a656789
Вынос templateDBLoaded на уровень конфигурации
nixel2007 Aug 29, 2024
63b2800
Небольшие правки документации
nixel2007 Aug 29, 2024
7f9bf6b
Исправлена ошибка компиляции
nixel2007 Aug 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,25 @@ pipeline1C()
]
}
```
## Загрузка эталонной базы

Реализована возможность загрузки эталонной базы на этапе инициализации информационной базы. Для этого необходимо указать в конфигурационном файле параметр `initInfobase` -> `baseDBPath`:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

так как это дефолт, то вполне можно добавить и в основной раздел.


```json
"initInfobase": {
"baseDBPath": "путь/до/файла/базы.dt"
}
```
* Поддерживается загрузка файлов формата `.dt` и `.1CD`.
* Путь к файлу базы может быть как локальным, так и удаленным (URL).
* Если эталонная база содержит пользователей, то необходимо указать файл конфигурации, который будет использован для дальнейшей инициализации базы данных. Для этого необходимо указать параметр `initInfobase` -> `vrunnerSettings`:

```json
"initInfobase": {
"baseDBPath": "путь/до/файла/базы.dt",
"vrunnerSettings": "tools/vrunner.json"
}
```
## Настройка шага YAXUnit

* Добавить расширение `YAXUnit` и дополнительные расширения с тестами можно в `jobConfiguration.json` -> `initInfobase` -> `extensions`. Они будут загружены при инициализации ИБ.
Expand Down
2 changes: 2 additions & 0 deletions resources/globalConfiguration.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@
"initMethod": "fromStorage",
"runMigration": true,
"additionalInitializationSteps": [],
"baseDBPath": "",
"vrunnerSettings": "",
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved
"extensions": []
},
"bdd": {
Expand Down
8 changes: 8 additions & 0 deletions resources/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,14 @@
"type" : "string"
}
},
"vrunnerSettings" : {
"type" : "string",
"description" : "Путь к конфигурационному файлу vanessa-runner.\n По умолчанию не заполнено, требуется, если на этапе подготовки база загружается из архива.\n "
},
"baseDBPath" : {
"type" : "string",
"description" : "Путь к конфигурационному файлу эталонной базы данных.\n * По умолчанию не заполнен.\n * Указывается путь к файлу *.dt или *.1CD\n "
},
"extensions" : {
"type" : "array",
"description" : "Массив расширений для загрузки в конфигурацию.",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,18 @@ class InitInfoBaseOptions implements Serializable {
""")
String[] additionalInitializationSteps

@JsonPropertyDescription("""Путь к конфигурационному файлу vanessa-runner.
По умолчанию не заполнено, требуется, если на этапе подготовки база загружается из архива.
Segate-ekb marked this conversation as resolved.
Show resolved Hide resolved
""")
String vrunnerSettings

@JsonPropertyDescription("""
Путь к конфигурационному файлу эталонной базы данных.
* По умолчанию не заполнен;
* Указывается путь к файлу *.dt или *.1CD.
""")
String baseDBPath
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved

@JsonPropertyDescription("Массив расширений для загрузки в конфигурацию.")
Extension[] extensions

Expand Down Expand Up @@ -62,6 +74,8 @@ class InitInfoBaseOptions implements Serializable {
return "InitInfoBaseOptions{" +
"initMethod=" + initMethod +
", runMigration=" + runMigration +
", vrunnerSettings=" + vrunnerSettings +
", baseDBPath=" + baseDBPath +
", additionalInitializationSteps=" + additionalInitializationSteps +
", extensions=" + extensions +
'}'
Expand Down
59 changes: 59 additions & 0 deletions src/ru/pulsar/jenkins/library/steps/CreateInfobase.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
package ru.pulsar.jenkins.library.steps
Segate-ekb marked this conversation as resolved.
Show resolved Hide resolved

import ru.pulsar.jenkins.library.IStepExecutor
import ru.pulsar.jenkins.library.configuration.JobConfiguration
import ru.pulsar.jenkins.library.ioc.ContextRegistry
import ru.pulsar.jenkins.library.utils.Logger
import ru.pulsar.jenkins.library.utils.VRunner
import hudson.FilePath
import ru.pulsar.jenkins.library.utils.FileUtils

class CreateInfobase implements Serializable {

private final JobConfiguration config;

CreateInfobase(JobConfiguration config) {
this.config = config
}

def run() {
IStepExecutor steps = ContextRegistry.getContext().getStepExecutor()

Logger.printLocation()

def env = steps.env()

steps.installLocalDependencies();
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved

String baseDBPath = config.initInfoBaseOptions.baseDBPath
if (baseDBPath == '') {
// Не указан путь к базе данных, создадим пустую базу данных.
createBase()
} else if (baseDBPath.endsWith('.1CD')) {
// Это файл базы данных 1С, просто скопируем его.
String pathToInfobase = "$env.WORKSPACE/build/ib/1Cv8.1CD"
FileUtils.loadFile(baseDBPath, env, pathToInfobase)
} else if (baseDBPath.endsWith('.dt')) {
// Это файл дампа БД, скопируем его и создадим БД.
String pathToDt = "$env.WORKSPACE/build/tmp/dump.dt"
FileUtils.loadFile(baseDBPath, env, pathToDt)
createBase('build/tmp/dump.dt')
} else {
Logger.println("Неизвестный формат базы данных. Поддерживаются только .1CD и .dt")
}

}

private void createBase(String dtPath = '') {
Logger.println("Создание информационной базы")
String vrunnerPath = VRunner.getVRunnerPath();
def initCommand = "$vrunnerPath init-dev --ibconnection \"/F./build/ib\""
VRunner.exec(initCommand)

if (dtPath) {
// Загрузка из dt в vrunner 2.2.2 не работает корректно, потому инициировать через init-dev не получится.
def loadDtCommand = "$vrunnerPath restore --ibconnection \"/F./build/ib\" $dtPath"
VRunner.exec(loadDtCommand)
}
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved
}
}
36 changes: 2 additions & 34 deletions src/ru/pulsar/jenkins/library/steps/GetExtensions.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ class GetExtensions implements Serializable {
buildExtension(it, srcDir, vrunnerPath, steps)
} else if (it.initMethod == InitExtensionMethod.FILE){
Logger.println("Загрузка расширения ${it.name} из ${it.path}")
loadExtension(it, env)
String pathToExtension = "$pathToExtensionDir/${it.name}.cfe"
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved
FileUtils.loadFile(it.path, env, pathToExtension)
} else {
Logger.println("Неизвестный метод инициализации расширения ${it.name}")
}
Expand All @@ -73,22 +74,6 @@ class GetExtensions implements Serializable {
}
}

private void loadExtension(Extension extension, def env) {
String pathToExtension = "$env.WORKSPACE/${EXTENSIONS_OUT_DIR}/${extension.name}.cfe"
FilePath localPathToExtension = FileUtils.getFilePath(pathToExtension)

if (isValidUrl(extension.path)) {
// If the path is a URL, download the file
localPathToExtension.copyFrom(new URL(extension.path))
} else {
// If the path is a local file, copy the file
String localPath = getAbsolutePath(extension.path, env)
FilePath localFilePath = FileUtils.getFilePath(localPath)
localPathToExtension.copyFrom(localFilePath)
}
}


private String initVRunnerPath() {
return VRunner.getVRunnerPath()
}
Expand All @@ -108,21 +93,4 @@ class GetExtensions implements Serializable {
steps.unzip(sourceDirName, EdtToDesignerFormatTransformation.EXTENSION_ZIP)
}
}

private static boolean isValidUrl(String url) {
try {
new URL(url)
return true
} catch (MalformedURLException e) {
return false
}
}

private static String getAbsolutePath(String path, def env) {
if (path.startsWith("/") || path.startsWith("\\") || path.matches("^[A-Za-z]:.*")) {
return path
} else {
return "${env.WORKSPACE}/${path}"
}
}
}
14 changes: 11 additions & 3 deletions src/ru/pulsar/jenkins/library/steps/InitFromFiles.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import ru.pulsar.jenkins.library.configuration.SourceFormat
import ru.pulsar.jenkins.library.ioc.ContextRegistry
import ru.pulsar.jenkins.library.utils.Logger
import ru.pulsar.jenkins.library.utils.VRunner
import ru.pulsar.jenkins.library.steps.CreateInfobase
Segate-ekb marked this conversation as resolved.
Show resolved Hide resolved

class InitFromFiles implements Serializable {

Expand All @@ -25,7 +26,6 @@ class InitFromFiles implements Serializable {
return
}

steps.installLocalDependencies();
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved

Logger.println("Распаковка файлов")

Expand All @@ -43,7 +43,15 @@ class InitFromFiles implements Serializable {

Logger.println("Выполнение загрузки конфигурации из файлов")
String vrunnerPath = VRunner.getVRunnerPath();
def initCommand = "$vrunnerPath init-dev --src $srcDir --ibconnection \"/F./build/ib\""
VRunner.exec(initCommand)
def command = "$vrunnerPath update-dev --src $srcDir --ibconnection \"/F./build/ib\""

def options = config.initInfoBaseOptions

String vrunnerSettings = options.vrunnerSettings
if (vrunnerSettings != '' && steps.fileExists(vrunnerSettings)) {
command += " --settings $vrunnerSettings"
}

VRunner.exec(command)
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved
}
}
14 changes: 12 additions & 2 deletions src/ru/pulsar/jenkins/library/steps/InitFromStorage.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import ru.pulsar.jenkins.library.utils.Logger
import ru.pulsar.jenkins.library.utils.RepoUtils
import ru.pulsar.jenkins.library.utils.VRunner
import ru.pulsar.jenkins.library.utils.VersionParser
import ru.pulsar.jenkins.library.steps.CreateInfobase
Segate-ekb marked this conversation as resolved.
Show resolved Hide resolved

import static ru.pulsar.jenkins.library.configuration.Secrets.UNKNOWN_ID

Expand All @@ -31,7 +32,6 @@ class InitFromStorage implements Serializable {
return
}

steps.installLocalDependencies()

String storageVersion = VersionParser.storage()
String storageVersionParameter = storageVersion == "" ? "" : "--storage-ver $storageVersion"
Expand All @@ -54,8 +54,18 @@ class InitFromStorage implements Serializable {
'RUNNER_STORAGE_NAME'
)
]) {
Logger.println("Выполнение загрузки конфигурации из хранилища")
String vrunnerPath = VRunner.getVRunnerPath()
VRunner.exec "$vrunnerPath init-dev --storage $storageVersionParameter --ibconnection \"/F./build/ib\""
def command = "$vrunnerPath update-dev --storage $storageVersionParameter --ibconnection \"/F./build/ib\""

def options = config.initInfoBaseOptions

String vrunnerSettings = options.vrunnerSettings
if (vrunnerSettings != '' && steps.fileExists(vrunnerSettings)) {
command += " --settings $vrunnerSettings"
}

VRunner.exec(command)
Segate-ekb marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand Down
7 changes: 7 additions & 0 deletions src/ru/pulsar/jenkins/library/steps/InitInfoBase.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,13 @@ class InitInfoBase implements Serializable {
command += executeParameter;
command += ' --ibconnection "/F./build/ib"'

def options = config.initInfoBaseOptions

String vrunnerSettings = options.vrunnerSettings
if (vrunnerSettings != '' && steps.fileExists(vrunnerSettings)) {
command += " --settings $vrunnerSettings"
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved
}

Segate-ekb marked this conversation as resolved.
Show resolved Hide resolved
// Запуск миграции
steps.catchError {
VRunner.exec(command)
Expand Down
2 changes: 1 addition & 1 deletion src/ru/pulsar/jenkins/library/steps/LoadExtensions.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ class LoadExtensions implements Serializable {
return ""
}

String optionsName = "${stageName.toLowerCase()}Options"
String optionsName = "${stageName}Options"

def optionsInstance = config."$optionsName"

Expand Down
32 changes: 32 additions & 0 deletions src/ru/pulsar/jenkins/library/utils/FileUtils.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,36 @@ class FileUtils {
.replaceAll('\\\\', '/')
.toString()
}

static void loadFile(String filePathFrom, def env, String filePathTo) {

FilePath localPathToFile = getFilePath(filePathTo)

if (isValidUrl(filePathFrom)) {
// If the path is a URL, download the file
localPathToFile.copyFrom(new URL(filePathFrom))
} else {
// If the path is a local file, copy the file
String localPath = getAbsolutePath(filePathFrom, env)
FilePath localFilePath = getFilePath(localPath)
localPathToFile.copyFrom(localFilePath)
}
}
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved

private static boolean isValidUrl(String url) {
try {
new URL(url)
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved
return true
} catch (MalformedURLException e) {
return false
}
}

private static String getAbsolutePath(String path, def env) {
if (path.startsWith("/") || path.startsWith("\\") || path.matches("^[A-Za-z]:.*")) {
return path
} else {
return "${env.WORKSPACE}/${path}"
}
}
}
10 changes: 10 additions & 0 deletions vars/createInfobase.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import ru.pulsar.jenkins.library.configuration.JobConfiguration
import ru.pulsar.jenkins.library.ioc.ContextRegistry
import ru.pulsar.jenkins.library.steps.CreateInfobase

def call(JobConfiguration config) {
ContextRegistry.registerDefaultContext(this)

def createInfobase = new CreateInfobase(config)
createInfobase.run()
}
nixel2007 marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 2 additions & 1 deletion vars/pipeline1C.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ void call() {
createDir('build/out/')

script {
createInfobase config
Segate-ekb marked this conversation as resolved.
Show resolved Hide resolved
if (config.infoBaseFromFiles()) {
// Создание базы загрузкой из файлов
initFromFiles config
Expand All @@ -113,7 +114,7 @@ void call() {
}
steps {
timeout(time: config.timeoutOptions.loadExtensions, unit: TimeUnit.MINUTES) {
loadExtensions config
loadExtensions config, 'initInfoBase'
}
}
}
Expand Down