v0.2.6

cdf-tk

Improved

• The --verbose flag is now moved to the end of the command. For example, instead of cdf-tk --verbose build,
  you should now write cdf-tk build --verbose. The old syntax is still supported but will raise a deprecation warning.
• When running cdf-tk deploy --verbose you will now get a detailed output for each resource that has changed
  (or will change if you use --dry-run).
• Allow values test and qa as type in the config.[env].yaml file.

Fixed

• When running cdf-tk build with Views with custom filters, the Toolkit would likely give a UnusedParameterWarning.
  This is now fixed by not validating the details of View.filters. The motivation is that View.filters is a complex
  structure, and it is likely that you will get a false warning. The users that starts to use View.filters are
  expected to know what they are doing.
• If you run cdf-tk deploy and you had a child view that overrides a property from a parent view, the Toolkit would
  log it as changed even though it was not. This is now fixed.

Templates

No changes to templates.

Details

What's Changed

• [CDF-21530] 🫡Round snapshots. by @doctrino in #703
• [CDF-21909] 🐺Warning on has data filter by @doctrino in #704
• [CDF-21912]🦦 Moved Verbose by @doctrino in #705
• [CDF-21903]🦸Diffs by @doctrino in #706
• [CDF-21918] 🖐Allow type test and qa by @doctrino in #707
• Prepare 0.2.6 by @doctrino in #708

Full Changelog: v0.2.5...v0.2.6

v0.2.5

cdf-tk

Fixed

• When running cdf-tk build, with RAW tables in the selected modules, the Toolkit would always warn that the
  tables were missing, even though they were present. This is now fixed.
• When running cdf-tk init --upgrade <YOUR PROJECT> form version 0.1.4 the user would get a
  ERROR (ToolkitMigrationError): Failed to find migration from version 0.1.4.. This is now fixed.
• When running cdf-tk build, the Toolkit would give you warning when referencing a system Space, View, Container
  or DataModel. This is now fixed.
• [Feature Preview] When running cdf-tk import transformation-cli on a manifest with a query file that
  is separated from the manifest, the toolkit would raise a FileNotFoundError. This is now fixed.

Templates

No changes to templates.

Details

What's Changed

• [CDF-21908] 🤦Fix manual upgrade 0.1.4 by @doctrino in #697
• [CDF-21905] 😞Bug in import transformation-cli manifest by @doctrino in #698
• [CDF-21904] 😕Warning missig raw table by @doctrino in #699
• [CDF-21902] 🤏Not warn about system spaces. by @doctrino in #700
• Prepare 0.2.5 by @doctrino in #701

Full Changelog: v0.2.4...v0.2.5

v0.2.4

cdf-tk

Added

• [Feature Preview] Support for resource type Asset in the assets folder. Turn on the feature by running
  cdf-tk features set assets --enable.

Improved

• When running cdf-tk build and the selected modules is missing, the user will now get a hint about
  how to fix the issue.
• When running cdf-tk build and a module contains non-resource directories, the user will now get a warning
  that the directory is not a resource directory.

Fixed

• The data type of variables config.[env].yaml file is now preserved. For example, if you had my_variable: "123",
  then the cdf-tk build would build the resource file with the number instead of the string, my_variable: 123.
  This is now fixed.
• File configurations given as a list/array, lost the dataSetExternalId in the cdf-tk deploy command.
  This is now fixed.
• Added missing required dependency packaging to cognite-toolkit.

Templates

No changes to templates.

Details

• [CDF-21868] ⛓️‍💥Missing dataset by @doctrino in #684
• [CDF-21053] 😋Support key column in raw by @doctrino in #685
• [CDF-21869]🤡 Persist variable data type by @doctrino in #682
• [CDF-21669, CDF-21832] 💦Module Creation Issues by @doctrino in #680
• [CDF-21861] 🏭 Preview feature support for Assets by @doctrino in #688
• [CDF-21894] 🥺Feature preview dump assets by @doctrino in #689
• Fix require packaging by @doctrino in #694
• Prepare 0.2.4 by @doctrino in #695

Full Changelog: v0.2.3...v0.2.4

v0.2.3

cdf-tk

Improved

• When running cdf-tk build and missing CDF_PROJECT environment variable, the user will now get a more informative
  error message.

Fixed

• The variable type in the environment section of the config.[env].yaml now raises an error if it is not
  set to dev, staging, or prod.

Added

• The preview feature IMPORT_CMD added. This enables you to import a transformation-cli manifest into
  resource configuration files compatible with the cognite-toolkit. Activate by running
  cdf-tk features set IMPORT_CMD --enable, and deactivate by running cdf-tk features set IMPORT_CMD --disable.
  Run cdf-tk import transformation-cli --help for more information about the import command.

Templates

No changes to templates.

Details

What's Changed

• Removed misleading info by @ronpal in #669
• [CDF-21830]🤕 Added flaky decorator on flaky test by @doctrino in #670
• Fine-tuned selection logic by @ronpal in #672
• [CDF-21817] 😈Check Toolkit Group Correctly setup by @doctrino in #666
• 🐟 Fixed outdated help text by @ronpal in #675
• [CDF-21820] 🧹Cleanup tests_migrations by @doctrino in #673
• [CDF-21796] 🤨 Improved cdf-tk build if .env is missing by @doctrino in #674
• [CDF-21862] 😏Import transformation-cli by @doctrino in #677
• 🧯 made init . safer by @ronpal in #676
• 🚀Prepare 0.2.3 by @doctrino in #678

Full Changelog: v0.2.2...v0.2.3

v0.2.2

cdf-tk

Improved

• The command line messages have been improved to be more informative and user-friendly when running
  cdf-tk auth verify.
• The variables section in config.[env].yaml is now optional.
• In cdf-tk build, more informative error message when a variable is unresolved in a resource file.

Fixed

• In the cdf-tk auth verify command, if the flag --interactive was set, the --update-group and create-group
  flags were not ignored. This is now fixed.
• In the cdf-tk auth verify command, if there was no .env or --cluster and --project flags, the toolkit
  would raise an AuthentciationError, instead of prompting the user for cluster and project. This is now fixed.
• In the cdf-tk auth verify command, the if function service was not activated, the toolkit will
  now activate it.
• When running cdf-tk build, and a resource file was missing its identifier, for example, externalId for a
  dataset, an error such as AttributeError: 'NoneType' object has no attribute 'split' was raised. This is now fixed.

Templates

No changes

Details

What's Changed

• [CDF-21790] 🚣 Fix auth verify by @doctrino in #661
• [CDF-21822] 🪛Fix attribute error by @doctrino in #663
• [CDF-21819] 🤫feat; make variables optional by @doctrino in #664
• [CDF-21824] 😊Removed obsolete warning by @doctrino in #665
• Modules alignment by @ronpal in #657
• 🤠Prepare for 0.2.2 by @doctrino in #667

Full Changelog: v0.2.1...v0.2.2

v0.2.20

cdf-tk

Fixed

• When deploying containers resources with an index, the cdf-tk deploy would consider the resource as changed
  even though it was not. This is now fixed.

Templates

Fixed

• Added missing common_function_code to the cognite_modules/inrobot/cdf_inrobot_common module.

v0.2.0

cdf-tk

Added

• Support for resource type TransformationNotification in the transformations folder.
• Support for resource type Label in the labels folder.
• Support for loading nodes with APICall arguments. The typical use case is when node types are part of a
  data model, and the default APICall arguments works well.
• Support for securityCategories in the auth folder.
• Added support for resource type DatapointSubscription in the timeseries folder.
• Support for the Workflow and WorkflowVersion resource type
• Support for interactive login. The user can now set LOGIN_FLOW=interactive in the .env file
  to use interactive login.
• Support for specifying selected as paths and parent paths. For example, you can
  now write cognite_modules/core/cdf_apm_base instead of just cdf_apm_base. This is to support
  modules that have the same name but are in different parent directories. In addition, this also better reflects
  the structure of the cognite_modules and custom_modules folder better.
• If a resource is referring to another resource, the cdf-tk build will now give a warning if the referred resource
  is not found in the same build. For example, if you have a data model and is missing the space, the build command
  will give a warning that the space required by the data model is missing.
• The cdf-tk build command will now give warnings on duplicated resource. For example, if you have two files with
  the same externalId in the same module, the build command will give a warning that the externalId is duplicated,
  and that only the first file is used.
• Variables can now have extra spaces between curly braces and the variable name. For example, {{ my_variable }} is now
  a valid variable. Before this change, you would have to write {{my_variable}}.
• If an environment variable is not found in a resource file, for example, ${CDF_CLUSTER}, when
  running cdf-tk deploy the user will now get a warning message that the variable is missing. Before this change,
  this would pass silently and potentially cause an error when trying to deploy to CDF that was hard to debug.

Changed

• [BREAKING] cognite-tookit no longer supports common_function_code. The code used by functions must be in each
  function directory. The reason for this is that cognite-toolkit is a tool for governance and deployment of
  modules, it is not for development of functions. The common_function_code was a feature to support easier
  development of functions. It is expected that functions are developed in a separate environment and then
  moved to the cognite_modules folder for deployment and promotion between environments.
• [BREAKING] In function YAML config externalDataSetId is renamed to dataSetExternalId to be consistent with
  the naming convention used in the rest of the toolkit.
• [BREAKING] The cdf-tk build will now clean the build directory by default before building the modules to avoid
  unwanted side effects from previous builds. To stop this behavior, use the --no-clean flag.
• [BREAKING] The _system.yaml is now required to be on the root level when running any cdf-tk command. This
  means that the _system.yaml file must be in the same directory as the config.[env].yaml files. This is to
  support running cdf-tk without the cognite_modules folder.
• [BREAKING] In the config.[env].yaml files, the modules section is now renamed to variables. This is to
  better reflect the content of this section, which is variables that can be used in the resource files.
• In addition to cognite_modules and custom_modules, the cognite-toolkit now also support modules as the
  top-level folder for modules. This together with the two changes above, is to have a better support for running
  the cognite-toolkit as a standalone CLI without the cognite_modules.
• The .gitignore file you get by running cdf-tk init now ignores the /build by default.
• The dependency cognite-sdk must now be >=7.37.0 to use the cdf-tk.
• [BREAKING] In functions, the function config file must be in the root function directory. This means
  that, for example, my_module/function/some_folder/function.yaml will no longer be included by
  the cdf-tk build command. Instead, it must be in my_module/function/function.yaml. The motivation
  is to allow arbitrary YAML files as part of the function code.
• In config.[env].yaml, in the environment section, selected_modules_and_packages is renamed to selected.
  The old names will still work, but will trigger a deprecation warning.
• The toolkit now only gives a TemplateVariableWarning (Variable my_variable has value <change_me> ...) if
  the variable is used by selected in the config.[env].yaml. This is to avoid unnecessary warnings.
• The FeaturePrevieWarnings are no longer printed when running cdf-tk deploy or cdf-tk clean. These warnings
  are from the cognite-sdk and can be confusing to the user.
• In config.[env].yaml, in the environment section, name is no longer required. Instead, the [env] part
  of the config.[env].yaml file is used as the name of the environment. This is to avoid redundancy.
• If two modules have the same name, the cdf-tk build command will now stop and raise an error. Before this change,
  the cdf-tk build command would continue and overwrite the first module with the second module.
• The verification of access by the tool is now scoped to the resources that are being deployed instead of
  the entire project. This means that if the user only has access to a subset of the resources in the project,
  the tool will still be able to deploy those resources.

Improved

• Gives a more informative error message when the authentication segment of a transformation resource file is
  missing a required field.
• Transformation queries can be inline, i.e. set in either the Transformation query property in the yaml or
  as a separate file. If set in both, an error is raised because it is ambiguous which query to use.
• In the cdf-tk pull command, if an error occurs, the temporary directory was not removed. This is now fixed.
• Improved error message when running cdf-tk deploy/clean before running cdf-tk build.
• When running cdf-tk clean --dry-run the output would show local resources regardless of whether they existed
  in CDF or not. This is now fixed and only resources that exist in CDF are shown in the output.
• Better error message (no exception raised) if the config file has selected_modules_and_packages, but with no list items.
• If yaml files are invalid, a link to the API docs for the resource is shown in the error message.

Fixed

• When running cdf-tk clean or cdf-tk deploy --drop --drop-data there was an edge case that triggered the bug
  ValueError: No capabilities given. This is now fixed.
• When deploying containers resources with an index, the cdf-tk deploy would consider the resource as changed
  even though it was not. This is now fixed.
• When running cdf-tk init --upgrade from version 0.1.4 the user would get a ToolkitMigrationError.
  This is now fixed.
• The toolkit now ensures Transformations and Functions are deployed before Workflows
• The toolkit now ensures TimeSeries and Groups are deployed before DatapointSubscriptions.
• Running the build command, cdf-tk build, with Group resources scoped will read to incorrect
  warning such as WARNING [HIGH]: Space 'spaceIds' is missing and is required by: and
  WARNING [HIGH]: DataSet 'ids' is missing and is required by:. This is now fixed.
• Running the build command, cdf-tk build, with a View resource with a hasData filter would print a
  UnusedParameterWarning: Parameter 'externalId' is not used in section ('filter', 'hasData', 0, 'externalId')..
  This is incorrect and is now fixed to not print this warning.
• If you had a container with a direct relation property with a required constraint, the cdf-tk build command
  would incorrectly yield a warning that the Parameter 'type' is not used .... This is now fixed.
• Error message displayed to console on failed cdf-tk deploy command could be modified. This is now fixed.
• Using display name instead of folder name on a failed cdf-tk deploy or cdf-tk clean command. For example,
  if datapoints subscription was failing the error message would be Failure to load/deploy timeseries as expected,
  now it is Failure to load/deploy timeseries.subscription as expected.
• Unique display names for all resource types.
• Fixed bug when deploying extraction pipeline config, when none existed from before:
  There is no config stored for the extraction pipeline.
• In a function config, if you did not set fileId you would get an error when running cdf-tk deploy,
  Missing required field: 'fileId'.. The fileId is generated automatically when the function is created,
  so it is not necessary to set it in the config file. This is now fixed.
• If you do cdf-tk init --upgrade, on a pre 0.2.0a3 version, you are met with
  ERROR (ToolkitModuleVersionError): Failed to load previous version, .... This is now fixed.
• The parameter container.properties.<property>.type.list was required to be set, even thought it is optional
  in the CDF API. This is now fixed.
• The ExtractionPipelineConfig create, update and delete report numbers were incorrect. This is now fixed.
• When deploying a FunctionSchedule that requires an update, the cdf-tk would fail with error
  Failed to update functions.schedules. Error 'FunctionSchedulesAPI' object has no attribute 'update'..
  This is now fixed.
• When calling cdf-tk init --upgrade, the user is met with a Failed to load previous version, ....
  This is now fixed.
• When running `cdf-tk auth verif...

v0.2.0b4

cdf-tk

Added

• Support for resource type TransformationNotification in the transformations folder.

Changed

• [BREAKING] In functions, the function config file must be in the root function directory. This means
  that, for example, my_module/functions/some_folder/function.yaml will no longer be included by
  the cdf-tk build command. Instead, it must be in my_module/functions/function.yaml. The motivation
  is to allow arbitrary YAML files as part of the function code.
• The toolkit now only gives a TemplateVariableWarning (Variable my_variable has value <change_me> ...) if
  the variable is used by selected in the config.[env].yaml. This is to avoid unnecessary warnings.
• The FeaturePrevieWarnings are no longer printed when running cdf-tk deploy or cdf-tk clean. These warnings
  are from the cognite-sdk and can be confusing to the user.

Fixed

• When running cdf-tk init --upgrade from version 0.1.4 the user would get a ToolkitMigrationError.
  This is now fixed.

Templates

Added

• Extended cognite_modules/examples/my_example_module with a TransformationNotification case.

Details

• [CDF-21671] Allow arbitrary YAML in coding directory by @doctrino in #609
• [CDF-21656] 🤾Only warnings on selected by @doctrino in #610
• [CDF-21679]🥷silence feature preview warnings by @doctrino in #611
• Init ux by @ronpal in #612
• [CDF-21666] 👷‍♂️ Remove templates subpackage by @doctrino in #613
• [CDF-21683] 🔔Support transformation notification by @doctrino in #614
• [CDF-21648] 🌳 Bug when upgrading for 1.4 by @doctrino in #615
• 🎉Prepare 0.2.0b4 by @doctrino in #616

Full Changelog: v0.2.0b3...v0.2.0b4

v0.2.0b3

cdf-tk

Added

• Support for resource type Label in the labels folder.

Fixed

• The toolkit now ensures Transformations and Functions are deployed before Workflows
• The toolkit now ensures TimeSeries and Groups are deployed before DatapointSubscriptions.

Templates

Added

• Extended cognite_modules/examples/my_example_module with a Label case.

Details

• [CDF-21496] ➡Fix dependencies for Workflows. by @doctrino in #595
• 🐵 Remove . requirement by @doctrino in #596
• 😅 No more . requirement and fix Datapoint Subscription Dependencies. by @doctrino in #597
• Interactive init by @ronpal in #581
• 🫥Remove dot by @doctrino in #598
• [CDF-21662] 🧑‍💼Describe command class based. by @doctrino in #600
• [CDF-21663] 🧑‍🔬Dump command to class based by @doctrino in #601
• [CDF-21664] 🧑‍💻Pull command class based by @doctrino in #602
• [CDF-21665] 🧑‍🎨 Run command class based by @doctrino in #603
• [CDF-21668] 🏯 Support labels by @doctrino in #605
• [CDF-21661] 🧑‍🔧Moved auth command to class based by @doctrino in #599
• Prepare 0.2.0b3 by @doctrino in #606

Full Changelog: v0.2.0b2...v0.2.0b3

v0.2.0b2

cdf-tk

Fixed

• Running the build command, cdf-tk build, with Group resources scoped will read to incorrect
  warning such as WARNING [HIGH]: Space 'spaceIds' is missing and is required by: and
  WARNING [HIGH]: DataSet 'ids' is missing and is required by:. This is now fixed.
• Running the build command, cdf-tk build, with a View resource with a hasData filter would print a
  UnusedParameterWarning: Parameter 'externalId' is not used in section ('filter', 'hasData', 0, 'externalId')..
  This is incorrect and is now fixed to not print this warning.
• If you had a container with a direct relation property with a required constraint, the cdf-tk build command
  would incorrectly yield a warning that the Parameter 'type' is not used .... This is now fixed.

templates

No changes to templates.

Detailes

• 🌞 Rename subpackage load to loaders by @doctrino in #586
• [CDF-21640]🐛 Bug in checking dependent Groups by @doctrino in #587
• [CDF-21639, CDF-21644]🧑‍🚒Warning on valid HasData filter by @doctrino in #590
• [CDF-21645] 👮Warning on a valid container with direct relation constraint. by @doctrino in #592
• Prepare 0.2.0b2 by @doctrino in #593

Full Changelog: v0.2.0b1...v0.2.0b2