Code

multilabel ner training#79

Code

Merged

paluchasz merged 35 commits intomainAstraZeneca/KAZU:mainfrom multilabel-llm-ner-distilAstraZeneca/KAZU:multilabel-llm-ner-distilCopy head branch name to clipboardDec 6, 2024

+2,314-77

Commits

Commits on Oct 23, 2024

• updated label studio API to be compatible with latest version, which doesn't allow docs to have strings for ids

  RichJackson

  committed
  87000fd

• LLMNER step now has options to parse results from structured outputs, and process document sections individually

  RichJackson

  committed
  e18f4fd

• added additional Entity filter actions to support clean parsing of LLMNER generated data

  RichJackson

  committed
  b04a1f6

• added missing param to LSWebUtils

  RichJackson

  committed
  072088c

• fixed type hint for update view

  RichJackson

  committed
  31f9638

• added training code for multilabel NER

  RichJackson

  committed
  8a553bc

• added pyarrow to mypy overrides

  RichJackson

  committed
  fe87417

Commits on Oct 24, 2024

• added epoch completion fraction to TrainingConfig

  RichJackson

  committed
  1b2070b

• fixed bug with non-json serialised ents in metadata not appearing in gold standard during eval

  RichJackson

  committed
  8a5615a

• minor code cleanups

  RichJackson

  committed
  6ee718a

• added tests for new LLMNER cleanup actions

  RichJackson

  committed
  ebb8629

• added annotate_with_llm.py example script

  RichJackson

  committed
  622bf4c

• added example configs for training and annotate_with_llm

  RichJackson

  committed
  ed750c3

• fix: make parameter configurable for different architectures

  paluchasz

  committed
  b560cf0

• renamed eval to test so we can add an eval stage at some future point. Also removed confusing HF cache from training config as is probably superfluous

  RichJackson

  committed
  3502309

• fixed docstrings for modelling.py

  RichJackson

  committed
  4f883a3

• fixed param keys for test_path and test_data_cache_dir

  RichJackson

  committed
  524836c

• added scaling and training documentation, with required nitpicks

  RichJackson

  committed
  d29a6c2

• keys_to_use is now inferred from architecture

  RichJackson

  committed
  64f4600

• twoncrier: scaling and NER training

  RichJackson

  committed
  a2cf1ec

• added missing entity_key config param to StructuredOutputResultParser

  RichJackson

  committed
  2a6bafa

• default epoch_completion_fraction_before_evals is now 0.75 of an epoch

  RichJackson

  committed
  a283f2e

• removed unused keys_to_use from config

  RichJackson

  committed
  e26487c

• false pos and neg items now correctly updated

  RichJackson

  committed
  64c9ffb

• fixed mypy ignore for some strange linux issue

  RichJackson

  committed
  2efd307

Commits on Nov 7, 2024

• refactor: resuasbility and edge cases

  paluchasz

  committed
  e239c64

• feat: add predict script to test trained model

  paluchasz

  committed
  1e03b9b

• refactor: delete specific params

  paluchasz

  committed
  1836f1d

Commits on Nov 15, 2024

• refactor: separate out parquet to kazu docs conversion

  Show description for 4855298
  paluchasz

  committed
  4855298

• fix: remove custom paths from config

  paluchasz

  committed
  505e6c9

Commits on Nov 25, 2024

• fix: issue with fps and move function out of class

  paluchasz

  committed
  1b5b6e9

• refactor: retrieve labels from the model

  Show description for 167107d
  paluchasz

  committed
  167107d

• feat: add separate eval stage

  paluchasz

  committed
  5734563

• refactor: move functions to a reusable module

  paluchasz

  committed
  74cd82c

Commits on Dec 3, 2024

• refactor: save out predictions in eval script

  paluchasz

  committed
  9b4f3ba