Update to v2.0.4

RapidAI · Aug 21, 2023 · b20fe2d · b20fe2d
1 parent eb325da
commit b20fe2d
Show file tree

Hide file tree

Showing 12 changed files with 400 additions and 175 deletions.
diff --git a/.github/workflows/gen_whl_to_pypi.yml b/.github/workflows/gen_whl_to_pypi.yml
@@ -0,0 +1,76 @@
+name: Push rapid_paraformer to pypi
+
+on:
+  push:
+    branches: [ main ]
+    paths:
+      - 'python/rapid_paraformer/**'
+      - 'python/docs/doc_whl.md'
+      - 'python/setup.py'
+      - '.github/workflows/gen_whl_to_pypi.yml'
+
+# env:
+#   RESOURCES_URL: https://github.com/RapidAI/RapidLatexOCR/releases/download/v0.0.0/models.zip
+
+jobs:
+  # UnitTesting:
+  #   runs-on: ubuntu-latest
+  #   steps:
+  #     - name: Pull latest code
+  #       uses: actions/checkout@v3
+
+  #     - name: Set up Python 3.7
+  #       uses: actions/setup-python@v4
+  #       with:
+  #         python-version: '3.7'
+  #         architecture: 'x64'
+
+  #     - name: Display Python version
+  #       run: python -c "import sys; print(sys.version)"
+
+  #     - name: Download models
+  #       run: |
+  #         wget $RESOURCES_URL
+  #         ZIP_NAME=${RESOURCES_URL##*/}
+  #         DIR_NAME=${ZIP_NAME%.*}
+  #         unzip $ZIP_NAME
+
+  #     - name: Unit testings with rapid_latex_ocr
+  #       run: |
+  #         pip install -r requirements.txt
+  #         pip install pytest
+  #         pytest tests/test*.py
+
+  GenerateWHL_PushPyPi:
+    needs: UnitTesting
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python 3.7
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.7'
+          architecture: 'x64'
+
+      - name: Run setup.py
+        run: |
+          cd python
+          pip install -r requirements.txt
+          python -m pip install --upgrade pip
+          pip install wheel get_pypi_latest_version
+          python setup.py bdist_wheel ${{ github.event.head_commit.message }}
+
+      # - name: Publish distribution 📦 to Test PyPI
+      #   uses: pypa/[email protected]
+      #   with:
+      #     password: ${{ secrets.TEST_PYPI_API_TOKEN }}
+      #     repository_url: https://test.pypi.org/legacy/
+      #     packages_dir:  dist/
+
+      - name: Publish distribution 📦 to PyPI
+        uses: pypa/[email protected]
+        with:
+          password: ${{ secrets.PYPI_API_TOKEN }}
+          packages_dir: python/dist/
diff --git a/.gitignore b/.gitignore
@@ -1,6 +1,5 @@
 *.onnx
-
-*.pth
+*.json
 
 # Created by .ignore support plugin (hsz.mobi)
 ### Python template
@@ -21,6 +20,8 @@ dist/
 downloads/
 eggs/
 .eggs/
+lib/
+lib64/
 parts/
 sdist/
 var/
@@ -138,6 +139,7 @@ dmypy.json
 .vs
 .vscode
 .idea
+/models
 
 #models
 

diff --git a/README.md b/README.md
@@ -36,7 +36,9 @@ A([wav]) --RapidVad--> B([各个小段的音频]) --RapidASR--> C([识别的文
 #### 📣更新日志
 <details>
 <summary>详情</summary>
-
+- 2023-08-21 v2.0.4 update:
+  - 添加whl包支持
+  - 更新文档
 - 2023-02-25
    - 添加C++版本推理，使用onnxruntime引擎，预/后处理代码来自： [FastASR](https://github.com/chenkui164/FastASR)
 - 2023-02-14 v2.0.3 update:

diff --git a/python/.pre-commit-config.yaml b/python/.pre-commit-config.yaml
@@ -0,0 +1,19 @@
+repos:
+- repo: https://gitee.com/SWHL/autoflake
+  rev: v2.1.1
+  hooks:
+    - id: autoflake
+      args:
+        [
+          "--recursive",
+          "--in-place",
+          "--remove-all-unused-imports",
+          "--remove-unused-variable",
+          "--ignore-init-module-imports",
+        ]
+      files: \.py$
+- repo: https://gitee.com/SWHL/black
+  rev: 23.1.0
+  hooks:
+    - id: black
+      files: \.py$
diff --git a/python/README.md b/python/README.md
@@ -1,8 +1,12 @@
-## Rapid ASR
+## rapid_paraformer
 
 <p align="left">
-    <a href=""><img src="https://img.shields.io/badge/Python->=3.7,<=3.10-aff.svg"></a>
     <a href=""><img src="https://img.shields.io/badge/OS-Linux%2C%20Win%2C%20Mac-pink.svg"></a>
+    <a href=""><img src="https://img.shields.io/badge/Python->=3.6,<3.12-aff.svg"></a>
+    <a href="https://pepy.tech/project/rapid_paraformer"><img src="https://static.pepy.tech/personalized-badge/rapid_paraformer?period=total&units=abbreviation&left_color=grey&right_color=blue&left_text=Downloads"></a>
+    <a href="https://pypi.org/project/rapid_paraformer/"><img alt="PyPI" src="https://img.shields.io/pypi/v/rapid_paraformer"></a>
+    <a href="https://semver.org/"><img alt="SemVer2.0" src="https://img.shields.io/badge/SemVer-2.0-brightgreen"></a>
+    <a href="https://github.com/psf/black"><img src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
 </p>
 
 - 模型出自阿里达摩院[Paraformer语音识别-中文-通用-16k-离线-large-pytorch](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary)
@@ -13,61 +17,44 @@
 - [ ] 整合vad + asr + pun三个模型，打造可部署使用的方案
 
 #### 使用步骤
-1. 安装环境
-   ```bash
-    pip install -r requirements.txt
-   ```
-2. 下载模型
-   - 由于模型太大（823.8M），上传到仓库不容易下载，
-        - （推荐）自助转换：基于modescope下的notebook环境，可一键转换，详情戳：[快速体验](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary)
-            - 打开notebook → Cell中输入`!python -m funasr.export.export_model 'damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch' "./export" true`, 执行即可。
-        - 提供百度云下载连接：[asr_paraformerv2.onnx](https://pan.baidu.com/s/1-nEf2eUpkzlcRqiYEwub2A?pwd=dcr3)（模型MD5: `9ca331381a470bc4458cc6c0b0b165de`）
-   - 模型下载之后，放在`resources/models`目录下即可，最终目录结构如下：
-        ```text
-        .
-        ├── demo.py
-        ├── rapid_paraformer
-        │   ├── __init__.py
-        │   ├── kaldifeat
-        │   ├── __pycache__
-        │   ├── rapid_paraformer.py
-        │   └── utils.py
-        ├── README.md
-        ├── requirements.txt
-        ├── resources
-        │   ├── config.yaml
-        │   └── models
-        │       ├── am.mvn
-        │       ├── asr_paraformerv2.onnx  # 放在这里
-        │       └── token_list.pkl
-        ├── test_onnx.py
-        ├── tests
-        │   ├── __pycache__
-        │   └── test_infer.py
-        └── test_wavs
-            ├── 0478_00017.wav
-            └── asr_example_zh.wav
+1. Install
+    1. 安装`rapid_paraformer`
+        ```bash
+        pip install rapid_paraformer
         ```
-
-3. 运行demo
+    2. 下载**resources.zip** ([Google Drive](https://drive.google.com/drive/folders/1RVQtMe0eB_k6G5TJlmXwPELx4VtF2oCw?usp=sharing) | [百度网盘](https://pan.baidu.com/s/1zf8Ta6QxFHY3Z75fHNYKrQ?pwd=6ekq))
+        ```bash
+        resources
+        ├── [ 700]  config.yaml
+        └── [4.0K]  models
+            ├── [ 11K]  am.mvn
+            ├── [824M]  asr_paraformerv2.onnx
+            └── [ 50K]  token_list.pkl
+        ```
+    3. **asr_paraformerv2.onnx**文件可基于modescope下的notebook环境自助转换：
+        1. 打开[快速体验](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary)
+        2. 打开notebook → Cell中输入以下命令, 执行即可。
+            ```python
+            !python -m funasr.export.export_model --model-name 'damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch' --export-dir "./export"
+            ```
+
+2. 使用
     ```python
     from rapid_paraformer import RapidParaformer
 
+    config_path = "resources/config.yaml"
 
-    config_path = 'resources/config.yaml'
     paraformer = RapidParaformer(config_path)
 
-    # 输入：支持Union[str, np.ndarray, List[str]] 三种方式传入
-    # 输出： List[asr_res]
     wav_path = [
-        'test_wavs/0478_00017.wav',
+        "test_wavs/0478_00017.wav",
+        "test_wavs/asr_example_zh.wav",
     ]
 
     result = paraformer(wav_path)
     print(result)
     ```
-4. 查看结果
-   ```text
-   ['呃说不配合就不配合的好以上的话呢我们摘取八九十三条因为这三条的话呢比较典型啊一些数字比较明确尤其是时间那么我们要投资者就是了解这一点啊不要轻信这个市场可以快速回来啊这些配市公司啊后期又利好了可
-   以快速快速攻能包括像前一段时间啊有些媒体在二三月份的时候']
-   ```
+3. 查看结果
+    ```text
+    ['y', '欢迎大家来体验达摩院推出的语音识别模型']
+    ```
diff --git a/python/demo.py b/python/demo.py
@@ -3,21 +3,15 @@
 # @Contact: [email protected]
 from rapid_paraformer import RapidParaformer
 
-
-config_path = 'resources/config.yaml'
+config_path = "resources/config.yaml"
 
 paraformer = RapidParaformer(config_path)
 
 wav_path = [
-    'test_wavs/0478_00017.wav',
-    'test_wavs/asr_example_zh.wav',
-    'test_wavs/0478_00017.wav',
-    'test_wavs/asr_example_zh.wav',
-    'test_wavs/0478_00017.wav',
-    'test_wavs/asr_example_zh.wav',
+    "test_wavs/0478_00017.wav",
+    "test_wavs/asr_example_zh.wav",
 ]
 
 print(wav_path)
-# wav_path = 'test_wavs/0478_00017.wav'
 result = paraformer(wav_path)
 print(result)
diff --git a/python/docs/doc_whl.md b/python/docs/doc_whl.md
@@ -0,0 +1,45 @@
+## rapid_paraformer
+
+<p align="left">
+    <a href=""><img src="https://img.shields.io/badge/OS-Linux%2C%20Win%2C%20Mac-pink.svg"></a>
+    <a href=""><img src="https://img.shields.io/badge/Python->=3.6,<3.12-aff.svg"></a>
+    <a href="https://pepy.tech/project/rapid_paraformer"><img src="https://static.pepy.tech/personalized-badge/rapid_paraformer?period=total&units=abbreviation&left_color=grey&right_color=blue&left_text=Downloads"></a>
+    <a href="https://pypi.org/project/rapid_paraformer/"><img alt="PyPI" src="https://img.shields.io/pypi/v/rapid_paraformer"></a>
+    <a href="https://semver.org/"><img alt="SemVer2.0" src="https://img.shields.io/badge/SemVer-2.0-brightgreen"></a>
+    <a href="https://github.com/psf/black"><img src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
+</p>
+
+
+### Use
+1. Install
+    1. Install the `rapid_paraformer`
+        ```bash
+        pip install rapid_paraformer
+        ```
+    2. Download the **resources.zip** ([Google Drive](https://drive.google.com/drive/folders/1RVQtMe0eB_k6G5TJlmXwPELx4VtF2oCw?usp=sharing) | [Baidu NetDisk](https://pan.baidu.com/s/1zf8Ta6QxFHY3Z75fHNYKrQ?pwd=6ekq))
+        ```bash
+        resources
+        ├── [ 700]  config.yaml
+        └── [4.0K]  models
+            ├── [ 11K]  am.mvn
+            ├── [824M]  asr_paraformerv2.onnx
+            └── [ 50K]  token_list.pkl
+        ```
+2. Use
+    ```python
+    from rapid_paraformer import RapidParaformer
+
+    config_path = "resources/config.yaml"
+
+    paraformer = RapidParaformer(config_path)
+
+    wav_path = [
+        "test_wavs/0478_00017.wav",
+        "test_wavs/asr_example_zh.wav",
+    ]
+
+    result = paraformer(wav_path)
+    print(result)
+    ```
+
+### See details for [RapidASR](https://github.com/RapidAI/RapidASR).