Import intrinsic library from openvino #2

luo-cheng2021 · 2023-07-06T04:23:18Z

Intrinsic library, import from PR: openvinotoolkit/openvino#17957.

zhangYiIntel

Some questions needs to be clarify.

include/llm_types.hpp

src/common/bf16.hpp

src/common/simple_parallel.hpp

src/emb_gpt_avx512.cpp

src/mha_gpt_amx.cpp

src/mm_kernel_common_amx.hpp

zhangYiIntel

LGTM, please add more comments for the each tile blocking strategy

src/mm_kernel_common_amx.hpp

(cherry picked from commit e5d88c4)

(cherry picked from commit fad8086)

(cherry picked from commit bef6924)

(cherry picked from commit 96b9a59)

luo-cheng2021 added 20 commits June 29, 2023 01:51

init library

ae336d5

strict N tail handling

2f8cd88

apply review comments

2cacaab

add mha support

a544031

move to gcc11; add mha_gpt bf16 support

3f18a48

restructure directory layout

d2ea598

use add_subdir to simplify cmake

3cfc8de

add clang12 support & apply review comments.

2a64a5a

mha int8 support

4aa2441

rotary bf16 gpt initial support

7ed062a

add chatglm support

28ba2d5

rotary embbeding supports contiguous pastkv

0590c6d

fc support int8 weight compression

67aabdc

support K<32(pad zero to 32)

f7723dc

import && rename

ae5fc0c

workaround MatmulVector K<=6*32

9725e59

export cmake target

aac4116

rotary use external cos/sin loopup table

323fa5f

qkv of emb changes to q, k, v

6d5971c

gelu tanh support

a741a8b

luo-cheng2021 mentioned this pull request Jul 6, 2023

Import intrinsic library from openvino #1

Closed

luo-cheng2021 added 4 commits July 6, 2023 18:41

add git commit id to package

eb427a6

add rotary avx2 kernel

16ea565

add bloom mha support

73f335d

use target_compile_options to change cxx flags

48e01f4

luo-cheng2021 force-pushed the luocheng/init_import branch 2 times, most recently from a000d16 to f906cdd Compare July 12, 2023 00:24

luo-cheng2021 added 3 commits July 12, 2023 16:20

mha uses tensor to support different strides

f906cdd

remove chatglm dependency

6205ab7

remove warning

27f324a

use custom allactor for map

7872bad

luo-cheng2021 force-pushed the luocheng/init_import branch 2 times, most recently from 0295279 to 7beabd6 Compare July 24, 2023 09:41

luo-cheng2021 added 3 commits July 24, 2023 17:45

support external causal mask[opt]

11833d8

fix coverity scan errors

7beabd6

remove ov namespace

2c0ae5b

luo-cheng2021 force-pushed the luocheng/init_import branch from 51ed425 to 2c0ae5b Compare July 25, 2023 04:11

luo-cheng2021 and others added 4 commits July 25, 2023 14:45

remove c++ global vars

e8455b7

fix simple_parallel_for type

31faf59

optimize mha_gpt_impl_amx::create

2326d60

add security.md and fix warnings.

04b8483

zhangYiIntel reviewed Jul 31, 2023

View reviewed changes

luo-cheng2021 added 2 commits August 1, 2023 01:25

apply review comments

77ac0bc

apply review comments

6de45ce

zhangYiIntel reviewed Aug 3, 2023

View reviewed changes

src/mm_kernel_common_amx.hpp Show resolved Hide resolved

src/mm_kernel_common_amx.hpp Show resolved Hide resolved

luo-cheng2021 added 4 commits August 5, 2023 04:22

fc weight support f32, add weight pack api

ca79cd9

(cherry picked from commit e5d88c4)

opt f32 weight pack

cbfe9ff

(cherry picked from commit fad8086)

remove writeable bufferC

3f31227

(cherry picked from commit bef6924)

use numa to alloc mem

76d5f0f

(cherry picked from commit 96b9a59)

luo-cheng2021 force-pushed the luocheng/init_import branch from 96b9a59 to 76d5f0f Compare August 15, 2023 06:23

luo-cheng2021 added 7 commits August 17, 2023 01:52

add falcon broadcast support before rotary

2dd181a

fix int8 compress

3a7432c

cache temp mem when K<32

4a4e48c

add numa as the first level task partition basis

c29681c

dlopen libnuma, remove compilation phase dependency

d30b5b5

avoid usage of vector

0d7dac4

support perchannel u8 compress weight

581a3cf

luo-cheng2021 force-pushed the luocheng/init_import branch from 3c7ac52 to 4f3ba52 Compare September 18, 2023 10:49

luo-cheng2021 added 2 commits September 19, 2023 02:43

opt for multi query

4f3ba52

fc M dimension splitting

aa2c57b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import intrinsic library from openvino #2

Import intrinsic library from openvino #2

luo-cheng2021 commented Jul 6, 2023

zhangYiIntel left a comment

zhangYiIntel left a comment

Import intrinsic library from openvino #2

Are you sure you want to change the base?

Import intrinsic library from openvino #2

Conversation

luo-cheng2021 commented Jul 6, 2023

zhangYiIntel left a comment

Choose a reason for hiding this comment

zhangYiIntel left a comment

Choose a reason for hiding this comment