* query and enable cooperative matrix

* fix build with old vulkan sdk

* implement cooperative matrix optimization

* add nvidia-t4 coverage

* adjust test option for more coverage

Co-authored-by: ElvisYu <elvisyuovo@gmail.com>
Co-authored-by: 余浩文 <m18107220188@163.com>
Co-authored-by: Zr2223 <67497651+Zr2223@users.noreply.github.com>

* Fix codepaths for fp16 weights when only AVX is enabled

* Disable opt overrides

* Update SDK url

* Update vulkan SDK download version

* Debugging risv pad

* apply code-format changes

* fix padding test

* fix mips slice test

* fix lrn test

* implement mish swish image shader, fix pooling adaptive image storage support, drop debug output

* update ci ubuntu 18.04

Co-authored-by: nihui <shuizhuyuanluo@126.com>

* quantize and dequantize tests

* unify activation and usability function

* drop NCNN_REQUANT cmake option, test dequantize requantize pack8, fix webassembly build

* benchmark use requantize int8 model

* perf(innerproduct-arm): add aarch64 gemm

* fix(innerproduct): fix compilation errror

* fix(armv7-innerproduct): fix armv7 compilation error

* fix(innerproduct): fix gemm param

* fix(int8): update mock scales and fix runtime error

* fix(compilation): fix compilation error

* build shared lib and enable lto

* reserved for layer and option

* allocator pimpl

* datareader pimpl

* paramdict pimpl, disable copy assign for allocator and datareader

* modelbin pimpl

* net extractor pimpl

* gpu pimple

* disable copy assign vulkandevice, code format

* command pimpl, dummy image readonly

* pipeline pipelinecache pimpl, export platform class

* code format, export simple family

* update ci

* disable lto on android armv7, merge webassembly ci

* link libgcc, fix macos dylib version

* pipeline pimpl, gpu info pimpl

* destroy gpu info after vulkan device

* ignore msvc stl class warning

* fix ncnn_paramdict_get_float return type

* fix vktransfer upload fp16 without flatten, add command test

* SSE2: BatchNorm

* Fixed batch norm in AVX configuration

* Optimized register size switch

* Attempt to pass CI

* Attempt to pass CI

* Bias op

* Element wise ops

* Support packing on x86 by default

* Fixed macro range in bias

* Use aligned read for packed data

* Update testutil.h

* Update pooling_x86.cpp

* Support wasn SIMD

* Fix emscripten compiler flags

* fix build

* more ci fix

* concat x86 pack4

* flatten x86 pack4

* more x86 pack4

* ci pass

* fix

* enable sse2 mathfun

* enable --experimental-wasm-simd

Co-authored-by: nihui <shuizhuyuanluo@126.com>
Co-authored-by: nihuini <nihuini@tencent.com>

* add interp param align_corner

add check support_vulkan after create_pipeline for tests

* code style

Co-authored-by: yangfengmax <yangfengmax@didichuxing.com>

* [build] add toolchain file w/o stdcxx dependency

* [build] link m and gcc lib explicitly

* [ncnn] complete simple stl impl

* [ncnn] adapt for ncnn simplestl

* [test] adapt for ncnn simplestl

* [ncnn] fix missing algorithm and list when simplestl disabled

* [ncnn] fix guard for operator new and delete

* [style] fix the code style

* [build] fix build failed on darwin and emscripten

* [ci] do not import cxx to avoid operator conflict

* [ncnn] add temporary partial_sort impl using bubble sort

heap sort should be used for better perf.

* [ncnn] add std greater and less function

* [ncnn] fix placement new operator overload

* [ncnn] add operator delete with size info

* [build] disable exception, rtti, example and tools when simplestl on

* [build] add toolchain for arm simplestl

* [build] add toolchain for aarch64 simplestl

* [ncnn] move initializer to constructor

* [ncnn] use deteiled type instead of auto

* [ncnn] use plain lib name in target_link_libraries

* runtime cpu dispatch

* force thread one

* disable openmp for coverage

* simplify test layer

* print NCNN_TARGET_ARCH

* less ci build variants

* weight fp16 storage option

* test convdw int8

* apple a12 a13

* ncnn_add_layer ncnn_add_shader cmake macro

* added avx versions of commonly used layers

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* Unroll loops and increase test coverage

* Restyled/pull 1897 (#13)

* Restyle

Co-authored-by: Restyled.io <commits@restyled.io>

* cleanup variable declaration

* fix ctest

* Restyled/pull 1897 (#14)

Restyle
Co-authored-by: Restyled.io <commits@restyled.io>

* Fix ctest for convolutions without AVX.

Co-authored-by: Restyled.io <commits@restyled.io>

* added fp16 weight storage version

* Small changes

* Fixed fp16 weight storage layers

* fix innerproduct

* fix loop error

* Fix windows build.

Disable fp 16 conversion when detecting int8 weights.

Implement requested changes.

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* Update option.cpp

Set fp16 storage based on vulkan being used or not.

* added ability for storing state in lstm layer

* added avx lstm

* added arm lstm

* fix innerproduct activation location and add 4 parallel channel version

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* revert arm file

* commit before switch

* implement requested changes

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* More x86 optimized implementations of common layers.

Added LSTM layers for arm and x86 + a ctest to verify the layer accuracy

Added fp16 innerproduct for arm

* fix non avx build

* Add fp16 arm compiler and cpu checks. Remove statefullness from LSTM implementation.

* Fix build check for fp16 arm

* Bypass lstm_fp16 if not supported

* Build order was incorrect

* fix std::min missing in windows build

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* attempting to fix gnu build by enabling: -mfp16-format=ieee to fix the missing __fp16 type

* remove double "fix"

* Specify ieee fp16 format

* implement requested changes

* fix arm non-fp16 build

* fix arm lstm

* Restyled/pull 1881 (#15)

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

Co-authored-by: Restyled.io <commits@restyled.io>

* Check blob size on arm lstm

* fix styling

Co-authored-by: Restyled.io <commits@restyled.io>

* added fp16 weight storage version

* Small changes

* Fixed fp16 weight storage layers

* fix innerproduct

* fix loop error

* Fix windows build.

Disable fp 16 conversion when detecting int8 weights.

Implement requested changes.

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* Update option.cpp

Set fp16 storage based on vulkan being used or not.

* fix innerproduct activation location and add 4 parallel channel version

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* revert arm file

* implement requested changes

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

Co-authored-by: Restyled.io <commits@restyled.io>

* added avx implementations of FC and Max pool

* Specify AVX2

* Small fixes and using Fused avx activations

* fix type casting

* fixing some CI errors

* Fix code format

* fix pooling test

* remove vector typedef

* More compile fixes

* remove vector typedef

* set c++ version to 17

* Force c++ 17

* Fixing mathfun

* Try and workaround typedef issues

* typefix

* Remove typedef

* switch to static inline

* attempting to fix msvc bug

* Verified MSVX FIX

* Fixing clang build

* commit before switch

* More avx and packing implementation

* Fix ctest

* starting the depthwise pack 8 implementation

* Unrolled loop

* add depthwise pack 8 implementations

* Working 1x1 pack 8 implementation added

* revert incorrect changes

* added conact elempack 8

* more elempack enabled layers added and started on the conversion of the winograd pack4 conv 3x3

* Added code formatting

* fix styling

* Unroll loops

* unrolling loops

* Added more elempac layers for mobilenet v3

* revert commit

* fix code style

* remove arm neon references

* remove pack4 references

* More cleanup

* added packing avx code

* fixing linux build ctests

* remove usage of aligned loads

* More aligned mem ops removed

* Cleanup, revert some files and remove not working winograd and shufflechannel implementation

* add stackoverflow referal

* Fix windows build

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* implement requested chaanges

* remove reshape

* revert arm file change

* Restyled by clang-format

* Restyled by astyle

* Restyled by clang-format

* Restyled by astyle

* fix unterminated directive

Co-authored-by: Restyled.io <commits@restyled.io>

* enable image storage test

* update swiftshader, use subzero backend, opt default

* wip

* wip

* fix

* image and imageview can not be destroyed until command execution ends

* fast copy path for tightly packed data

* wip

* texture load works

* 1d 3d image

* record clone image, multiple commands share one image reference

* upload download image

* layer forward accept vkimagemat

* vkimagemat graph works

* staging vkimagemat for passing dynamic parameters, macro for fp32+image shader, padding image shader

* vkimagemat elemsize

* convolution test pass

* conv1x1s1 image shader

* fast staging image allocator from host memory, pooling image shader

* convolutiondepthwise image shader

* innerproduct image shader

* packing image shader

* crop deconvolution image shader

* resolve spirv binding types

* image fp16 and fp16a, cast image shader

* eltwise image shader

* wip

* absval image shader

* deconvolutiondepthwise image shader

* concat image shader, squeezenet works

* noop split image shader

* uniform precision hint

* layer support_image_storage

* wip

* vulkan device utility operator

* command is storage and packing option aware

* fallback to cpu on image allocation failed, mobilenetssd works

* flatten image shader, enable more test

* ci test

* check imgfp32 imgfp16 imgfp16a features

* fix ci test

* fix ci test

* upgrade swiftshader

* wip

* opt aggressive

* imgfp16p

* opt none

* convolution winograd image shader

* fix flush range, fast copy path for continous buffer

* minor fix

* fix innerproduct

* wip ...

* wip

* cast fix

* packing test

* wip

* image fp16p is fp16p

* wip

* silence

* more line info

* code clean

* softmax image shader

* vkmat and command api breaks

* always use compute queue for compute buffer transfer

* no barrier for readonly weight buffer

* record clone, drop queue_owner

* bring back layer forward

* fix validation errors

* lifecycle inside command makes life easier

* update doc

* record_import_android_hardware_buffer

* wip

* wip

* wip

* fix avx2 test

* wip

* fix slice pack8 test

* fix flatten pack8 test

* fix binaryop pack8 test

* fix interp pack8 test

* rewrite cast test for different blob type and packing