Arjan van de Ven
850b73dbb9
saxpy_haswell: Add AVX512 support
avx512 support fits nicely in the C+intrinsics code and gets a
speed improvement for vectors where the saxpy operation is not fully
memory bound
8 years ago
Arjan van de Ven
06ea72f5a5
write saxpy_haswell kernel using C intrinsics and don't disallow inlining
the intrinsics version of saxpy is more readable than the inline asm version,
and in the intrinsics version there's no reason anymore to ban inlining
(since the compiler has full visibility now) which gives a mid single digits
improvement in performance
8 years ago
Arjan van de Ven
d86604687f
saxpy_haswell: Use named arguments in inline asm
Improves readability
8 years ago
Arjan van de Ven
ef30a7239c
sdot_haswell: similar to ddot: turn into intrinsics based C code that supports AVX512
do the same thing for SDOT that the previous patches did for DDOT; the perf gain
is in the 60% range so at least somewhat interesting
8 years ago
Arjan van de Ven
21c6220d63
fix typo in dsymv avx512 code path
8 years ago
Arjan van de Ven
34d63df4b3
Add AVX512 support to DDOT
now that it's written in C + intrinsics it's easy to add AVX512 support
for DDOT
8 years ago
Arjan van de Ven
ae38fa55c3
Use intrinsics instead of inline asm
Intrinsics based code is generally easier to read for the non-math part
of the algorithm and it's easier to add, say, AVX512 to it later
8 years ago
Arjan van de Ven
847bbd6f4c
use named arguments in the inline asm
makes the asm easier to read
8 years ago
Arjan van de Ven
9c29524f50
various code cleanups and comments
8 years ago
Arjan van de Ven
f2810beafb
Add AVX512 support to dsymv_L_microk_haswell-2.c
Now that the code is written in intrinsics it's relatively easy to add AVX512 support
8 years ago
Arjan van de Ven
c202e06297
Write dsymv_kernel_4x4 for Haswell using intrinsics
intrinsics make the non-math part of the code easier to follow
than all hand coded asm, and it also helps getting ready for
adding avx512 support
8 years ago
Arjan van de Ven
0faba28adb
dsymv_L haswell: use symbol names for inline asm
symbolic names for gcc inline assembly are much easier to read
8 years ago
Arjan van de Ven
df31ec064e
Add AVX512 support to the dgemv_n_microk_haswell-4.c kernel
Now that the kernel is written in C-with-intrinsics, adding
AVX512 support to this kernel is trivial and yields a pretty significant
performance increase
8 years ago
Arjan van de Ven
e52d01cfe7
Also make the kernel_4x2 use intrinsics for readability and consistency
8 years ago
Arjan van de Ven
4a8ae8b8aa
replace the hasell dgemv_kernel_4x4 kernel with a the same code written in intrinsics
using intrinsics is a bit easier to read (at least for the non-math part of the code)
and also allows the compiler to be better about register allocation and optimizing the
non-math (loop/setup) code.
It also allows the code to honor the "no fma" flag if the user so desires.
The result of this change is (measured for a size of 16) a 15% performance increase.
And it is a step towards being able to add an AVX512 version of the code.
8 years ago
Arjan van de Ven
350531e76a
dgemv_n_microk_haswell: Use symbolic names for asm inputs to make the code more readable
gcc assembly syntax supports symbolic names in addition to numeric parameter order;
it's generally more readable to have code use the symbolic names
8 years ago
Martin Kroeker
6e54b0a027
Disable the 16x2 DTRMM kernel on SkylakeX as well
8 years ago
Martin Kroeker
f0a8dc2eec
Disable the AVX512 DGEMM kernel for now
due to #1643
8 years ago
Craig Donner
c2545b0fd6
Fixed a few more unnecessary calls to num_cpu_avail.
I don't have as many benchmarks for these as for gemm, but it should still
make a difference for small matrices.
8 years ago
Arjan van de Ven
89372e0993
Use AVX512 also for DGEMM
this required switching to the generic gemm_beta code (which is faster anyway on SKX)
for both DGEMM and SGEMM
Performance for the not-retuned version is in the 30% range
8 years ago
Arjan van de Ven
99c7bba8e4
Initial support for SkylakeX / AVX512
This patch adds the basic infrastructure for adding the SkylakeX (Intel Skylake server)
target. The SkylakeX target will use the AVX512 (AVX512VL level) instruction set,
which brings 2 basic things:
1) 512 bit wide SIMD (2x width of AVX2)
2) 32 SIMD registers (2x the number on AVX2)
This initial patch only contains a trivial transofrmation of the Haswell SGEMM kernel
to AVX512VL; more will follow later but this patch aims to get the infrastructure
in place for this "later".
Full performance tuning has not been done yet; with more registers and wider SIMD
it's in theory possible to retune the kernels but even without that there's an
interesting enough performance increase (30-40% range) with just this change.
8 years ago
Martin Kroeker
840e01061f
Merge pull request #1491 from martin-frbg/ddot_mt
Add multithreading support for Haswell DDOT
8 years ago
Martin Kroeker
a55694dd5b
Declare dot_compute static to avoid conflicts in multiarch builds
8 years ago
Martin Kroeker
85a41e9cdb
Add multithreading support for Haswell DDOT
copied from ashwinyes' implementation in dot_thunderx2t99.c
8 years ago
Martin Kroeker
81215711a2
Re-enable DAXPY microkernels for x86_64
as the inaccuracies seen in the original testcase for #1332 appear to be due to an artefact that amplifies the very small rounding differences between FMA and discrete multiply+add
8 years ago
Martin Kroeker
497f0c3d8a
Replace .align with .p2align in the Nehalem microkernels
8 years ago
Martin Kroeker
ea37db828e
Convert .align to .p2align for OSX compatibility
8 years ago
Martin Kroeker
7c1925acec
Use .p2align instead of .align for compatibility on Sandybridge as well
8 years ago
Martin Kroeker
2359c7c1a9
Use .p2align instead of .align for portability
The OSX assembler apparently mishandles the argument to decimal .align, leading to a significant loss of performance
as observed in #730 , #901 and most recently #1470
8 years ago
Martin Kroeker
e388459a27
Merge pull request #1419 from brada4/develop
Initialize unitialized values for repeated calls
8 years ago
Andrew
4938faa822
core.IdenticalExpr clang501 checker
8 years ago
Martin Kroeker
42285d8e70
Merge pull request #1410 from brada4/develop
Address warnings #1357
8 years ago
Andrew
4d0b005e5b
Eliminate remaining unused results in kernels (clang5 analyzer)
8 years ago
Martin Kroeker
b81656936f
Merge pull request #1409 from martin-frbg/issue1292-2
Tag %1 and %2 as both input and output operands
8 years ago
Martin Kroeker
b973990df2
Tag %1 and %2 as both input and output operands
fix from #1292 extended to the other gemv microkernels
8 years ago
Martin Kroeker
1e31124eb0
Merge pull request #1406 from martin-frbg/issue1292
Tag %1 and %2 as both input and output
8 years ago
Martin Kroeker
723f396a20
Tag %1 and %2 as both input and output
The inline assembly modifies its input operands, so mark them as output to avoid surprises with optimization. Fixes #1292
8 years ago
Martin Kroeker
43c0622e7b
Retire Piledriver/Steamroller/Excavator daxpy microkernels as well
related to issue #1332
8 years ago
Martin Kroeker
0623636c98
Use Sandybridge daxpy kernel on Haswell and Zen for now
The testcase from #1332 exposes a problem in daxpy_microk_haswell-2.c that is not seen with
any of the other Intel x86_64 microkernels.
8 years ago
Andrew
281a2b952f
warning cleanup ( #1380 )
* dead increments in driver/level2
* dead increments in kernel/generic
* part dead increments in kernel/x86_64
8 years ago
Martin Kroeker
6c77b5f267
Merge pull request #1369 from martin-frbg/dsdot
Add optimized dsdot to all other x86_64 kernels that use sdot.c
8 years ago
Martin Kroeker
c92cd6d162
Add trivially optimized dsdot based on sdot
8 years ago
Martin Kroeker
cae5d9a20b
Add trivially optimized dsdot based on sdot
8 years ago
Martin Kroeker
3d891c3106
Add trivially optimized dsdot based on sdot
8 years ago
Martin Kroeker
4fbdcfa823
Add trivially optimized dsdot based on sdot
8 years ago
Martin Kroeker
1bb6a96ebc
Add trivially optimized dsdot based on sdot
8 years ago
Martin Kroeker
6bd163f37a
Add trivially optimized dsdot based on sdot
8 years ago
Martin Kroeker
f0333333d1
Add trivially optimized dsdot based on sdot
8 years ago
Andrew
e89b979b2c
fix spurious compiler warning fix (no code change)
8 years ago
Andrew
7e9b29b9b8
fix spurious compiler warning (no code change)
8 years ago