[Nasm-devel] problem with vcvtph2pd

[Henrik Gramner]

On Fri, Mar 22, 2024 at 6:30 PM Henrik Gramner <henrik at gramner.com> wrote:
>
> From what I can tell the root cause seems to be at
> https://github.com/netwide-assembler/nasm/blob/master/asm/assemble.c#L2508
> where opsize == 4 and BITS64 >> SIZE_SHIFT == 8, so the result of the
> division becomes 0.
>
> I don't really understand the logic in this function though so I'm not
> sure what the correct fix would be.

Fwiw the following seems to fix the issue, but I don't know if there
are side effects that ends up breaking something else:

--- a/asm/assemble.c
+++ b/asm/assemble.c
@@ -2505,7 +2505,7 @@ static uint8_t get_broadcast_num(opflags_t
opflags, opflags_t brsize)
      * The shift term is to take care of the extra BITS80 inserted
      * between BITS64 and BITS128.
      */
-    brcast_num = ((opsize / (BITS64 >> SIZE_SHIFT)) * (BITS64 / brsize))
+    brcast_num = ((opsize / (BITS32 >> SIZE_SHIFT)) * (BITS32 / brsize))
         >> (opsize > (BITS64 >> SIZE_SHIFT));

     return brcast_num;