[nasm:master] preproc: fix pasting of TOKEN_HERE, TOKEN_BASE and TOKEN_QMARK

[nasm-bot for H. Peter Anvin (Intel)]

Commit-ID:  5368e4579403daaf6c12165eb857893f8acfc7f8
Gitweb:     http://repo.or.cz/w/nasm.git?a=commitdiff;h=5368e4579403daaf6c12165eb857893f8acfc7f8
Author:     H. Peter Anvin (Intel) <hpa at zytor.com>
AuthorDate: Wed, 24 Mar 2021 10:46:45 -0700
Committer:  H. Peter Anvin (Intel) <hpa at zytor.com>
CommitDate: Wed, 24 Mar 2021 10:46:45 -0700

preproc: fix pasting of TOKEN_HERE, TOKEN_BASE and TOKEN_QMARK

Make the pasting behavior of TOKEN_QMARK, TOKEN_HERE and TOKEN_BASE
match the NASM 2.15 behavior: ? is a keyword and pastes as an ID, $
and $$ are treated as operators (which doesn't seem to make much
sense, but it is the current legacy behavior.)

Reported-by: C. Masloch <pushbx at ulukai.org>
Bugzilla: https://bugzilla.nasm.us/show_bug.cgi?id=3392733
Signed-off-by: H. Peter Anvin (Intel) <hpa at zytor.com>


---
 asm/preproc.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/asm/preproc.c b/asm/preproc.c
index ab229c9c..d1b9b31b 100644
--- a/asm/preproc.c
+++ b/asm/preproc.c
@@ -1,6 +1,6 @@
 /* ----------------------------------------------------------------------- *
  *
- *   Copyright 1996-2020 The NASM Authors - All Rights Reserved
+ *   Copyright 1996-2021 The NASM Authors - All Rights Reserved
  *   See the file AUTHORS included with the NASM distribution for
  *   the specific copyright holders.
  *
@@ -1478,7 +1478,7 @@ static Token *tokenize(const char *line)
             p++;
         } else if (nasm_isidstart(*p) || (*p == '$' && nasm_isidstart(p[1]))) {
             /*
-             * A regular identifier. This includes keywords, which are not
+             * A regular identifier. This includes keywords which are not
              * special to the preprocessor.
              */
             type = TOKEN_ID;
@@ -4907,7 +4907,8 @@ static inline bool pp_concat_match(const Token *t, enum concat_flags mask)
 
     switch (t->type) {
     case TOKEN_ID:
-        ctype = CONCAT_ID;      /* Ought this include $ and $$? */
+    case TOKEN_QMARK:           /* Keyword, treated as ID for pasting */
+        ctype = CONCAT_ID;
         break;
     case TOKEN_LOCAL_MACRO:
         ctype = CONCAT_LOCAL_MACRO;
@@ -4922,6 +4923,11 @@ static inline bool pp_concat_match(const Token *t, enum concat_flags mask)
     case TOKEN_FLOAT:
         ctype = CONCAT_NUM;
         break;
+    case TOKEN_HERE:
+    case TOKEN_BASE:
+        /* NASM 2.15 treats these as operators, but is that sane? */
+        ctype = CONCAT_OP;
+        break;
     case TOKEN_OTHER:
         ctype = CONCAT_OP;      /* For historical reasons */
         break;