• Leandro Melo's avatar
    C++: Core changes in preprocessing · d6ccffc0
    Leandro Melo authored
    Summary of most relevant items:
    - Preprocessor output format change. No more gen true/false. Instead
      a more intuitive and natural expansion (like from a real compiler) is
      performed directly corresponding to the macro invocation. Notice that
      information about the generated tokens is not lost, because it's now
      embedded in the expansion section header (in terms of lines and columns
      as explained in the code). In addition the location on where the macro
      expansion happens is also documented for future use.
    - Fix line control directives and associated token line numbers.
      This was not detected in tests cases because some of them were
      actually wrong: Within expansions the line information was being
      considered as originally computed in the macro definition, while
      the desired and expected for Creator's reporting mechanism (just
      like regular compilers) is the line from the expanded version
      of the tokens.
    - Do not allow for eager expansion. This was previously being done
      inside define directives. However, it's not allowed and might
      lead to incorrect results, since the argument substitution should
      only happen upon the macro invocation (and following nested ones).
      At least GCC and clang are consistent with that. See test case
      tst_Preprocessor:dont_eagerly_expand for a detailed explanation.
    - Revive the 'expanded' token flag. This is used to mark every token
      that originates from a macro expansion. Notice, however, that
      expanded tokens are not necessarily generated tokens (although
      every generated token is a expanded token). Expanded tokens that
      are not generated are those which are still considered by our
      code model features, since they are visible on the editor. The
      translation unit is smart enough to calculate line/column position
      for such tokens based on the information from the expansion section
    - How expansions are tracked has also changed. Now, we simply add
      two surrounding marker tokens to each "top-level" expansion
      sequence. There is an enumeration that control expansion states.
      Also, no "previous" token is kept around.
    - Preprocessor client methods suffered a change in signature so
      they now receive the line number of the action in question as
      a paramater. Previously such line could be retrieved by the client
      implementation by accessing the environment line. However, this
      is not reliable because we try to avoid synchronization of the
      output/environment lines in order to avoid unnecessary output,
      while expanding macros or handling preprocessor directives.
    - Although macros are not expanded during define directives (as
      mentioned above) the preprocessor client is now "notified"
      when it sees a macro. This is to allow usage tracking.
    - Other small stuff.
    This is all in one patch because the fixes are a consequence
    of the change in preprocessing control.
    Change-Id: I8f4c6e6366f37756ec65d0a93b79f72a3ac4ed50
    Reviewed-by: default avatarRoberto Raggi <roberto.raggi@nokia.com>
FastPreprocessor.cpp 2.6 KB