Change-Id: I898668a91eb7d3fa18c0a8d3e7e7507f0fb5a917
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>

Preprocessor did not correctly handle when variadic macro arguments were not
provided at all, if there were other arguments: macro was not expanded
in case only the non variadic arguments were given.

 #define MACRO(...)       used to work fine for 0 or more arguments.
 #define MACRO(ARG0, ...) used to work only for 2 or more arguments, now fixed.

Change-Id: I64e9199ceccae05618a49931c2adad8e4f9471ba
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>

Change-Id: Iacf8cb12dd623c908538d80ee2595297a9bdde71
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>

... and adjust INCLUDEPATH accordingly.

while i'm at messing with include statements, also re-order the include
blocks according to policy and sort them within bigger blocks.

Change-Id: I7762abfd7c4ecf59432b99db2f424e4fa25733a5
Reviewed-by: Tobias Hunger <tobias.hunger@digia.com>
Reviewed-by: Eike Ziller <eike.ziller@digia.com>

Change-Id: Ic6a9ff0359625021ebc061d22db6811814534205
Reviewed-by: Kai Koehne <kai.koehne@digia.com>

Track the typical #ifndef/#define/#endif usage in header files to see if
the macro is an include guard. If so, store it in the Document. No
behavioural change, just recording the name.

This can be used in the future to track if a file needs to be re-parsed
when a macro changes: if it was used in the file, and not defined in it
nor being the include-guard, a file should be re-preprocessed and
re-parsed.

It can also be used to check if two files have the same include guard.

Change-Id: I2715f529997a7b24a11bdbc6150652e2669f1a46
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>

When in 'keep comments' mode, the preprocessor does not properly handle macro calls with
comments between the macro name and the opening parenthesis: "FOO /*something to say*/
(45)".

Change-Id: I6fe733242e4d2ccff2985d17399d0a084917415a
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>

For shadow builds some tests couldn't find their test data. Solved by
using the same pattern as in Qt tests: Inject $$PWD as SRCDIR via a
DEFINES statement.

Note: It was considered to use QFINDTESTDATA for Qt5, but this was
rejected due to:
 1) It's Qt5 only (would mean to introduce some wrapper...)
 2) As the doc states, it will not work with QTEST_APPLESS_MAIN tests.

Change-Id: Ie6cf59570fe61725b7f6f83b7da650331ddb38fc
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>

Change-Id: Ice592c6de9951ee3b2c4cb52ed0bb3b6770e0825
Reviewed-by: Eike Ziller <eike.ziller@digia.com>

Task-number: QTCREATORBUG-7815

Change-Id: Id3e6b018bfd58b0f0072c637d69721a4bde1603a
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
Reviewed-by: Eike Ziller <eike.ziller@nokia.com>

Task-number: QTCREATORBUG-7780

Change-Id: Ie93704feff17ad8229e50fb1133048f2c7598dea
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
Reviewed-by: hjk <qthjk@ovi.com>

Notice that a similar problem still exists for which we
need to fix the lexer when there's a C style commend which
 ends with a backslash-newline.

Task-number: QTCREATORBUG-7713

Change-Id: I0f6d561703984f917fa5ed29de020ad0bdc5aaf0
Reviewed-by: David Schulz <david.schulz@nokia.com>
Reviewed-by: hjk <qthjk@ovi.com>

It was creating extra line(s)...

Change-Id: If28a9ccf16195c71747479db838a5589ea6683a0
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>

Task-number: QTCREATORBUG-7702
Change-Id: I0ec2e1eb9bf1c556b0a426d4405df1c48b5653ed
Reviewed-by: Leandro Melo <leandro.melo@nokia.com>

Change-Id: I7134d7de30bcf9f9dcfad42520dd45ee083a852d
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@nokia.com>

Summary of most relevant items:

- Preprocessor output format change. No more gen true/false. Instead
  a more intuitive and natural expansion (like from a real compiler) is
  performed directly corresponding to the macro invocation. Notice that
  information about the generated tokens is not lost, because it's now
  embedded in the expansion section header (in terms of lines and columns
  as explained in the code). In addition the location on where the macro
  expansion happens is also documented for future use.

- Fix line control directives and associated token line numbers.
  This was not detected in tests cases because some of them were
  actually wrong: Within expansions the line information was being
  considered as originally computed in the macro definition, while
  the desired and expected for Creator's reporting mechanism (just
  like regular compilers) is the line from the expanded version
  of the tokens.

- Do not allow for eager expansion. This was previously being done
  inside define directives. However, it's not allowed and might
  lead to incorrect results, since the argument substitution should
  only happen upon the macro invocation (and following nested ones).
  At least GCC and clang are consistent with that. See test case
  tst_Preprocessor:dont_eagerly_expand for a detailed explanation.

- Revive the 'expanded' token flag. This is used to mark every token
  that originates from a macro expansion. Notice, however, that
  expanded tokens are not necessarily generated tokens (although
  every generated token is a expanded token). Expanded tokens that
  are not generated are those which are still considered by our
  code model features, since they are visible on the editor. The
  translation unit is smart enough to calculate line/column position
  for such tokens based on the information from the expansion section
  header.

- How expansions are tracked has also changed. Now, we simply add
  two surrounding marker tokens to each "top-level" expansion
  sequence. There is an enumeration that control expansion states.
  Also, no "previous" token is kept around.

- Preprocessor client methods suffered a change in signature so
  they now receive the line number of the action in question as
  a paramater. Previously such line could be retrieved by the client
  implementation by accessing the environment line. However, this
  is not reliable because we try to avoid synchronization of the
  output/environment lines in order to avoid unnecessary output,
  while expanding macros or handling preprocessor directives.

- Although macros are not expanded during define directives (as
  mentioned above) the preprocessor client is now "notified"
  when it sees a macro. This is to allow usage tracking.

- Other small stuff.

This is all in one patch because the fixes are a consequence
of the change in preprocessing control.

Change-Id: I8f4c6e6366f37756ec65d0a93b79f72a3ac4ed50
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>

This reverts commit 44d8e7be.

The original test case was actually right. I guess I was mislead
by the recently added preprocessor tests, which are the ones
that should be corrected (together with the preprocessor, which
seems to have issues with # lineno generation).

Conflicts:

	tests/auto/cplusplus/preprocessor/tst_preprocessor.cpp

Change-Id: I47f82ed23a37086d0d81c4b3ea2cac48fb753451
Reviewed-by: hjk <qthjk@ovi.com>
Reviewed-by: Francois Ferrand <thetypz@gmail.com>

Do not expand function-like macros at all when there's a mismatch
in the parameter/argument count.

The report below raises the issue but its expected result is not
correct. This would be the more appropriate fix.

Task-number: QTCREATORBUG-7225
Change-Id: Ide8580faa7b724d3e8b396ec1f899cc5ca7f9e7e
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: I6d226b4e42f94ebee41d012050c5109895f0c432
Reviewed-by: hjk <qthjk@ovi.com>

It was previously beeing skipped.

Change-Id: Iadd9b03acbcf0ee0fb4db537b8597661cb93af3f
Reviewed-by: hjk <qthjk@ovi.com>

Make sure the environment line is consistent during preprocessor
directives and identifier handling so clients can rely on consistent
information. Particularly important for macro usages.

New tests also added.

Change-Id: I962a39a86cd17b8d945d2959c2c95e2d258ea3e6
Reviewed-by: hjk <qthjk@ovi.com>

I believe those were actually incorrect, since the # mark
from the generated tokens `int f` should be relative to <stdin>
on the line they are defined, which is 1.

Change-Id: I663ef49ad75eb8bb0a4a4b18d4899a952011536c
Reviewed-by: Francois Ferrand <thetypz@gmail.com>
Reviewed-by: hjk <qthjk@ovi.com>

An missing closing parantheses after "defined(..." caused an endless loop.

Change-Id: I19b17cebc1a56880216c3bfb67c9d296f80cc064
Reviewed-by: Leandro Melo <leandro.melo@nokia.com>

Change-Id: I902ebd73e039b8c3f44eca456be87809d1e1d3a4
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: I9d51798cb4d95141d30b0609d03cd1d199088f0a
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: I67367e501c8f19cee9023e3782c6a9175b49c6c6
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: I126143daeb572a555b73dc33ad7996b22c1387f8
Reviewed-by: hjk <qthjk@ovi.com>

Task-number: QTCREATORBUG-7256
Change-Id: I6149e56e5f50f75c9e039be9c162036a3890e2d7
Reviewed-by: David Schulz <david.schulz@nokia.com>
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: I1381ce17098a87dbc3924dc86902d7cc1b1eec76
Reviewed-by: hjk <qthjk@ovi.com>

Easier to follow.

Change-Id: I6da1ca862ddb7048a6d16bf5e6ea18d6b2cfc7f8
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: Ic81e3ebec1b42ae0698a340ee7e1c711225c1162
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: Idb93d359bd45696a38cb863dcea30c5c377eeabb
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: I7c843a449bb6db348ab374b4bd2c378b797b94cd
Reviewed-by: hjk <qthjk@ovi.com>

Change-Id: Iab9fb8ecbad94e08d39b6fad60caaf98a85d5980
Reviewed-by: hjk <qthjk@ovi.com>

The line corresponding to the macro definition is simply the token's
line. This caused errors in the macro definitions, since there were
multiple macros defined on the same line of the same file.

Task-number: QTCREATORBUG-7217
Change-Id: I56d17eeba677ac51f9eed283c0e964019bdfe4b7
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>

- This enables some tests which had been unexpectedly disabled in
earlier test.
- Tests to show issue QTCREATORBUG-7225.
- Tests which validates somehow behavior of preprocessor for
QTCREATORBUG-7217: the test passes, so the problem is likely not in the
startExpandingMacro() notification.

Change-Id: Ib7c1433e4dcdd8c1d19fa117371f94c1645ec3fb
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>

- Notify end of macro if function expansion is skipped.
- Do not notify client of generated macros expansion.

Change-Id: Ic027fc13ee391425a5ebadc8e84b9305912dbcf0
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>

Change-Id: I60d25109ae8fe3628b1899078a21010263787c33
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>

Change-Id: Id1c4046d028ad26d17b28feb53a832f3b48363f1
Reviewed-by: Erik Verbruggen <erik.verbruggen@nokia.com>

This rewrite fixes a couple of issues with the pre-processor. It now
supports:
- macros in macro bodies
- stringification of parameters [cpp.stringize]
- the concatenation operator [cpp.concat]
- #include MACRO_HERE
- defined() inside macro bodies used in pp-conditions.

Change-Id: Ifdb78041fb6afadf44f939a4bd66ce2832b8601f
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>