c_preproc

C/C++ preprocessor for finding dependencies

Reasons for using the Waf preprocessor by default

  1. Some c/c++ extensions (Qt) require a custom preprocessor for obtaining the dependencies (.moc files)
  2. Not all compilers provide .d files for obtaining the dependencies (portability)
  3. A naive file scanner will not catch the constructs such as “#include foo()”
  4. A naive file scanner will catch unnecessary dependencies (change an unused header -> recompile everything)

Regarding the speed concerns:

  • the preprocessing is performed only when files must be compiled
  • the macros are evaluated only for #if/#elif/#include
  • system headers are not scanned by default

Now if you do not want the Waf preprocessor, the tool +gccdeps* uses the .d files produced during the compilation to track the dependencies (useful when used with the boost libraries). It only works with gcc >= 4.4 though.

A dumb preprocessor is also available in the tool c_dumbpreproc

waflib.Tools.c_preproc.POPFILE

Constant representing a special token used in waflib.Tools.c_preproc.c_parser.start() iteration to switch to a header read previously

waflib.Tools.c_preproc.recursion_limit

Limit on the amount of files to read in the dependency scanner

waflib.Tools.c_preproc.go_absolute

Set to True to track headers on files in /usr/include, else absolute paths are ignored (but it becomes very slow)

waflib.Tools.c_preproc.use_trigraphs

Apply trigraph rules (False by default)

waflib.Tools.c_preproc.strict_quotes

Reserve the “#include <>” quotes for system includes (do not search for those includes). False by default.

waflib.Tools.c_preproc.g_optrans

Operators such as and/or/xor for c++. Set an empty dict to disable.

waflib.Tools.c_preproc.re_lines

Match #include lines

waflib.Tools.c_preproc.re_mac

Match macro definitions

waflib.Tools.c_preproc.re_fun

Match macro functions

waflib.Tools.c_preproc.re_pragma_once

Match #pragma once statements

waflib.Tools.c_preproc.re_nl

Match newlines

waflib.Tools.c_preproc.re_cpp

Filter C/C++ comments

waflib.Tools.c_preproc.trig_def

Trigraph definitions

waflib.Tools.c_preproc.chr_esc

Escape characters

waflib.Tools.c_preproc.NUM

Number token

waflib.Tools.c_preproc.OP

Operator token

waflib.Tools.c_preproc.IDENT

Identifier token

waflib.Tools.c_preproc.STR

String token

waflib.Tools.c_preproc.CHAR

Character token

waflib.Tools.c_preproc.tok_types

Token types

waflib.Tools.c_preproc.exp_types

Expression types

waflib.Tools.c_preproc.re_clexer

Match expressions into tokens

waflib.Tools.c_preproc.accepted

Parser state is accepted

waflib.Tools.c_preproc.ignored

Parser state is ignored, for example preprocessor lines in an #if 0 block

waflib.Tools.c_preproc.undefined

Parser state is undefined at the moment

waflib.Tools.c_preproc.skipped

Parser state is skipped, for example preprocessor lines in a #elif 0 block

waflib.Tools.c_preproc.repl(m)[source]

Replace function used with waflib.Tools.c_preproc.re_cpp

waflib.Tools.c_preproc.filter_comments(filename)[source]

Filter the comments from a c/h file, and return the preprocessor lines. The regexps waflib.Tools.c_preproc.re_cpp, waflib.Tools.c_preproc.re_nl and waflib.Tools.c_preproc.re_lines are used internally.

Returns:the preprocessor directives as a list of (keyword, line)
Return type:a list of string pairs
waflib.Tools.c_preproc.prec

Operator precendence rules required for parsing expressions of the form:

#if 1 && 2 != 0
waflib.Tools.c_preproc.trimquotes(s)[source]

Remove the single quotes around an expression:

trimquotes("'test'") == "test"
Parameters:s (string) – expression to transform
Return type:string
waflib.Tools.c_preproc.reduce_nums(val_1, val_2, val_op)[source]

Apply arithmetic rules to compute a result

Parameters:
  • val1 (int or string) – input parameter
  • val2 (int or string) – input parameter
  • val_op (string) – C operator in +, /, -, etc
Return type:

int

waflib.Tools.c_preproc.get_num(lst)[source]

Try to obtain a number from a list of tokens. The token types are defined in waflib.Tools.ccroot.tok_types.

Parameters:lst (list of tuple (tokentype, value)) – list of preprocessor tokens
Returns:a pair containing the number and the rest of the list
Return type:tuple(value, list)
waflib.Tools.c_preproc.get_term(lst)[source]

Evaluate an expression recursively, for example:

1+1+1 -> 2+1 -> 3
Parameters:lst (list of tuple(token, value)) – list of tokens
Returns:the value and the remaining tokens
Return type:value, list
waflib.Tools.c_preproc.reduce_eval(lst)[source]

Take a list of tokens and output true or false for #if/#elif conditions.

Parameters:lst (list of tuple(token, value)) – a list of tokens
Returns:a token
Return type:tuple(NUM, int)
waflib.Tools.c_preproc.error(*k, **kw)

Wrap logging.errors, display the origin of the message when ‘-vv’ is set

waflib.Tools.c_preproc.debug(*k, **kw)

Wrap logging.debug, the output is filtered for performance reasons

waflib.Tools.c_preproc.stringize(lst)[source]

Merge a list of tokens into a string

Parameters:lst (list of tuple(token, value)) – a list of tokens
Return type:string
waflib.Tools.c_preproc.paste_tokens(t1, t2)[source]

Token pasting works between identifiers, particular operators, and identifiers and numbers:

a ## b  ->  ab
> ## =  ->  >=
a ## 2  ->  a2
Parameters:
  • t1 (tuple(type, value)) – token
  • t2 (tuple(type, value)) – token
waflib.Tools.c_preproc.reduce_tokens(lst, defs, ban=[])[source]

Replace the tokens in lst, using the macros provided in defs, and a list of macros that cannot be re-applied

Parameters:
  • lst (list of tuple(token, value)) – list of tokens
  • defs (dict) – macro definitions
  • ban (list of string) – macros that cannot be substituted (recursion is not allowed)
Returns:

the new list of tokens

Return type:

value, list

waflib.Tools.c_preproc.eval_macro(lst, defs)[source]

Reduce the tokens by waflib.Tools.c_preproc.reduce_tokens() and try to return a 0/1 result by waflib.Tools.c_preproc.reduce_eval().

Parameters:
  • lst (list of tuple(token, value)) – list of tokens
  • defs (dict) – macro definitions
Return type:

int

waflib.Tools.c_preproc.extract_macro(txt)[source]
Process a macro definition of the form::
#define f(x, y) x * y

into a function or a simple macro without arguments

Parameters:txt (string) – expression to exact a macro definition from
Returns:a tuple containing the name, the list of arguments and the replacement
Return type:tuple(string, [list, list])
waflib.Tools.c_preproc.extract_include(txt, defs)[source]

Process a line in the form:

#include foo
Parameters:
  • txt (string) – include line to process
  • defs (dict) – macro definitions
Returns:

the file name

Return type:

string

waflib.Tools.c_preproc.parse_char(txt)[source]

Parse a c character

Parameters:txt (string) – character to parse
Returns:a character literal
Return type:string
class waflib.Tools.c_preproc.c_parser(nodepaths=None, defines=None)[source]

Bases: object

Used by waflib.Tools.c_preproc.scan() to parse c/h files. Note that by default, only project headers are parsed.

lines

list of lines read

nodepaths

Include paths

nodes

List of waflib.Node.Node found so far

names

List of file names that could not be matched by any file

curfile

Current file

ban_includes

Includes that must not be read (#pragma once)

cached_find_resource(node, filename)[source]

Find a file from the input directory

Parameters:
Returns:

the node if found, or None

Return type:

waflib.Node.Node

tryfind(filename)[source]

Try to obtain a node from the filename based from the include paths. Will add the node found to waflib.Tools.c_preproc.c_parser.nodes or the file name to waflib.Tools.c_preproc.c_parser.names if no corresponding file is found. Called by waflib.Tools.c_preproc.c_parser.start.

Parameters:filename (string) – header to find
Returns:the node if found
Return type:waflib.Node.Node
addlines(node)[source]

Add the lines from a header in the list of preprocessor lines to parse

Parameters:node (waflib.Node.Node) – header
start(node, env)[source]

Preprocess a source file to obtain the dependencies, which are accumulated to waflib.Tools.c_preproc.c_parser.nodes and waflib.Tools.c_preproc.c_parser.names.

Parameters:
waflib.Tools.c_preproc.scan(task)[source]

Get the dependencies using a c/c++ preprocessor, this is required for finding dependencies of the kind:

#include some_macro()

This function is bound as a task method on waflib.Tools.c.c and waflib.Tools.cxx.cxx for example

Previous topic

c_osx

Next topic

c_tests

This Page