Scroll to navigation

XTOKID(1) User Commands XTOKID(1)

NAME

xtokid - Print all tokens found in a source file.

SYNOPSIS

xtokid [OPTION]... [FILE]...

DESCRIPTION

Print all tokens found in a source file.

include languages in LANGS (default: "C C++ asm")
exclude languages in LANGS

-l, --lang-option=L:OPT pass OPT as a default for language L (see below)

use MAPFILE to map file names onto source language
make LANG the default source language
exclude the named files and/or directories
tokenize only the files specified by NUL-terminated names in file F
display this help and exit --version output version information and exit

The following arguments apply to the language-specific scanners:

C language:

Allow CHARS in single-token strings, keep the result
Allow CHARS in single-token strings, toss the result
Strip a leading underscore from single-token strings

C++ language:

Allow CHARS in single-token strings, keep the result
Allow CHARS in single-token strings, toss the result
Strip a leading underscore from single-token strings

Java language:

Allow CHARS in single-token strings, keep the result
Allow CHARS in single-token strings, toss the result
Strip a leading underscore from single-token strings

Assembly language:

Any of CHARS starts a comment until end-of-line
Allow CHARS in tokens, and keep the result
Allow CHARS in tokens, and toss the result
Strip a leading underscore from tokens
Don't handle C pre-processor directives

Text language:

Treat characters of CHAR-CLASS as token constituents
Treat characters of CHAR-CLASS as token delimiters

Perl language:

Treat characters of CHAR-CLASS as token constituents
Treat characters of CHAR-CLASS as token delimiters
Include documentation tags

Lisp language:

REPORTING BUGS

Report bugs to bug-idutils@gnu.org

SEE ALSO

The full documentation for xtokid is maintained as a Texinfo manual. If the info and xtokid programs are properly installed at your site, the command

info xtokid

should give you access to the complete manual.

February 2012 xtokid - 4.6