NAME¶

tok - space separated token stream routines

SYNOPSIS¶

#include <ast.h>

void* tokopen(char* string) char* tokread(void* tok) void tokclose(void* tok)

int tokscan(char* string, char** next, const char* format, ...);

Sfio_t* tokline(const char* input, int flags, int* line);

DESCRIPTION¶

tokopen returns a pointer to a space separated token stream on the 0 terminated string string. tokread returns a pointer to the next space separated token in the token stream tok as returned by tokopen. 0 is returned when no tokens remain. tokread temporarily modifies string by inserting 0's to terminate each token. tokclose closes the token stream and restores string to its original state.

tokscan scans the string string for tokens specified in format. It is a more forgiving sscanf(3). If next != 0 then it will point to the next unread character in string on return. The number of scanned tokens is returned. -1 is returned if string was not empty and format failed to match and tokens.

space in format matches 0 or more space or tab characters. newline in format eats the remainder of the current line in string. "...", '...' and \character quotes are interpreted. A quoted carriage-return is converted to newline. newline in string is equivalent to end of string except when quoted. \newline is a line splice.

% in format prefixes format conversion characters; each conversion character corresponds to a tokscan argument following the format argument. The format conversions are:

%c

A single char.

%hd %d %ld

[short, int, long] base 10 integer.

%hn %n %ln

[short, int, long] C-style base integer.

%ho %o %lo

[short, int, long] base 8 integer.

%s

String.

%hu %u %lu

[short, int, long] C-style base unsigned integer.

%v

The next two arguments are a pointer to a char** argument vector and the maximum number of elements in the vector.

%hx %x %lx

[short, int, long] base 16 integer.

%s and %v data may also be counted length strings of the form (count:data) where count is the number of characters in data and the terminating ) may also be a tab, or the data may be (null) which represents the NULL string.

tokline returns an sfio(3) stream to a file or string that splices \newline into single lines, allows "..." and '...' to quotes to span newlines (done by translating quoted newline to carriage-return; tokscan above converts quoted carriage-return back to newline), and deletes # ... newline comments. This is done by pushing an sfio discipline onto a string or file stream. Seeks are disabled on the resulting stream. If flags == SF_READ then input is a file name; If flags == SF_STRING then input is a 0 terminated string; otherwise input is an open Sfio_t* stream. If line != 0 then it points to a line count that is initialized to 0 and is incremented for each input line.

SEE ALSO¶

sfio(3)