Your mileage may vary but it's also a common issue having to filter out empty items out of something like ",foo,bar ,,,,baz,,xyzzy," where you only really want those four words. You will especially encounter this in parsing user input which might have extra whitespace, or badly formed lists of items.

Use strtok() to split C strings into tokens delimited by the given set of delimiters. If you want to catch each comma but skip over any whitespace and split at the first '=' only, then use something else.

Let's say if I'm going to need a configuration file for my program I'd most certainly start with something I can parse with strtok(). I would really need very specific needs to warrant a more complex format that would require a more sophisticated parser in which case using one would be a no-brainer.

Even if this is true, the reasoning here is disturbingly short-sighted. Copying code that you do not understand is unacceptable behavior, and I'd say the sooner it blows up in your face, the better. The goal of code examples is to illustrate how things work in a simplified way, and code without error checks is often easier to understand at first. Imagine a hello world with all the possible error checks. That would be incomprehensible.

If you have needs that require a general parsing library then why are you criticizing strtok()? It doesn't parse XML either, not C source code, nor any unspecified configuration file formats.

You know what else isn't in POSIX? All the rest of your C code.

    static char *p;
    if (!s && !(s = p)) return NULL;
    s += strspn(s, sep);
    if (!*s) return p = 0;
    p = s + strcspn(s, sep);
    if (*p) *p++ = 0;
    else p = 0;
    return s;

Overwriting the input in C is more efficient than maintaining more internal state and returning a pointer and the length of each token which you would need to strncpy() to get the token into a C string. strtok() does not want to do the initial strdup() for you because only you will know whether your input can already be mutated or whether you need to use a copy.

As I pointed in the other reply, strtok() does not break on strings like "keyword=value1,, , value3" unless you skipped RTFM and expect it to do something completely different. And more often than not that's exactly what you want when parsing non-computer readable input which you can expect to take a specific form.

If you want to handle escape sequences, parse from a stream (without having the option to fgets() the next line into memory), or parse CSV tables without collapsin colunms then you will want to use something more specific to that. Luckily, strtok() was not advertised as a Swiss army knife so it's off the hook for specific parsing purposes like those.

Generally though, C is just a terrible language to do anything other than write extremely low level routines in. You should probably just never use strtok() and go straight to something like Ragel or re2c for building high performance tokenizers... then call those from a higher level language.

And you cannot really do transformation instead of only matching with RE. Those are needed already in simple cases like string and number literals. Now your code will look more like

    %{
    #include "y.tab.h"
    int num_lines = 1;
    int comment_mode=0;
    int stack =0;
    %}
    digit ([0-9])
    integer ({digit}+)
    float_num ({digit}+\.{digit}+)
    %%
    {integer} {  //deal with integer 
                    printf("#%d: NUM:",num_lines); ECHO;printf("\n");
                    yylval.Integer = atoi(yytext);
                    return INT;
                   }
    {float_num} {// deal with float
                     printf("#%d: NUM:",num_lines);ECHO;printf("\n");
                     yylval.Float = atof(yytext);
                     return FLOAT;
                     }
    \n         { ++num_lines; }
    .          if(strcmp(yytext," "))ECHO;
    %%
    int yywrap() {
    return 1;
    }

Why isn't it a good choice exactly? Could you sum it up?

This is why it's crucial to get APIs right first time.

If you're implying that we should then use a regex implementation instead: Coding up a lexer (for a mainstream programming language) using simple counter increments and such is not a lot of work. It has the advantage that it results in faster code (unless you're going for real heavy machinery) and that you can easily code up additional transformations. For example, how would you parse string literals (including escape sequences) with a regex?

The one use-case I'm envisioning is quickly exposing POSIX conformant regexen to the command-line.

There are advantages and disadvantages of using libraries in any language. There are some aspects of C —e.g. the lack of garbage collection— that make it harder to reuse code compared to some languages. But it is definitely not against "C's spirit".

Wait, what? If C does not require A..Z to be contiguous, the distinction between 'A' and 0x41 is extremely significant to portable programs intending to parse ASCII when the native compiler encoding is whatever franken-coding doesn't have contiguous latin characters.