strtok() in C

Syntax of strtok() in C

The syntax of the strtok() function is as follows:

char *strtok(char *str, const char *delim);

Here, str refers to the string that needs to be separated into tokens, and delim is the delimiter string containing all possible delimiters.

Parameters of C strtok() Function

• str: A pointer to the string that needs to be tokenized. On the first call, this is the string to be split. On subsequent calls, it should be NULL to continue tokenizing the same string.
• delim: A pointer to the string of delimiters. These characters are used to split the string into tokens.

Return Value of strtok() (Pointer Format)

• Return Value: A pointer to the next token found in the string, or NULL if no more tokens exist.

Examples of strtok()

Example 1: Tokenizing a Simple String

#include <stdio.h>
#include <string.h>

int main() {
    char str[] = "Hello, how are you?";
    char *token;
    
    // Get the first token
    token = strtok(str, " ,?");
    
    // Loop through the string to extract all tokens
    while (token != NULL) {
        printf("%s\n", token);
        token = strtok(NULL, " ,?");
    }
    
    return 0;
}

Output:

Hello
how
are
you

Example 2: Tokenizing a String with Multiple Delimiters

#include <stdio.h>
#include <string.h>

int main() {
    char str[] = "C|Programming&is|fun!";
    char *token;
    
    // Get the first token
    token = strtok(str, "|&!");
    
    // Loop through the string to extract all tokens
    while (token != NULL) {
        printf("%s\n", token);
        token = strtok(NULL, "|&!");
    }
    
    return 0;
}

Output:

C
Programming
is
fun

Example 3: Tokenizing a String by Space and Comma

#include <stdio.h>
#include <string.h>

int main() {
    char str[] = "apple, orange, banana, grape";
    char *token;
    
    // Get the first token
    token = strtok(str, ", ");
    
    // Loop through the string to extract all tokens
    while (token != NULL) {
        printf("%s\n", token);
        token = strtok(NULL, ", ");
    }
    
    return 0;
}

Output:

apple
orange
banana
grape

Practical Application of strtok()

Practical applications of using strtok() for common string parsing tasks:

1. CSV Parser (parseCSV):
   • Splits a comma-separated line into individual fields
   • Useful for processing CSV files or similar data formats
   • Tracks column numbers for better output formatting
2. Sentence Splitter (splitSentence):
   • Breaks a sentence into individual words
   • Handles multiple types of whitespace (space, tab, newline)
   • Counts total words in the sentence
3. Config File Parser (parseConfig):
   • Parses key-value pairs separated by '='
   • Handles whitespace around keys and values
   • Common use case for reading configuration files

Things to Note about C strtok()

While strtok() is a valuable tool, there are few things you should bear in mind:
 

• The strtok() function modifies the string it is tokenizing. It places a \0 character at the location of each delimiter to break up the string.
   
• It's not thread-safe because it uses a static pointer internally. For a thread-safe version, you might want to consider using strtok_r().
   
• It cannot handle multiple delimiters in a row, or tokens that begin or end with delimiter characters.

and Floyd's Triangle in C

Frequently Asked Questions

What does strtok() do in C?

strtok() splits a string into smaller tokens based on a set of delimiters. It modifies the original string by replacing delimiters with null terminators (\0) and returns pointers to the tokens for subsequent processing.

What is the use of strtok in C?

strtok() is used to break a string into smaller tokens based on specified delimiters, such as spaces, commas, or other characters. It allows easy extraction of substrings for further processing in programs that require parsing structured input.

Is strtok() a safe function to use in C?

strtok() is not entirely safe because it modifies the original string and is not thread-safe due to internal static state. For safer alternatives, consider using strtok_r() which avoids these issues.

What is the difference between strstr and strtok?

strstr() searches for a substring within a string and returns a pointer to its first occurrence, while strtok() splits a string into tokens based on specified delimiters. strstr() doesn't modify the string, but strtok() does.

Can strtok() handle multiple delimiters?

Yes, strtok() can handle multiple delimiters, but not multiple consecutive delimiters. It treats them as a single delimiter.

Conclusion

In this article, we discussed about strtok() in C. In C programming, strtok() is a powerful function for tokenizing strings, making it useful for parsing input or breaking down complex strings. However, it modifies the original string and is not thread-safe, so caution is advised when using it. For safer, multi-threaded applications, strtok_r() is a better alternative. 

Check out these useful blogs on - 

• Compilation process of a C program
• Conditional Operator in C
• Introduction to C
• Getch in C
• Call by value & call by reference in C
• Ternary Operator in C
• Bubble Sort Program in C
• odd or even program in c

Check out the following problems - 

• Sum Of Two Arrays.
• Merge K Sorted Arrays.
• Intersection Of Two Arrays.

Happy Reading!!‍