CS 1713: The StringTokenizer

The Java StringTokenizer class

The Java StringTokenizer class can be used to break a string into pieces called tokens. The tokens in the string must be separated by an known set of characters called delimitors. By default, the delimitors used are the blank, tab, newline, carriage return, and form feed.

Look at the documentation at http://java.sun.com/j2se/1.5/docs/api/java/util/StringTokenizer.html.

The basic idea is to use the constructor to make a StringTokenizer from a string and use the methods:
int countTokens()
boolean hasMoreTokens()
String nextToken()
Example 1: Print the tokens of a String, one per line: 

    String s = "abcd efg hjik  lmn  opq";
    StringTokenizer stk = new StringTokenizer(s);
    while (stk.hasMoreTokens())
        System.out.println(stk.nextToken();
This will print: 
abcd
efg
hijk
lmn
opq
Example 2: Break a String into lines: 
    String s = readFromFile("myfile.txt");
    StringTokenizer stk = new StringTokenizer(s,"\n\r");
    while (stk.hasMoreTokens())
        System.out.println(stk.nextToken();