How can I specify which characters StreamTokenizer treats as token delimiters?
Location: http://www.jguru.com/faq/view.jsp?EID=241596 Created: Oct 31, 2000 Modified: 2001-08-18 18:29:35.681 Author: Tim Rohaly (http://www.jguru.com/guru/viewbio.jsp?EID=10) Question originally posed by Satish Bhakta (http://www.jguru.com/guru/viewbio.jsp?EID=223736 StreamTokenizer breaks the input stream into tokens using whitespace as a delimiter. By default, Unicode characters \u0000 through \u0020 are considered whitespace. This encompasses things like space, tab, newline, etc. If you want to change this list, you need to invoke the method whitespaceChars(int low, int high); all characters having Unicode values between low and high will be considered whitespace, in addition to the default set. You can call whitespaceChars() any number of times – each invocation will add to the list of whitespace characters. The only way to clear out the list is to set those characters to be something other than whitespace – you might use ordinaryChar(int ch), ordinaryChars(int low, int high), wo
Location: http://www.jguru.com/faq/view.jsp?EID=241596 Created: Oct 31, 2000 Modified: 2001-08-18 18:29:35.681 Author: Tim Rohaly (http://www.jguru.com/guru/viewbio.jsp?EID=10) Question originally posed by Satish Bhakta (http://www.jguru.com/guru/viewbio.jsp?EID=223736 StreamTokenizer breaks the input stream into tokens using whitespace as a delimiter. By default, Unicode characters \u0000 through \u0020 are considered whitespace. This encompasses things like space, tab, newline, etc. If you want to change this list, you need to invoke the method whitespaceChars(int low, int high); all characters having Unicode values between low and high will be considered whitespace, in addition to the default set. You can call whitespaceChars() any number of times – each invocation will add to the list of whitespace characters. The only way to clear out the list is to set those characters to be something other than whitespace – you might use ordinaryChar(int ch), ordinaryChars(int low, int high), wo
