org.apache.lucene.analysis
Class WhitespaceTokenizer
public class WhitespaceTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace.
Adjacent sequences of non-Whitespace characters form tokens.
protected boolean | isTokenChar(char c) - Collects only characters which do not satisfy
Character.isWhitespace(char) .
|
WhitespaceTokenizer
public WhitespaceTokenizer(Reader in)
Construct a new WhitespaceTokenizer.
isTokenChar
protected boolean isTokenChar(char c)
Collects only characters which do not satisfy
Character.isWhitespace(char)
.
- isTokenChar in interface CharTokenizer
Copyright © 2000-2007 Apache Software Foundation. All Rights Reserved.