|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectnet.sourceforge.pmd.cpd.AnyTokenizer
public class AnyTokenizer
This class does a best-guess try-anything tokenization.
Field Summary | |
---|---|
static java.lang.String |
TOKENS
|
Constructor Summary | |
---|---|
AnyTokenizer()
|
Method Summary | |
---|---|
void |
tokenize(SourceCode sourceCode,
Tokens tokenEntries)
|
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final java.lang.String TOKENS
Constructor Detail |
---|
public AnyTokenizer()
Method Detail |
---|
public void tokenize(SourceCode sourceCode, Tokens tokenEntries)
tokenize
in interface Tokenizer
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |