edu.northwestern.at.morphadorner.corpuslinguistics.tokenizer

## Class WhitespaceWordTokenizer

- java.lang.Object
- - edu.northwestern.at.utils.IsCloseableObject
  - - edu.northwestern.at.morphadorner.corpuslinguistics.tokenizer.AbstractWordTokenizer
    - - edu.northwestern.at.morphadorner.corpuslinguistics.tokenizer.WhitespaceWordTokenizer

- All Implemented Interfaces:
    WordTokenizer, IsCloseable, UsesLogger

      public class WhitespaceWordTokenizer
      extends AbstractWordTokenizer
      implements WordTokenizer

  Simple word tokenizer which splits on whitespace only.

- - ### Field Summary

    - ### Fields inherited from class edu.northwestern.at.morphadorner.corpuslinguistics.tokenizer.AbstractWordTokenizer

      `abbreviations, aposTokens, apostropheCanBeQuote, coalesceAsterisks, coalesceHyphens, contractions, contractionsURL, hyphensMatcher, hyphensPattern, logger, preTokenizer`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `WhitespaceWordTokenizer()` Create a word tokenizer than breaks on whitespace only.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `java.util.List<java.lang.String>` | `extractWords(java.lang.String text)` Break text into word tokens.

    - ### Methods inherited from class edu.northwestern.at.morphadorner.corpuslinguistics.tokenizer.AbstractWordTokenizer

      `addWordToSentence, findWordOffsets, getLogger, getPreTokenizer, isClosingQuote, isLetterOrSingleQuote, isMultipleHyphens, isSingleOpeningQuote, loadContractions, preprocessToken, setAbbreviations, setAposTokens, setLogger, setPreTokenizer, splitToken`

    - ### Methods inherited from class edu.northwestern.at.utils.IsCloseableObject

      `close`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

    - ### Methods inherited from interface edu.northwestern.at.morphadorner.corpuslinguistics.tokenizer.WordTokenizer

      `addWordToSentence, close, findWordOffsets, getPreTokenizer, preprocessToken, setAbbreviations, setAposTokens, setPreTokenizer`

    - ### Methods inherited from interface edu.northwestern.at.utils.IsCloseable

      `close`

- - ### Constructor Detail

    - #### WhitespaceWordTokenizer

          public WhitespaceWordTokenizer()

      Create a word tokenizer than breaks on whitespace only.

  - ### Method Detail

    - #### extractWords

          public java.util.List<java.lang.String> extractWords(java.lang.String text)

      Break text into word tokens.
      Specified by:
        `extractWords` in interface `WordTokenizer`
      Specified by:
        `extractWords` in class `AbstractWordTokenizer`
      Parameters:
        `text` - Text to break into word tokens.
      Returns:
        Input text broken on white space.