edu.northwestern.at.morphadorner.corpuslinguistics.sentencesplitter

## Class AbstractSentenceSplitter

- java.lang.Object
- - edu.northwestern.at.utils.IsCloseableObject
  - - edu.northwestern.at.morphadorner.corpuslinguistics.sentencesplitter.AbstractSentenceSplitter

- All Implemented Interfaces:
    SentenceSplitter, IsCloseable, UsesLogger
  Direct Known Subclasses:
    ICU4JBreakIteratorSentenceSplitter

      public abstract class AbstractSentenceSplitter
      extends IsCloseableObject
      implements SentenceSplitter, IsCloseable, UsesLogger

  Abstract sentence splitter.

  The base class for sentence splitters.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected Abbreviations` | `abbreviations` Abbreviations.
    `protected static java.lang.String` | `disallowedSentenceStarters` Characters not allowed to start a sentence.
    `protected Logger` | `logger` Logger used for output.
    `protected Names` | `names` Name recognizer.
    `protected PartOfSpeechGuesser` | `partOfSpeechGuesser` Part of speech guesser used by some sentence splitters.
    `protected SentenceSplitterIterator` | `sentenceSplitterIterator` Sentence iterator.
    `protected WordTokenizer` | `wordTokenizer` Default word tokenizer used if none specified.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `AbstractSentenceSplitter()`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected void` | `addSentence(java.util.List<java.lang.String> sentence, java.util.List<java.util.List<java.lang.String>> sentenceList)` Add sentence to sentence list.
    `protected void` | `addSentenceBad(java.util.List<java.lang.String> sentence, java.util.List<java.util.List<java.lang.String>> sentenceList)` Add sentence to sentence list.
    `java.util.List<java.util.List<java.lang.String>>` | `extractSentences(java.lang.String text)` Break text into sentences and tokens.
    `java.util.List<java.util.List<java.lang.String>>` | `extractSentences(java.lang.String text, WordTokenizer tokenizer)` Break text into sentences and tokens.
    `int[]` | `findSentenceOffsets(java.lang.String text, java.util.List<java.util.List<java.lang.String>> sentences)` Find starting offsets of sentences extracted from a text.
    `protected boolean` | `fixUpSentence(java.util.List<java.lang.String> sentenceWords, java.util.List<java.lang.String> previousSentenceWords)` Fix up a sentence.
    `Logger` | `getLogger()` Get the logger.
    `boolean` | `isClosingPunctuationOnly(java.util.List<java.lang.String> sentenceWords)` Check if sentence contains only closing punctuation.
    `protected boolean` | `isNoun(java.lang.String word)` Check if word is a possible noun.
    `protected boolean` | `isPronoun(java.lang.String word)` Check if word is a possible pronoun.
    `protected boolean` | `isProperNoun(java.lang.String word)` Check if word is a possible proper noun.
    `protected boolean` | `isVerb(java.lang.String word)` Check if word is a possible verb.
    `boolean` | `quoteOnlySentence(java.util.List<java.lang.String> sentenceWords)` Check if sentence contains only a double quote.
    `void` | `setAbbreviations(Abbreviations abbreviations)` Set abbreviations.
    `void` | `setLogger(Logger logger)` Set the logger.
    `void` | `setPartOfSpeechGuesser(PartOfSpeechGuesser partOfSpeechGuesser)` Set the part of speech guesser.
    `void` | `setSentenceSplitterIterator(SentenceSplitterIterator sentenceSplitterIterator)` Set sentence splitter iterator.
    `protected java.util.List<java.util.List<java.lang.String>>` | `splitSentenceWordList(java.util.List<java.lang.String> sentenceWords)` Break sentence word list into subsentences on special marker.
    `protected boolean` | `verbSeen(java.util.List<java.lang.String> tokenList)` See if potential verb found in token list.

    - ### Methods inherited from class edu.northwestern.at.utils.IsCloseableObject

      `close`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

    - ### Methods inherited from interface edu.northwestern.at.utils.IsCloseable

      `close`

- - ### Field Detail

    - #### wordTokenizer

          protected WordTokenizer wordTokenizer

      Default word tokenizer used if none specified.

    - #### partOfSpeechGuesser

          protected PartOfSpeechGuesser partOfSpeechGuesser

      Part of speech guesser used by some sentence splitters.

    - #### sentenceSplitterIterator

          protected SentenceSplitterIterator sentenceSplitterIterator

      Sentence iterator.

    - #### names

          protected Names names

      Name recognizer.

    - #### abbreviations

          protected Abbreviations abbreviations

      Abbreviations.

    - #### logger

          protected Logger logger

      Logger used for output.

    - #### disallowedSentenceStarters

          protected static final java.lang.String disallowedSentenceStarters

      Characters not allowed to start a sentence.
      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### AbstractSentenceSplitter

          public AbstractSentenceSplitter()

  - ### Method Detail

    - #### setPartOfSpeechGuesser

          public void setPartOfSpeechGuesser(PartOfSpeechGuesser partOfSpeechGuesser)

      Set the part of speech guesser.
      Specified by:
        `setPartOfSpeechGuesser` in interface `SentenceSplitter`
      Parameters:
        `partOfSpeechGuesser` - The part of speech guesser.

    - #### setAbbreviations

          public void setAbbreviations(Abbreviations abbreviations)

      Set abbreviations.
      Specified by:
        `setAbbreviations` in interface `SentenceSplitter`
      Parameters:
        `abbreviations` - Abbreviations.

    - #### setSentenceSplitterIterator

          public void setSentenceSplitterIterator(SentenceSplitterIterator sentenceSplitterIterator)

      Set sentence splitter iterator.
      Specified by:
        `setSentenceSplitterIterator` in interface `SentenceSplitter`
      Parameters:
        `sentenceSplitterIterator` - Sentence splitter iterator.

    - #### fixUpSentence

          protected boolean fixUpSentence(java.util.List<java.lang.String> sentenceWords,
                              java.util.List<java.lang.String> previousSentenceWords)

      Fix up a sentence.
      Parameters:
        `sentenceWords` - Sentence to fix up.
        `previousSentenceWords` - Previous sentence.
      Returns:
        true if end of sentence found.

    - #### isClosingPunctuationOnly

          public boolean isClosingPunctuationOnly(java.util.List<java.lang.String> sentenceWords)

      Check if sentence contains only closing punctuation.
      Parameters:
        `sentenceWords` - Words in sentence.
      Returns:
        true if all words are closing punctuation, false otherwise.

    - #### extractSentences

          public java.util.List<java.util.List<java.lang.String>> extractSentences(java.lang.String text,
                                                                          WordTokenizer tokenizer)

      Break text into sentences and tokens.
      Specified by:
        `extractSentences` in interface `SentenceSplitter`
      Parameters:
        `text` - Text to break into sentences and tokens.
        `tokenizer` - Word tokenizer to use for breaking sentences into words.
      Returns:
        List of sentences. Each sentence is itself a list of word tokens.

        Word tokens may be words, numbers, punctuation, etc.

    - #### quoteOnlySentence

          public boolean quoteOnlySentence(java.util.List<java.lang.String> sentenceWords)

      Check if sentence contains only a double quote.
      Parameters:
        `sentenceWords` - List of sentence words.
      Returns:
        true if sentence starts with quote and contains only XML section marker characters.

    - #### extractSentences

          public java.util.List<java.util.List<java.lang.String>> extractSentences(java.lang.String text)

      Break text into sentences and tokens.
      Specified by:
        `extractSentences` in interface `SentenceSplitter`
      Parameters:
        `text` - Text to break into sentences and tokens.
      Returns:
        List of sentences. Each sentence is itself a list of word tokens.

        Word tokens may be words, numbers, punctuation, etc. The default word tokenizer is used.

    - #### findSentenceOffsets

          public int[] findSentenceOffsets(java.lang.String text,
                                  java.util.List<java.util.List<java.lang.String>> sentences)

      Find starting offsets of sentences extracted from a text.
      Specified by:
        `findSentenceOffsets` in interface `SentenceSplitter`
      Parameters:
        `text` - Text from which sentences were extracted.
        `sentences` - List of sentences (each a list of words) extracted from text. N.B. If the sentences aren't from the specified text, the resulting offsets will be meaningless.
      Returns:
        int array of starting offsets in text for each sentence. The first offset starts at 0. There is one more offset than the number of sentences -- the last offset is where the sentence after the last sentence would start.

    - #### addSentenceBad

          protected void addSentenceBad(java.util.List<java.lang.String> sentence,
                            java.util.List<java.util.List<java.lang.String>> sentenceList)

      Add sentence to sentence list.
      Parameters:
        `sentence` - List of words in sentence.
        `sentenceList` - List of sentences.

        The sentence is added to the sentence list after performing any further sentence splitting.

    - #### addSentence

          protected void addSentence(java.util.List<java.lang.String> sentence,
                         java.util.List<java.util.List<java.lang.String>> sentenceList)

      Add sentence to sentence list.
      Parameters:
        `sentence` - List of words in sentence.
        `sentenceList` - List of sentences.

    - #### isVerb

          protected boolean isVerb(java.lang.String word)

      Check if word is a possible verb.
      Parameters:
        `word` - The word to check.

        The check uses the part of speech guesser to get the parts of speech for word. The list of parts of speech is checked for an entry with a major word class of verb.

    - #### isProperNoun

          protected boolean isProperNoun(java.lang.String word)

      Check if word is a possible proper noun.
      Parameters:
        `word` - The word to check.

        The check first looks up the word in the MorphAdorner lists of proper names and places. If the word is not found there, the part of speech guesser is used to get the parts of speech for the word. If the word can be a proper noun, or at least a noun and it begins with a capital letter, the word is assumed to be a possible proper noun.

    - #### isPronoun

          protected boolean isPronoun(java.lang.String word)

      Check if word is a possible pronoun.
      Parameters:
        `word` - The word to check.

    - #### isNoun

          protected boolean isNoun(java.lang.String word)

      Check if word is a possible noun.
      Parameters:
        `word` - The word to check.

    - #### splitSentenceWordList

          protected java.util.List<java.util.List<java.lang.String>> splitSentenceWordList(java.util.List<java.lang.String> sentenceWords)

      Break sentence word list into subsentences on special marker.
      Parameters:
        `sentenceWords` - Sentence words as a list.
      Returns:
        List of lists of sentence words.

        Breaks the word list for a sentence into subsentences based upon the occurrence of the XML surround text marker character. This special character always indicates where a list of words should be split. Also split at a bare period assuming it ends a sentence. We may rejoin some of the split subsentences later into longer sentences.

    - #### verbSeen

          protected boolean verbSeen(java.util.List<java.lang.String> tokenList)

      See if potential verb found in token list.
      Parameters:
        `tokenList` - The token list.
      Returns:
        True if at least one of the tokens in the token list is a potential verb.

    - #### getLogger

          public Logger getLogger()

      Get the logger.
      Specified by:
        `getLogger` in interface `UsesLogger`
      Returns:
        The logger.

    - #### setLogger

          public void setLogger(Logger logger)

      Set the logger.
      Specified by:
        `setLogger` in interface `UsesLogger`
      Parameters:
        `logger` - The logger.