edu.northwestern.at.morphadorner.corpuslinguistics.textsegmenter.texttiling

## Class TextTiling

- java.lang.Object
- - edu.northwestern.at.morphadorner.corpuslinguistics.textsegmenter.texttiling.TextTiling

-     public class TextTiling
      extends java.lang.Object

  An implementation of Marti Hearst's text tiling algorithm.
  Author:
    Freddy Choi, Philip R. Burns. Modified for integration in MorphAdorner.

    Use of this code is free for academic, education, research and other non-profit making uses only.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected RawText` | `C` Collection for segmentation.
    `protected double[]` | `depth_score` Depth scores.
    `protected static java.io.PrintStream` | `printStream` Wrapper for printStream to allow utf-8 output.
    `protected int` | `s` Step size.
    `protected StopWords` | `S` Stopwords for noise reduction.
    `protected java.util.List<java.lang.Integer>` | `segmentation` Segment boundaries.
    `protected double[]` | `sim_score` Similarity scores and the corresponding locations.
    `protected int[]` | `site_loc`
    `protected java.util.Map<java.lang.String,java.lang.String>` | `stemOf` Token -> stem dictionary
    `protected int` | `w` Size of sliding window.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `TextTiling()` Create text tiler.
    `TextTiling(RawText c, StopWords s)` Create text tiler from text collection and stop word set.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected void` | `blockAdd(java.lang.String term, java.util.Map<java.lang.String,java.lang.Integer> B)` Add a term to a block Creation date: (07/12/99 01:41:24)
    `protected double` | `blockCosine(java.util.Map<java.lang.String,java.lang.Integer> B1, java.util.Map<java.lang.String,java.lang.Integer> B2)` Compute the cosine similarity measure for two blocks Creation date: (07/12/99 01:49:16)
    `protected void` | `blockRemove(java.lang.String term, java.util.Map<java.lang.String,java.lang.Integer> B)` Remove a term from the block.
    `void` | `boundaryIdentification()` Identify the boundaries
    `void` | `depthScore()` Compute depth score after applying similarityDetermination()
    `java.util.List<java.lang.Integer>` | `getSegmentation()` Return segmentation list.
    `java.util.Map<java.lang.String,java.lang.String>` | `getStemOf()` Return stem dictionary.
    `StopWords` | `getStopWords()` Return stop words.
    `protected boolean` | `include(int i)` Decide whether word i is worth using as feature for segmentation.
    `protected void` | `preprocess()` Perform some preprocessing to save execution time
    `void` | `setStepSize(int stepSize)`
    `void` | `setWindowSize(int windowSize)`
    `void` | `similarityDetermination()` Compute the similarity score.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### printStream

          protected static java.io.PrintStream printStream

      Wrapper for printStream to allow utf-8 output.

    - #### w

          protected int w

      Size of sliding window.

    - #### s

          protected int s

      Step size.

    - #### C

          protected RawText C

      Collection for segmentation.

    - #### S

          protected StopWords S

      Stopwords for noise reduction.

    - #### stemOf

          protected java.util.Map<java.lang.String,java.lang.String> stemOf

      Token -> stem dictionary

    - #### sim_score

          protected double[] sim_score

      Similarity scores and the corresponding locations.

    - #### site_loc

          protected int[] site_loc

    - #### depth_score

          protected double[] depth_score

      Depth scores.

    - #### segmentation

          protected java.util.List<java.lang.Integer> segmentation

      Segment boundaries.

  - ### Constructor Detail

    - #### TextTiling

          public TextTiling()

      Create text tiler.

    - #### TextTiling

          public TextTiling(RawText c,
                    StopWords s)

      Create text tiler from text collection and stop word set.
      Parameters:
        `c` - Text collection to segment.
        `s` - Stop word set.

  - ### Method Detail

    - #### getSegmentation

          public java.util.List<java.lang.Integer> getSegmentation()

      Return segmentation list.
      Returns:
        segmentation as a list of integer sentence indices.

    - #### getStemOf

          public java.util.Map<java.lang.String,java.lang.String> getStemOf()

      Return stem dictionary.
      Returns:
        stem dictionary.

    - #### getStopWords

          public StopWords getStopWords()

      Return stop words.
      Returns:
        Stop words.

    - #### blockAdd

          protected void blockAdd(java.lang.String term,
                      java.util.Map<java.lang.String,java.lang.Integer> B)

      Add a term to a block Creation date: (07/12/99 01:41:24)
      Parameters:
        `term` - java.lang.String
        `B` - java.util.HashMap

    - #### setWindowSize

          public void setWindowSize(int windowSize)

    - #### setStepSize

          public void setStepSize(int stepSize)

    - #### blockCosine

          protected double blockCosine(java.util.Map<java.lang.String,java.lang.Integer> B1,
                           java.util.Map<java.lang.String,java.lang.Integer> B2)

      Compute the cosine similarity measure for two blocks Creation date: (07/12/99 01:49:16)
      Parameters:
        `B1` - java.util.HashMap
        `B2` - java.util.HashMap
      Returns:
        double

    - #### blockRemove

          protected void blockRemove(java.lang.String term,
                         java.util.Map<java.lang.String,java.lang.Integer> B)

      Remove a term from the block.
      Parameters:
        `term` - java.lang.String
        `B` - java.util.HashMap

    - #### boundaryIdentification

          public void boundaryIdentification()

      Identify the boundaries

    - #### depthScore

          public void depthScore()

      Compute depth score after applying similarityDetermination()

    - #### include

          protected boolean include(int i)

      Decide whether word i is worth using as feature for segmentation.
      Parameters:
        `i` - int
      Returns:
        boolean

    - #### preprocess

          protected void preprocess()

      Perform some preprocessing to save execution time

    - #### similarityDetermination

          public void similarityDetermination()

      Compute the similarity score.