edu.northwestern.at.morphadorner.tools.punktabbreviationdetector

Class PunktToken

java.lang.Object

edu.northwestern.at.morphadorner.tools.punktabbreviationdetector.PunktToken

public class PunktToken
extends java.lang.Object

A token for use by the Punkt abbreviation detection algorithm.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	startPosition The starting position of the token.
protected java.lang.String	tokenText The text of the token.
protected PunktTokenType	tokenType The Punkt token type.

Constructor Summary

Constructors

Constructor and Description
PunktToken(java.lang.String tokenText, int start, PunktTokenType tokenType) Create a Punkt token.
PunktToken(java.lang.String tokenText, PunktTokenType tokenType) Create a Punkt token.

Method Summary

Methods

Modifier and Type	Method and Description
int	getEndPosition() Get the ending position of the token.
int	getLength() Get the length of the token.
java.lang.String	getSingleCharCode() Get the character code for a single character token.
int	getStartPosition() Get the starting position of the token.
java.lang.String	getTokenText() Get the token text.
PunktTokenType	getTokenType() Get the token type.
java.lang.String	toString() Return token as a string.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

tokenText

protected java.lang.String tokenText

The text of the token.

startPosition

protected int startPosition

The starting position of the token.

tokenType

protected PunktTokenType tokenType

The Punkt token type.

Constructor Detail

PunktToken

public PunktToken(java.lang.String tokenText,
          int start,
          PunktTokenType tokenType)

Create a Punkt token.

Parameters:

tokenText - The text of the token.

start - Starting position of the token.

tokenType - The type of the token.

PunktToken

public PunktToken(java.lang.String tokenText,
          PunktTokenType tokenType)

Create a Punkt token.

Parameters:

tokenText - The text of the token.

tokenType - The type of the token.

Method Detail

getTokenText

public java.lang.String getTokenText()

Get the token text.

Returns:

The token text.

getTokenType

public PunktTokenType getTokenType()

Get the token type.

Returns:

The token type.

getStartPosition

public int getStartPosition()

Get the starting position of the token.

Returns:

The starting position of the token.

getEndPosition

public int getEndPosition()

Get the ending position of the token.

Returns:

The ending position of the token.

getSingleCharCode

public java.lang.String getSingleCharCode()

Get the character code for a single character token.

Returns:

Unicode string for character. Null if token is not a single character.

getLength

public int getLength()

Get the length of the token.

Returns:

The length of the token.

toString

public java.lang.String toString()

Return token as a string.

Overrides:

toString in class java.lang.Object

Returns:

The token string.