edu.northwestern.at.morphadorner

## Class IDFixerFilter

- java.lang.Object
- - org.xml.sax.helpers.XMLFilterImpl
  - - edu.northwestern.at.utils.xml.ExtendedXMLFilterImpl
    - - edu.northwestern.at.morphadorner.IDFixerFilter

- All Implemented Interfaces:
    org.xml.sax.ContentHandler, org.xml.sax.DTDHandler, org.xml.sax.EntityResolver, org.xml.sax.ErrorHandler, org.xml.sax.XMLFilter, org.xml.sax.XMLReader

      public class IDFixerFilter
      extends ExtendedXMLFilterImpl

  XML filter which updates tag IDs and attributes.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected java.lang.String` | `baseFileName` Base XML file name for generating ID values.
    `protected QueueStack<java.lang.String>` | `divStack` Div tag stack.
    `protected java.lang.String` | `elementURI` URI for elements without one.
    `protected int` | `emittedWordCount` Current number of words emitted.
    `protected java.lang.String` | `facsFromPB` "facs" or "ref" from current element.
    `protected QueueStack<java.lang.String>` | `foreignStack` Foreign language attribute stack.
    `protected int` | `gapCount` Gap count.
    `protected java.lang.String` | `id` Current ID value as string.
    `protected static java.text.NumberFormat` | `ID_FORMATTER` Word ID formatters.
    `protected java.lang.String` | `idAttrName` ID attribute name.
    `protected int` | `idSpacing` ID Spacing.
    `protected MorphAdornerSettings.XMLIDType` | `idType` ID Type.
    `protected boolean` | `isFirstWord` True if we're processing first word in a sentence.
    `protected QueueStack<XMLWriterState>` | `jumpStack` Jump tag stack.
    `protected static java.util.Map<java.lang.String,java.lang.String>` | `languageTags` Map of XML language tags to language name.
    `protected java.lang.String` | `lastID` Previous ID value.
    `protected java.lang.String` | `lastIDString`
    `protected MorphAdornerSettings` | `morphAdornerSettings` MorphAdorner settings.
    `protected java.lang.String` | `nFromPB` "n" from current element.
    `protected boolean` | `outputNonredundantAttributesOnly` True to output non-redundant attributes only.
    `protected boolean` | `outputNonredundantEosAttribute` True to output non-redundant eos attributes only.
    `protected boolean` | `outputNonredundantPartAttribute` True to output non-redundant part attributes only.
    `protected boolean` | `outputNonredundantTokenAttribute` True to output non-redundant token attributes only.
    `protected boolean` | `outputPseudoPageBoundaryMilestones` True to output page boundary milestones.
    `protected boolean` | `outputSentenceBoundaryMilestones` True to output sentence boundary milestones.
    `protected boolean` | `outputWhitespace` True to output whitespace elements.
    `protected boolean` | `outputWordOrdinal` Output word ordinal.
    `protected static java.text.NumberFormat` | `PAGE_FORMATTER` Page number formatter.
    `protected int` | `pageColumn` Column (0-based) within current page based upon repeated facs= value of pb.
    `protected int` | `pageNumber` Running page number.
    `protected PendingElement` | `pendingWordElement` Pending word element.
    `protected PartOfSpeechTags` | `posTags` Part of speech tags used in XML file.
    `protected java.lang.String` | `prevFacsFromPB` "facs" or "ref" from previous element.
    `protected java.util.Set<java.lang.String>` | `pseudoPageContainerDivTypes` Pseudo-page ending div types.
    `protected int` | `pseudoPageCount` Current pseudo page count.
    `protected int` | `pseudoPageSize` Page size in number of tokens.
    `protected boolean` | `pseudoPageStarted` True if pseudo page started.
    `protected int` | `pseudoPageWordCount` Current pseudo page word count.
    `protected XMLSentenceMelder` | `sentenceMelder` XML sentence melder.
    `protected SortedArrayList<SentenceAndWordNumber>` | `sortedWords` Sorted sentence and word number information.
    `protected java.util.Map<java.lang.Integer,java.lang.Integer>` | `splitWords` Split words map of word ID to # of word parts.
    `protected java.util.Map<java.lang.Integer,java.lang.Integer>` | `splitWordsCopy` Copy of split words map.
    `protected boolean` | `tokenizingOnly` True if only doing tokenization.
    `protected int` | `totalWordsToEmit` Total number of words to emit.
    `protected boolean` | `usePCToMarkEndOfSentence` True to use element to mark end of sentence.
    `protected static java.text.NumberFormat` | `WORD_FORMATTER` Word within page formatter.
    `protected int` | `wordNumberWithinPage` Word within page number.
    `protected int` | `wordOrdinal` Word ordinal.
    `protected XMLWriter` | `writer` XML writer.
    `protected java.lang.String` | `xmlTokenLabelAttribute` Token label attribute.
    `protected boolean` | `xmlTokenLabelEmit` Emit token label?
    `protected boolean` | `xmlTokenLabelPrependWorkName` Token label prepend work name flag.
    `protected int` | `xmlTokenLabelSpacing` Token label spacing.
    `protected boolean` | `zzzzljTagSeen` "zzzzlj" tag found.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `IDFixerFilter(org.xml.sax.XMLReader reader, PartOfSpeechTags posTags, java.lang.String outFile, int maxID, SortedArrayList<SentenceAndWordNumber> sortedWords, java.util.Map<java.lang.Integer,java.lang.Integer> splitWords, int totalWords, int totalPageBreaks, MorphAdornerSettings morphAdornerSettings, boolean tokenizingOnly)` Create ID filter.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `characters(char[] ch, int start, int length)` Handle character data.
    `PendingElement` | `createPseudoPageElement(java.lang.String uri, boolean forcedEmit, boolean start, java.lang.String path)` Create a pseudo page milestone.
    `void` | `emitPseudoPageElement(PendingElement pseudoPageElement)` Emit a pseudo page milestone.
    `void` | `emitWordElement(java.lang.String uri, java.lang.String localName, java.lang.String qName, org.xml.sax.helpers.AttributesImpl atts, java.lang.String wordText, boolean allowOutputWhitespace, boolean forceEOS)` Emit a word element.
    `void` | `endElement(java.lang.String uri, java.lang.String localName, java.lang.String qName)` Handle end of an element.
    `protected java.lang.String` | `generateTokenLabel()` Generate token label based upon current pb element attributes.
    `protected static java.lang.String` | `getDisplayableAttributes(org.xml.sax.Attributes atts)` Get displayable list of element attributes and values.
    `java.lang.String` | `getForeignLanguageTag(java.lang.String qName, org.xml.sax.Attributes atts)` Get the foreign language tag for XML element.
    `XMLSentenceMelder` | `getSentenceMelder()` Get associated sentence melder.
    `protected void` | `setIDFormat(java.lang.String outFile, int maxID, int maxPageBreaks)` Set word ID format.
    `void` | `setPosTags(PartOfSpeechTags posTags)` Set the part of speech tags.
    `void` | `setSentenceMelder(XMLSentenceMelder sentenceMelder)` Set associated sentence melder.
    `protected void` | `setSplitWords(java.util.Map<java.lang.Integer,java.lang.Integer> splitWords)` Set split words.
    `void` | `setWriter(XMLWriter writer)` Set associated XML writer.
    `void` | `startElement(java.lang.String uri, java.lang.String localName, java.lang.String qName, org.xml.sax.Attributes atts)` Handle start of an XML element.

    - ### Methods inherited from class edu.northwestern.at.utils.xml.ExtendedXMLFilterImpl

      `removeAttribute, setAttributeValue, setAttributeValue, setAttributeValue`

    - ### Methods inherited from class org.xml.sax.helpers.XMLFilterImpl

      `endDocument, endPrefixMapping, error, fatalError, getContentHandler, getDTDHandler, getEntityResolver, getErrorHandler, getFeature, getParent, getProperty, ignorableWhitespace, notationDecl, parse, parse, processingInstruction, resolveEntity, setContentHandler, setDocumentLocator, setDTDHandler, setEntityResolver, setErrorHandler, setFeature, setParent, setProperty, skippedEntity, startDocument, startPrefixMapping, unparsedEntityDecl, warning`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### ID_FORMATTER

          protected static final java.text.NumberFormat ID_FORMATTER

      Word ID formatters.

    - #### PAGE_FORMATTER

          protected static final java.text.NumberFormat PAGE_FORMATTER

      Page number formatter.

    - #### WORD_FORMATTER

          protected static final java.text.NumberFormat WORD_FORMATTER

      Word within page formatter.

    - #### wordOrdinal

          protected int wordOrdinal

      Word ordinal.

    - #### lastID

          protected java.lang.String lastID

      Previous ID value.

    - #### lastIDString

          protected java.lang.String lastIDString

    - #### id

          protected java.lang.String id

      Current ID value as string.

    - #### idAttrName

          protected java.lang.String idAttrName

      ID attribute name.

    - #### baseFileName

          protected java.lang.String baseFileName

      Base XML file name for generating ID values.

    - #### posTags

          protected PartOfSpeechTags posTags

      Part of speech tags used in XML file.

    - #### elementURI

          protected java.lang.String elementURI

      URI for elements without one.

    - #### outputWhitespace

          protected boolean outputWhitespace

      True to output whitespace elements.

    - #### outputNonredundantAttributesOnly

          protected boolean outputNonredundantAttributesOnly

      True to output non-redundant attributes only.

    - #### outputNonredundantTokenAttribute

          protected boolean outputNonredundantTokenAttribute

      True to output non-redundant token attributes only.

    - #### outputNonredundantPartAttribute

          protected boolean outputNonredundantPartAttribute

      True to output non-redundant part attributes only.

    - #### outputNonredundantEosAttribute

          protected boolean outputNonredundantEosAttribute

      True to output non-redundant eos attributes only.

    - #### outputSentenceBoundaryMilestones

          protected boolean outputSentenceBoundaryMilestones

      True to output sentence boundary milestones.

    - #### usePCToMarkEndOfSentence

          protected boolean usePCToMarkEndOfSentence

      True to use element to mark end of sentence.

    - #### outputPseudoPageBoundaryMilestones

          protected boolean outputPseudoPageBoundaryMilestones

      True to output page boundary milestones.

    - #### pseudoPageSize

          protected int pseudoPageSize

      Page size in number of tokens.

    - #### pseudoPageCount

          protected int pseudoPageCount

      Current pseudo page count.

    - #### pseudoPageWordCount

          protected int pseudoPageWordCount

      Current pseudo page word count.

    - #### pseudoPageStarted

          protected boolean pseudoPageStarted

      True if pseudo page started.

    - #### emittedWordCount

          protected int emittedWordCount

      Current number of words emitted.

    - #### sentenceMelder

          protected XMLSentenceMelder sentenceMelder

      XML sentence melder.

    - #### isFirstWord

          protected boolean isFirstWord

      True if we're processing first word in a sentence.

    - #### pendingWordElement

          protected PendingElement pendingWordElement

      Pending word element.

    - #### splitWords

          protected java.util.Map<java.lang.Integer,java.lang.Integer> splitWords

      Split words map of word ID to # of word parts.

    - #### splitWordsCopy

          protected java.util.Map<java.lang.Integer,java.lang.Integer> splitWordsCopy

      Copy of split words map.

    - #### foreignStack

          protected QueueStack<java.lang.String> foreignStack

      Foreign language attribute stack.

    - #### jumpStack

          protected QueueStack<XMLWriterState> jumpStack

      Jump tag stack. Saves state across jump tags.

    - #### divStack

          protected QueueStack<java.lang.String> divStack

      Div tag stack.

    - #### pseudoPageContainerDivTypes

          protected java.util.Set<java.lang.String> pseudoPageContainerDivTypes

      Pseudo-page ending div types.

    - #### sortedWords

          protected SortedArrayList<SentenceAndWordNumber> sortedWords

      Sorted sentence and word number information.

    - #### writer

          protected XMLWriter writer

      XML writer.

    - #### totalWordsToEmit

          protected int totalWordsToEmit

      Total number of words to emit.

    - #### pageNumber

          protected int pageNumber

      Running page number. Starts at 0.

    - #### wordNumberWithinPage

          protected int wordNumberWithinPage

      Word within page number. Starts at 1.

    - #### nFromPB

          protected java.lang.String nFromPB

      "n" from current element.

    - #### facsFromPB

          protected java.lang.String facsFromPB

      "facs" or "ref" from current element.

    - #### prevFacsFromPB

          protected java.lang.String prevFacsFromPB

      "facs" or "ref" from previous element.

    - #### pageColumn

          protected int pageColumn

      Column (0-based) within current page based upon repeated facs= value of pb.

    - #### idSpacing

          protected int idSpacing

      ID Spacing.

    - #### idType

          protected MorphAdornerSettings.XMLIDType idType

      ID Type.

    - #### xmlTokenLabelEmit

          protected boolean xmlTokenLabelEmit

      Emit token label?

    - #### xmlTokenLabelAttribute

          protected java.lang.String xmlTokenLabelAttribute

      Token label attribute.

    - #### xmlTokenLabelSpacing

          protected int xmlTokenLabelSpacing

      Token label spacing.

    - #### xmlTokenLabelPrependWorkName

          protected boolean xmlTokenLabelPrependWorkName

      Token label prepend work name flag.

    - #### outputWordOrdinal

          protected boolean outputWordOrdinal

      Output word ordinal.

    - #### languageTags

          protected static java.util.Map<java.lang.String,java.lang.String> languageTags

      Map of XML language tags to language name.

    - #### gapCount

          protected int gapCount

      Gap count.

    - #### morphAdornerSettings

          protected MorphAdornerSettings morphAdornerSettings

      MorphAdorner settings.

    - #### tokenizingOnly

          protected boolean tokenizingOnly

      True if only doing tokenization.

    - #### zzzzljTagSeen

          protected boolean zzzzljTagSeen

      "zzzzlj" tag found.

  - ### Constructor Detail

    - #### IDFixerFilter

          public IDFixerFilter(org.xml.sax.XMLReader reader,
                       PartOfSpeechTags posTags,
                       java.lang.String outFile,
                       int maxID,
                       SortedArrayList<SentenceAndWordNumber> sortedWords,
                       java.util.Map<java.lang.Integer,java.lang.Integer> splitWords,
                       int totalWords,
                       int totalPageBreaks,
                       MorphAdornerSettings morphAdornerSettings,
                       boolean tokenizingOnly)

      Create ID filter.
      Parameters:
        `reader` - The XML reader to filter.
        `posTags` - The part of speech tags.
        `outFile` - The output file name.
        `maxID` - The maximum integer word ID.
        `sortedWords` - Sentence and word numbers sorted by word ID.
        `splitWords` - Split words.
        `totalWords` - Total words.
        `totalPageBreaks` - Total page breaks.
        `tokenizingOnly` - True if only emitting tokenization.

  - ### Method Detail

    - #### generateTokenLabel

          protected java.lang.String generateTokenLabel()

      Generate token label based upon current pb element attributes.

    - #### startElement

          public void startElement(java.lang.String uri,
                          java.lang.String localName,
                          java.lang.String qName,
                          org.xml.sax.Attributes atts)
                            throws org.xml.sax.SAXException

      Handle start of an XML element.
      Specified by:
        `startElement` in interface `org.xml.sax.ContentHandler`
      Overrides:
        `startElement` in class `org.xml.sax.helpers.XMLFilterImpl`
      Throws:
        `org.xml.sax.SAXException`

    - #### characters

          public void characters(char[] ch,
                        int start,
                        int length)
                          throws org.xml.sax.SAXException

      Handle character data.
      Specified by:
        `characters` in interface `org.xml.sax.ContentHandler`
      Overrides:
        `characters` in class `org.xml.sax.helpers.XMLFilterImpl`
      Parameters:
        `ch` - Array of characters.
        `start` - The starting position in the array.
        `length` - The number of characters.
      Throws:
        `org.xml.sax.SAXException` - If there is an error.

    - #### emitWordElement

          public void emitWordElement(java.lang.String uri,
                             java.lang.String localName,
                             java.lang.String qName,
                             org.xml.sax.helpers.AttributesImpl atts,
                             java.lang.String wordText,
                             boolean allowOutputWhitespace,
                             boolean forceEOS)
                               throws org.xml.sax.SAXException

      Emit a word element.
      Parameters:
        `uri` - The word element's URI.
        `localName` - The word element's local name.
        `qName` - The word element's qname.
        `atts` - The word element's attributes.
        `wordText` - The word element's text.
        `allowOutputWhitespace` - True to allow outputting whitespace element for word.
        `forceEOS` - True to force end of sentence for this word.
      Throws:
        `org.xml.sax.SAXException`

    - #### endElement

          public void endElement(java.lang.String uri,
                        java.lang.String localName,
                        java.lang.String qName)
                          throws org.xml.sax.SAXException

      Handle end of an element.
      Specified by:
        `endElement` in interface `org.xml.sax.ContentHandler`
      Overrides:
        `endElement` in class `org.xml.sax.helpers.XMLFilterImpl`
      Parameters:
        `uri` - The XML element's URI.
        `localName` - The XML element's local name.
        `qName` - The XML element's qname.
      Throws:
        `org.xml.sax.SAXException`

    - #### createPseudoPageElement

          public PendingElement createPseudoPageElement(java.lang.String uri,
                                               boolean forcedEmit,
                                               boolean start,
                                               java.lang.String path)

      Create a pseudo page milestone.
      Parameters:
        `uri` - Element URI.
        `forcedEmit` - Emit pseudo page milestone even if not enough words accumulated, as long as at least one word in current block.
        `start` - true if starting milestone, false if ending.
        `path` - Path attribute. May be null.
      Returns:
        The pseudo page element.

    - #### emitPseudoPageElement

          public void emitPseudoPageElement(PendingElement pseudoPageElement)

      Emit a pseudo page milestone.
      Parameters:
        `pseudoPageElement` - The pseudo page element to emit.

    - #### setPosTags

          public void setPosTags(PartOfSpeechTags posTags)

      Set the part of speech tags.
      Parameters:
        `posTags` - The part of speech tags.

    - #### setSplitWords

          protected void setSplitWords(java.util.Map<java.lang.Integer,java.lang.Integer> splitWords)

      Set split words.
      Parameters:
        `splitWords` - Map of split words.

    - #### setIDFormat

          protected void setIDFormat(java.lang.String outFile,
                         int maxID,
                         int maxPageBreaks)

      Set word ID format.
      Parameters:
        `outFile` - Output file name used to derive word IDs.
        `maxID` - Maximum integer word ID value.
        `maxPageBreaks` - Maximum number of page breaks.

    - #### setWriter

          public void setWriter(XMLWriter writer)

      Set associated XML writer.
      Parameters:
        `writer` - XML writer.

    - #### setSentenceMelder

          public void setSentenceMelder(XMLSentenceMelder sentenceMelder)

      Set associated sentence melder.
      Parameters:
        `sentenceMelder` - Sentence melder.

    - #### getSentenceMelder

          public XMLSentenceMelder getSentenceMelder()

      Get associated sentence melder.
      Returns:
        Sentence melder.

    - #### getForeignLanguageTag

          public java.lang.String getForeignLanguageTag(java.lang.String qName,
                                               org.xml.sax.Attributes atts)

      Get the foreign language tag for XML element.
      Parameters:
        `qName` - XML element name.
        `atts` - XML element attributes.
      Returns:
        The language part of speech tag for this element.

    - #### getDisplayableAttributes

          protected static java.lang.String getDisplayableAttributes(org.xml.sax.Attributes atts)

      Get displayable list of element attributes and values.
      Parameters:
        `atts` - Attributes.
      Returns:
        Displayable string in name=value format.