Parsingslides.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Parsing programs\n",
    "\n",
    "_Hannes Vlaardingerbroek, ETCBC_\n",
    "\n",
    "hannes@vlaardingerbroek.nl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Data from many sources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Different file formats, transcriptions etc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In ETCBC: PIL files, AT/AN files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- outside: Word Documents, compact text databases (SEDRA III), less compact databases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Transcriptions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "![tr_table](imgs2/tr_table.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "ETCBC Parsing programs\n",
    " \n",
    " - ETCBC has internal programs for parsing the data\n",
    "\n",
    " - For use in Python, need to parse output of those parsers\n",
    "\n",
    " - Or instead, make Python parsers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "PIL parser\n",
    "- to generate running text of Peshitta edition (removes variants and comments)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```\n",
    "@Ex2\n",
    " 1 w'z#l gbr' mn dbyt lwy^ wns#b lbrt lwy=.;\n",
    " 2 wbTn#t 'ntt' wyldt br'=. wHzth d$pyr h#w=. wT$yth tlt' yr\"Hyn=.;\n",
    " 3 wl' '$k#Ht twb lmT$ywth=. wnsbt lh^ qbwt' [dqys'/+9b1, 9l2, 10j1, 11l2,\n",
    "   12a1fam, 12b1, 12b2, L, M, U] d`rq'=. w$`th^ bkwpr' wbzpt'=. wsm#t bh^\n",
    "   Tly' [l-/+12a1, L, M, U] =.  wsmth^ brqq' `l spth dnhr'=.;\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```\n",
    "@Ex2\n",
    " 1 w'z#l gbr' mn dbyt lwy^ wns#b lbrt lwy=.;\n",
    " 2 wbTn#t 'ntt' wyldt br'=. wHzth d$pyr h#w=. wT$yth tlt' yr\"Hyn=.;\n",
    " 3 wl' '$k#Ht twb lmT$ywth=. wnsbt lh^ qbwt' d`rq'=. w$`th^ bkwpr' wbzpt'=. wsm#t bh^\n",
    "   Tly' =.  wsmth^ brqq' `l spth dnhr'=.;\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- AN parser\n",
    "  - to use morpholically encoded ETCBC texts as input for morphological analyzer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```\n",
    "  0,1 TWB                       TWB\n",
    "  0,1 KTB>                      KTB=/~>\n",
    "  0,1 DNM\"WS>                   D-NMWS/(J~>\n",
    "  0,1 D>TR\"WT>                  D->TR/&WT=~>\n",
    "  1,1 MN                        MN\n",
    "  1,1 QDM                       QDM\n",
    "  1,1 JWM\"T>                    JWM/T=~>\n",
    "  1,1 <LJN                      <L=[/JN\n",
    "  1,1 HWJN                      HWJ[N\n",
    "  1,1 LMS<R                     L-!M!S<R=[/\n",
    "  1,1 LCMCGRM                   L-CMCGRM/\n",
    "  1,1 >XWN                      >X/&W-N\n",
    "  1,1 W>T>                      W->T(J&>[\n",
    "  1,1 >CKXN                     ]>]CKX[-N\n",
    "  1,1 TMN                       TMN\n",
    "  1,1 BRDJYN                    BRDJYN/\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```\n",
    "0,1\tD>TR\"WT>\tD->TR/&WT=~>\n",
    "    D\n",
    "\tmorphemes: (('lex', 'D'),)\n",
    "\tfunctions: (('nu', False), ('gn', False), ('st', False), ('vt', False), ('vs', False), ('ps', False), ('sp', 'prep'), ('ls', 'pcon'))\n",
    "\tlex      : ('7789', (('sp', 'prep'), ('ls', 'pcon'), ('gl', '(relative)')))\n",
    "    >TR/&WT=~>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- SyrNT (parsing NT file shipped with Syromorph)\n",
    "- SedraIII (parsing relational DB SEDRA III)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import syrnt, sedra\n",
    "nt1 = sedra.BFBS(sedra.tosyr).words()\n",
    "nt2 = syrnt.SyrNT(syrnt.tosyr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "w = next(nt2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Annotation(stem='ܟܬܒܐ', lexeme='ܟܬܒܐ', root='ܟܬܒ', prefix='', suffix='', seyame=0, verbal_conjugation=0, aspect=0, state=3, number=1, person=0, gender=2, pronoun_type=0, demonstrative_category=0, noun_type=2, numeral_type=0, participle_type=0, grammatical_category=2, suffix_contraction=0, suffix_gender=0, suffix_person=0, suffix_number=0, feminine_he_dot=0)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w.annotation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## MorphAn\n",
    "\n",
    "### Morphological analyzer based on transformations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Purpose: to recognize morphological patterns, and predict the probabilities of several possible taggings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training text:\n",
    "\n",
    "Used Syriac New Testament with annotations containing: tags (lexeme, PoS, GNP, etc)\n",
    "\n",
    "Calculate transformation patterns between text form and lexeme form"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}