In Brief.  “In December of 2000, a new consortium was established to sustain and develop the Text Encoding Initiative (TEI). Initially launched in 1987, the TEI is an international and interdisciplinary standard that helps libraries, museums, publishers, and individual scholars represent all kinds of literary and linguistic texts for online research and teaching, using an encoding scheme that is maximally expressive and minimally obsolescent.” [http://www.tei-c.org/Consortium/index.html]

For a full explanation of the history, objectives and approaches of the Text Encoding Initiative it would be best if you visited them at http://www.tei-c.org/. There you will find links and information to all sorts of TEI related stuff. Of note are the Guidelines.

The TEI consortium provides an approach to transcribing and creating documents in SGML (that was the old version of the TEI Guidelines, up to P3) and XML (from P4 on), is as obsolescence proof as possible and allows free exchange of content and ideas. We will only be using a very small subset of this system's capacity.

In these sections we are going to step through a basic introduction to building a marked up dictionary based on the Text Encoding Initiative TEI2 P4 Dictionary Document Type Definition. First we deal with some technical information. If you are familiar with XML and DTDs, you might want to skip the next sections.

The TEI system of markup can be approached in a number of ways. Here we treat it pretty much like any other XML based markup system. The consortium itself has not registered a formal XML name space (as of Feb 2004), but does have unofficial Public and System identifiers we may use. If you don't know what all that means, don't worry. Cut and paste examples are given below. While, in the following "Walking with" section there are some basic explanations of what this all means. For a deeper understanding a visit to the W3C XML pages may help. OASIS and associated XML portals usually have technical and introductory matter as well.

http://www.w3.org/

http://xml.coverpages.org/

http://www.xml.com

<?xml version='1.0' encoding="UTF-8" ?>

The xml declaration is required on all valid xml documents. For our purposes the above is all we need. Be aware though, that this declaration can contain much more than we use here. The encoding="UTF-8" attribute and value pair is sometimes considered optional as all XML documents assume or default to this unless told otherwise. I believe it is better to add it now as future versions of XML will probably require it.

It is the next section that defines our Text Encoding Initiative format in particular. So far we have just told a validating parser that this document should be valid XML version 1.0.

<!DOCTYPE TEI.2 PUBLIC "-//TEI P4//DTD Main Document Type//EN"
"http://www.tei-c.org/Guidelines/DTD/tei2.dtd" [
 <!ENTITY % TEI.XML          "INCLUDE" >
 <!ENTITY % TEI.dictionaries "INCLUDE" >
      ]>

The above is called the DOCTYPE declaration. The UPPER case words are all keywords that are used to set variables and types in parsing and validating tools. The rest of the definition is very carefully structured to provide absolute definitions for a number of things. We step through them here, as they are understood by the protocols involved. The entire set between the opening <! and the closing > is called the document prolog or internal DTD subset. Functionally it defines or even redefines a specific named document type.

PUBLIC "-//TEI P4//DTD Main Document Type//EN"

This "phrase" is called the Formal Public Identifier. It starts with the word PUBLIC and in this case ends with a language identifier. The quoting is important as is the string -// at the start of the actual definition.

This means that this is an informal definition, or an unregistered definition. A registered definition would start start with +//. Most Formal Public Identifiers use this type of definition, there is nothing unusual or pejorative about it.

The FPI must be exactly as shown or it means “something else”, and probably not what we need it to. It is also one of the ways your local system knows which DTD to compare your document to, and so must be correct.

	Note
	Text Encoding Initiative documents also use other FPIs! If you visit their website you will see many other examples. The TEI2 zip file currently has HTML documentation explaining all those options, as that is likely to be up to date please refer to it rather than this for usages other than FreeDict. Please see the section called “Finding and downloading the DTDs” if you haven't already :)

<!DOCTYPE TEI.2 PUBLIC "-//TEI P4//DTD Main Document Type//EN"
"/home/your-account/TEI/TEI2.dtd" [
 <!ENTITY % TEI.XML          "INCLUDE" >
 <!ENTITY % TEI.dictionaries "INCLUDE" >
	]>

Headers usually give meta information, ie. information about something. The TEI header gives general information about the following text like title, authors, publisher and revision history. It is defined in The TEI Guidelines, Chapter 5.

The TEI header is used to created the 00-database-info entry in the dictd database files, as well as to convert it to meta information for other supported dictionary formats. The contents of the title element go into the 00-database-short entry; the content of the sourceDesc element goes into the 00-database-url entry.

In the following you find a header taken from the kha-deu.tei file. You can copy it and replace element contents as they fit your database.

  <teiHeader>
    <fileDesc>
      <titleStmt>
	<title>Khasi-German FreeDict Dictionary</title>
	<respStmt>
	  <resp>Maintainer</resp>
	  <name>Michael Bunk &lt;micha@luetzschena.de></name>
	</respStmt>
      </titleStmt>
      <editionStmt>
	<edition>0.1</edition>
      </editionStmt>
      <extent>about 1000 headwords</extent>
      <publicationStmt>
	<publisher>FreeDict</publisher>
	<availability>
	  <p>GNU GENERAL PUBLIC LICENSE</p>
	</availability>
	<date>2002-2004</date>
	<pubPlace>http://freedict.org/</pubPlace>
      </publicationStmt>
      <seriesStmt>
	<title>free dictionaries</title>
      </seriesStmt>
      <notesStmt>
	<note>Thanks go to the supporters of this project:
	  Karl-Heinz Grüßner, University of Tübingen; Rebekah Tham,
	  CIEFL, Shillong; Brother Sngi, Sacred Heart College,
	  Shillong; University of Leipzig, Depts of Indology and
	  Computer Science; Rik Faith for dictd server &amp;
	  protocol; TEI for their wonderful guidelines; Horst
	  Eyermann for his generous freedict project; the open source
	  community: thrust for trust &amp; development</note>
      </notesStmt>
      <sourceDesc>
	<p>Home: <xptr url="http://freedict.org/"/></p>
      </sourceDesc>
    </fileDesc>
    <encodingDesc>
      <projectDesc>
	<p>This dictionary comes to you through nice people making it
	  available for free and for good. It is part of the FreeDict project,
	  http://www.freedict.org / http://freedict.de. This
	  project aims to make available many translating dictionaries
	  for free. Your contributions are welcome!</p>
      </projectDesc>
    </encodingDesc>
    <revisionDesc>
      <change>
	<date>2005-Nov-24</date>
	<respStmt>
	  <name>Michael Bunk</name>
	</respStmt>
	<item>Note that this version is quite final. I concentrate on
	  the Khasi-English dictionary now.</item>
      </change>
      <change>
	<date>2003-05-01</date>
	<respStmt>
	  <name>Michael Bunk</name>
	</respStmt>
	<item>Updated some things in XML header information</item>
      </change>
      <change>
	<date>2002-02-18</date>
	<respStmt>
	  <name>Michael Bunk</name>
	</respStmt>
	<item>First Draft</item>
      </change>
      <change>
	<date>before February 2002</date>
	<respStmt>
	  <name>Karl-Heinz Grüßner</name>
	</respStmt>
	<item>compilation of wordlists</item>
      </change>
    </revisionDesc>
  </teiHeader>

That header is transformed to this 00-database-info text:

Khasi-German FreeDict Dictionary

Maintainer: Michael Bunk <micha@luetzschena.de>

Edition: 0.1
Size: about 1000 headwords

Published by: FreeDict, 2002-2004
at: http://freedict.org/

Availability:

  GNU GENERAL PUBLIC LICENSE

Series: free dictionaries

Notes:

 * Thanks go to the supporters of this project: Karl-Heinz Grüßner,
   University of Tübingen; Rebekah Tham, CIEFL, Shillong; Brother Sngi,
   Sacred Heart College, Shillong; University of Leipzig, Depts of
   Indology and Computer Science; Rik Faith for dictd server & protocol;
   TEI for their wonderful guidelines; Horst Eyermann for his generous
   freedict project; the open source community: thrust for trust &
   development

Source(s):

  Home: http://freedict.org/

The Project:

  This dictionary comes to you through nice people making it available for
  free and for good. It is part of the FreeDict project,
  http://www.freedict.org / http://freedict.de. This project aims to make
  available many translating dictionaries for free. Your contributions are
  welcome!

Changelog:

 * 2007-02-02 $Id: teiheader2txt.xsl,v 1.6 2006/05/21 12:57:18 micha137 Exp $:
   Converted TEI file into text format

 * 2005-Nov-24 Michael Bunk:
   Note that this version is quite final. I concentrate on the
   Khasi-English dictionary now.

 * 2003-05-01 Michael Bunk:
   Updated some things in XML header information

 * 2002-02-18 Michael Bunk:
   First Draft

 * before February 2002 Karl-Heinz Grüßner:
   compilation of wordlists
    

<entry>
  <form><orth>dog</orth></form>
  <trans><tr>Hund</tr></trans>
</entry>

dog
    Hund

<entry>
  <form>
    <orth>dog</orth>
    <pron>dɔg</pron>
  </form>
  <gramGrp><pos>n</pos></gramGrp>
  <sense>
    <trans>
      <tr>Hund</tr><gen>m</gen>
    </trans>
    <eg>
      <q>The dog is barking.</q>
      <trans><tr>Der Hund bellt.</tr></trans>
    </eg>
    <note>Dogs bite as well.</note>
  </sense>
</entry>

dog [dɔg] n.
  Hund m.
  "The dog is barking." = "Der Hund bellt."
  (Dogs bite as well.)

orth gives the orthography, ie. how the word is correctly written. If there are several ways, you can use multiple orth elements. pron gives the pronunciation (sic!), ie how the word is spoken. In FreeDict we use the IPA (International Phonetic Alphabet) that is also contained in Unicode. The FreeDict-Editor might help you in entering it here. In a gramGrp element, all grammatical information is grouped. You can give the part of speech (here n for noun), the gender and also the number (singular, plural) of the headword (Number is not given in this example). The values allowed for part-of-speech are prescribed in this document, so that one doesn't write n and the other one noun and the third one N or whatever. See Table 5.1, “Part of Speech Typology (recommended contents of the pos element)”.

You can group the different senses of homographs with sense. The numbering is optional. Translations are given in the tr element. Multiple tr elements may be given. For each grammatical information is optional.

With eg (exempla grata) you can give examples of usage and optionally their translation.

    <entry>
      <form>
	<orth>ban</orth>
      </form>
      <gramGrp>
	<pos>prep</pos>
      </gramGrp>
      <sense>
	<trans><tr>to</tr></trans>
	<def>denotes infinitive of the following verb</def>
	<eg><q>U nang ban thoh.</q><trans><tr>Come here!</tr></trans></eg>
      </sense>
    </entry>

     <entry>
      <form>
	<orth>pynhiar</orth>
      </form>
      <gramGrp>
	<pos>v</pos>
      </gramGrp>
      <sense>
	<trans><tr>abase</tr></trans>
	<xr type="syn"><ref>pynrit</ref></xr>
      </sense>
    </entry>

    <entry>
      <form>
	<orth>pungkjat</orth>
      </form>
      <gramGrp>
	<pos>n</pos><gen>f</gen>
      </gramGrp>
      <sense>
	<usg type="dom">bio</usg>
	<trans><tr>leg</tr></trans>
      </sense>
    </entry>

Please refer to the TEI Guidelines and take a look at the XML markup of a dictionary for more examples.

You can use the FreeDict Dictionary TEI XML Template as a starting point for your own dictionaries. You can download it from here.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE TEI.2
 PUBLIC "-//TEI P4//DTD Main Document Type//EN"
 "http://www.tei-c.org/Guidelines/DTD/tei2.dtd" [
<!ENTITY % TEI.dictionaries "INCLUDE">
<!ENTITY % TEI.XML "INCLUDE">
<!ENTITY % TEI.linking "INCLUDE">
<!ATTLIST xptr url CDATA #IMPLIED>
<!ATTLIST xref url CDATA #IMPLIED>
]>
<TEI.2>
  <teiHeader>
    <fileDesc>
      <titleStmt>
	<title>Language1-Language2 FreeDict Dictionary</title>
	<respStmt>
	  <resp>Author</resp>
	  <name>Someone Somewho &lt;her@email.address&gt;</name>
	</respStmt>
	<respStmt>
	  <resp>Maintainer</resp>
	  <name>Someone Else &lt;or@the-same.example.com&gt;</name>
	</respStmt>
      </titleStmt>
      <editionStmt>
	<edition>0.1</edition>
      </editionStmt>
      <extent>less than 1000 headwords</extent>
      <publicationStmt>
	<publisher>FreeDict</publisher>
	<availability>
	  <p>GNU GENERAL PUBLIC LICENSE</p>
	</availability>
	<date>200X</date>
	<pubPlace>http://freedict.org/</pubPlace>
      </publicationStmt>
      <seriesStmt>
	<title>free dictionaries</title>
      </seriesStmt>
     <notesStmt>
       <note type="status">new dictionary project</note>
       <note>Thanks to ...</note>
     </notesStmt>
     <sourceDesc>
       <p>Home: <xptr url="http://www.somewhere.in.cspace/"/></p>
      </sourceDesc>
    </fileDesc>
    <encodingDesc>
        <projectDesc>
          <p>This dictionary comes to you through nice people making it
          available for free and for good. It might be part of the FreeDict project,
          http://www.freedict.org / http://freedict.de. This
          project aims to make available many translating dictionaries
          for free. Your contributions are welcome!</p>
      </projectDesc>
    </encodingDesc>
   <revisionDesc>
       <change>
	<date>200x-mm-dd</date>
	<respStmt><name>Someone</name></respStmt>
	<item>Start</item>
      </change>
      <change>
	<date>2005-04-02</date>
	<respStmt><name>Peter Gossner / Michael Bunk</name></respStmt>
	<item>FreeDict Dictionary TEI XML Template V0.1</item>
      </change>
    </revisionDesc>
  </teiHeader>

  <text>
    <body>

      <!-- Sample Entry -->
      <entry>
        <form>
	  <orth>A</orth>
	  <orth>a</orth>
	</form>
	<gramGrp>
	  <pos>n</pos>
	  <gen>m</gen>
	</gramGrp>
        <sense>
	  <def>the first letter of the Language1 Alphabet</def>
	</sense>
      </entry>
      
    </body>
  </text>
</TEI.2>

As dictionaries are used as authoritative sources for people looking up the spelling of words or learning foreign languages, it is important that they maintain a high quality standard.

The following quality criteria are important: