开放语言典藏组织OLAC与语言典藏后设之标准课件

1、開放語言典藏組織（OLAC）與語言典藏後設資料之標準黃居仁、張如瑩1Outline Introduction to OLACDublin Core & OAIOLAC StandardsOLAC Metadata Set OLAC and Asian LanguagesExamplesSome Relative Web SiteOLAC Launch2The Open Language Archives Community 3OLAC AimsOLAC, the Open Language Archives Community, is an international partnership

2、of institutions and individuals who are creating a worldwide virtual library of language resources by:developing consensus on best current practice for the digital archiving of language resources;developing a network of interoperating repositories and services for housing and accessing such resource

3、s.4OLAC OrganizationCoordinators: Steven Bird & Gary SimonsAdvisory Board: Helen Aristar Dry, Susan Hockey, Chu-Ren Huang, Mark Liberman, Brian MacWhinney, Michael Nelson, Nicholas Ostler, Henry Thompson, Hans Uszkoreit, Antonio ZampolliParticipating Archives & Services: LDC, ELRA, DFKI, CBOLD, ANLC

4、, LACITO, Perseus, SIL, APS, UtrechtProspective Participants: ASEDA, Academia Sinica, AISRI, INALF, LCAAJ, Linguist, MPI, NAA, OTA, Rosetta, Tibetan Digital Library (UVA) Individual Members: 1205Introduction to OLAC許多協會需要語言資源，如：語言學家、工程師、教師、演說家許多機構提供片段性的架構，如：檔案管理員、軟體發展者和出版者。前所未有的契機：延伸性標誌語言（Extensible

5、 Markup Language,XML）和 Unicode題供以結構化方式彈性呈現以及長期儲存資料。線上或非線上的數位化出版品有效且實際上達到分享語言資源涵義Dublin Core 後設資料集（資源分類標準模組）連同Open Archives Initiative所提供的交換方法，可建立一個跨越多個儲存器與檔案櫃的架構。6The Vision for an Open Language Archives Community使用者透過一個OLAC的服務題供者網站搜尋與呈現OLAC的metadata欄位。7The Vision for an Open Language Archives Commu

6、nity#2理論上使用者可取得任何需要的資源DATA：任何描述語言的相關資訊。問卷結果：25%數位化，但並未採用相同的後設資料欄位。TOOLS：有助於創造、瀏覽、查詢或使用語言資料的計算機資源。ADVICE：什麼資源是可靠的？什麼工具適用於此情境？創造新資料時該如何作？8The Vision for an Open Language Archives Community #3實際上無法得到想要的資源在不同網站擁有不同名字（Name）造成召回率低 (low recall).在其他領域有相同意義，造成正確率低（precision）.是否運用適當軟體以及判斷ADVICE的價值？許多語言資源並非以文字

7、為基礎。語言資源散佈在不同的網站.9The Vision for an Open Language Archives Community Bridging the gap through community infrastructure Gateway：使用者可獲得data,tool,advice的單一入口網站。Metadata： data,tool,advice的統一描述，包含所有項目的連結以及解釋如何存取。Review：瀏覽 data,tool,advice的評價。Standards：上述各項過程與協定的基礎，例如：metadata schema,harvesting protocol.1

8、0The Vision for an Open Language Archives Community Summary: Seven Layers Complete the BridgeCONVERTCREATECREATEEXPORTDELIVERFORMAT OAICONTENTMETADATAOLAC REPOSITORIESOLAC SERVICESUSER SERVICESOLACPROCOLAC MHP OAI MS DCSoftwareRecommendationsInitiativesStandards11Dublin Core Metadata Initiative起於199

9、5挖掘web資源的一個會議 /Dublin Core後設資料元素一個廣泛跨學科的核心元素,有效廣泛支援資源挖掘,適用於任何以數位化或傳統型態存在的資源描述.包含十五個可任選與重複的元素(elements): Title, Creator, Subject, Description, Publisher, Contributor, Date,Type, Format, Identifier, Source, Language, Relation, Coverage and Rights.2002/01/07-以RDF/XML呈現: http:/dublin

10、/documents/2001/11/28/dcmes-xml/12The Open Archives Initiative #11999/10成立,一般性的跨電子印刷品的檔案櫃(Archives)架構,不論是哪一種學術性媒材的數位儲存器(repositories)OAI基礎建設必須有的兩個標準:OAI Shared Metadata Set (Dublin Core): 使內部跨儲存器運作容易.OAI Metadata Harvesting Protocol: http協定下使用軟體查詢儲存器.13The Open Archives Initiative #2The Rela

11、tionship Between an OAI Repository and an Archive14Applying the OAI to Language ResourceOAI特色透過單一介面以metadata為基礎搜尋各data provider.Web分散式與由下而上的特色集中式資料庫結構化的本質適合使用者獲取成長迅速的資源和大量使用者導向的資源描述.支援以Dublin Code延伸的後設資料(metadata). 收集meta-archives在單一地方,使用者同時搜尋多個檔案館.OAI的ArchiveOAI的SERVICEPROVIDER15The Open Language A

12、rchives Community2000年十二月在workshop on Web-Based Language Documentation and Description由來自北美、南美、歐洲、非洲、中東、亞洲、澳洲的語言學家與軟體發展者所創。OLAC gateway:/16Foundation: OLAC & OAIRecall: OAI data providers must support:Dublin Core MetadataOAI Metadata harvesting protocolBUT: OAI data providers can support:a more spec

13、ialized metadata formata more specialized harvesting protocolWhat OLAC does:specialized metadata for language resourcesspecialized harvesting (extra validation)17OLAC StandardsAside:standards = the protocols and interfaces that allow the community to functionrecommendations = standards for represent

14、ing linguistic contentOLAC has three primary standards:OLACMS: the OLAC Metadata Set (Qualified DC)OLAC MHP: refinements to the OAI protocolOLAC Process: a procedure for identifying Best Common Practice Recommendations18OLAC Metadata Set #1以Dublin Core的15個元素(elements)為基礎，元素經進一步組織與定義，元素的限制準則為DC-Q，釋例D

15、CQ-HTML可由XML DTD或Schema編碼驗證.OLAC最新版的XML Schema: /OLAC/0.4/olac.xsd 例子：/OLAC/0.4/olac.xml 19The OLAC Metadata Set #2The three categories of metadata:Work language: describes information entities and their intellectual attributes e.g. names of works and their creatorsDocument language: describes and p

16、rovides access to the physical manifestation of information e.g. format, publisher, date, rightsSubject language: describes what a document is about e.g. subject, description20OLAC Metadata Set #3refine:：其element較精細或更多含意的規格.code ： encoding scheme精準的控制後設資料的值scheme ： 規範元素內容文字其標準化的名稱lang ：元素內容(element

17、content)所使用的語言langs ：屬於這元素的屬性,規範後設資料(metadata)閱讀時的語言ElementrefinecodeschemelangControl VocabularyControl VocabularyControl VocabularyControl VocabularyelementattributescontrolvocabularySmith21OLAC Metadata Set #3Name:標籤的正式名稱。Definition:以一行說明描述如何使用該元素(element).Comments:詳細描述如何使用該元素.包括DCMS和OLAC如何使用.Att

18、ributes: XML中該元素的屬性.Examples:例子.每個元素可重複出現.22OLAC Metadata SetLanguage #1Name: Audience LanguageDefinition:資源內容所使用的語言.Comments:創造者讓觀眾了解作品所使用的語言.請與Subject.language比較.例如:文學作品或僅使用一種語言的文件，演講者輔助的特殊語言,聲音記錄所使用的語言，句法描述所使用的語言,註解文字和雙語字典的解釋所使用的語言,但被註解的文字以及雙語字典中被定義的文字都要以Subject.language標註.Attributes:code:控制詞彙請參見

19、OLAC-Language.控制詞彙不足或與控制詞彙用語不同時,則以元素內容加以描述.23OLAC Metadata SetLanguage #2ExamplesA resource in English about the Sikaiana language:A Yemba-French dictionary, where the alternate name Dschang is preferred.DschangThe American Heritage Dictionary, which is both in and about American English:A resource

20、about a language for which the controlled vocabulary does not yet provide a code:Ancient Sumerian24OLAC and Asian LanguagesTWO IssuesLanguage IdentificationIs current OLAC/Enthnologue vocabulary rich enough to describe all Asian languages?Multilingual ResourcesIs current OLACMS and Processes compreh

21、ensive enough to describe multilingual resources?25Language IdentificationThe DC two letter code (e.g. en for English) is not enough to describe all the languages in the worldEnthnologue () is currently the most comprehensive description of the worlds languages Potential Prob

22、lems of using Ethnologue (or any existing language list)over-splitting over-chunking omission 26Solution LI Problems #1Use controlled vocabulary for elaboration:Northern/TakituduhNorthern/TakibakhaCentral/TakbanuazCentral/TakivatanSouthern/Isbukun 27Solution LI Problems #2Registering language groups

23、 with an OLAC registration service :OLAC language classification server would house a comprehensive list of language family names (defined by users) and their extensional definitions (i.e. sets of Ethnologue codes) AS:Amis = ALV, AIS 28Multilingual Resources #1Directionality is crucial in multilingu

24、al resourcesHowever, OLAC metadata is flat and unordered In MT systems: lost information but sufficient for resource harvestingBi-directional MT 29Multilingual Resources #2One-to-many MT: Many-to-one MT: 30Multilingual Resources #3Text: languageBitext (bilingual aligned corpus) There is always an directionalityOriginal-language Tra