12,303
edits
LexVoc: Difference between revisions
From LexBib
→LexVoc development and full text indexation process
| (7 intermediate revisions by the same user not shown) | |||
| Line 28: | Line 28: | ||
== Elexifinder Categories == | == Elexifinder Categories == | ||
Terms that serve as [[Elexifinder]] category belong to the first three [[Property:P72|skos:broader]] hierarchy levels below the [[Item:Q1|root concept]]. Terms deeper in the hierarchy are considered in article indexation, and so are closeMatch terms without own broader-hierarchy, but the assigned category visible on [[Elexifinder]] will be the corresponding broader category of the third level below root. | Terms that serve as [[Elexifinder]] category belong to the first three [[Property:P72|skos:broader]] hierarchy levels below the [[Item:Q1|root concept]]. Terms deeper in the hierarchy are considered in article indexation, and so are [[Property:P77|closeMatch]] terms without own broader-hierarchy, but the assigned category visible on [[Elexifinder]] will be the corresponding broader category of the third level below root. | ||
* Graph view, [[Elexifinder]] categories only (upper three hierarchy levels): [https://lexbib.elex.is/query/#%23defaultView%3AGraph%0A%23%20this%20shows%20only%20three%20levels%20of%20skos%3Abroader%20relation%20below% | * Graph view, [[Elexifinder]] categories only (upper three hierarchy levels): [https://lexbib.elex.is/query/#%23defaultView%3AGraph%0A%23%20this%20shows%20only%20three%20levels%20of%20skos%3Abroader%20relation%20below%20Q1.%0APREFIX%20lwb%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fentity%2F%3E%0APREFIX%20ldp%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fdirect%2F%3E%0APREFIX%20lp%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2F%3E%0APREFIX%20lps%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fstatement%2F%3E%0APREFIX%20lpq%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fqualifier%2F%3E%0APREFIX%20lpr%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Freference%2F%3E%0APREFIX%20lno%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fnovalue%2F%3E%0A%0Aselect%20distinct%20%0A%3Fs%20%3FsLabel%20%0A%23%3Fdistance%20%0A%3Frgb%20%0A%3Ft%20%3FtLabel%0A%0Awhere%20%7B%0A%20%20%3Fs%20ldp%3AP5%20lwb%3AQ7.%0A%20%20%0A%20%20%7B%20lwb%3AQ1%20ldp%3AP73%20%3Ft.%20%7D%20%0A%20%20UNION%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%7B%20lwb%3AQ1%20ldp%3AP73%2Fldp%3AP73%20%3Ft.%20%7D%20%0A%20%20UNION%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%7B%20lwb%3AQ1%20ldp%3AP73%2Fldp%3AP73%2Fldp%3AP73%20%3Ft.%20%7D%20%0A%20%20%0A%20%20%3Ft%20ldp%3AP72%20%3Fs.%0A%20%20%3Fs%20rdfs%3Alabel%20%3FsLabel%20.%20FILTER%20%28lang%28%3FsLabel%29%3D%22en%22%29%0A%20%20%3Ft%20rdfs%3Alabel%20%3FtLabel%20.%20FILTER%20%28lang%28%3FtLabel%29%3D%22en%22%29%0A%20%20%0A%20%20%23%20distance%20from%20root%20node%20%22Lexicography%22%20%28number%20of%20broader%20concepts%29%0A%20%20%7B%20select%20%3Fs%20%28count%20%28%3Fbroader%29%20as%20%3Fdistance%29%20where%20%7B%0A%20%20%20%20%20%20OPTIONAL%20%7B%3Fs%20ldp%3AP72%2B%20%3Fbroader%20.%20%7D%7D%20GROUP%20BY%20%3Fs%20%3Fdistance%20%7D%0A%0A%20%20%23%20colouring%0A%20%20BIND%20%28%0A%20%20COALESCE%28%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%221%22%20%2C%20%22CCFFE5%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%222%22%20%2C%20%22FF9999%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%223%22%20%2C%20%22FFB266%22%2C%201%2F0%29%2C%0A%20%20%20%23%20IF%28str%28%3Fdistance%29%3D%223%22%20%2C%20%22FFFF99%22%2C%201%2F0%29%2C%0A%20%20%20%20%22FFFF99%22%0A%20%20%29%20AS%20%3Frgb%0A%20%20%29%0A%20%20%7D%20GROUP%20BY%20%0A%20%20%3Fs%20%3FsLabel%20%20%20%0A%20%20%23%3Fdistance%0A%20%20%3Frgb%20%0A%20%20%3Ft%20%3FtLabel%0A%20%20 Query]. | ||
== LexVoc main branches == | == LexVoc main branches == | ||
| Line 46: | Line 46: | ||
* Software <TBD> ([https://lexbib.elex.is/query/#%23defaultView%3AGraph%0A%23%20Graph%20view%20with%20specified%20root%20node%0APREFIX%20lwb%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fentity%2F%3E%0APREFIX%20ldp%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fdirect%2F%3E%0APREFIX%20lp%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2F%3E%0APREFIX%20lps%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fstatement%2F%3E%0APREFIX%20lpq%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fqualifier%2F%3E%0APREFIX%20lpr%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Freference%2F%3E%0APREFIX%20lno%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fnovalue%2F%3E%0A%0Aselect%20%3Fs%20%3FsLabel%0A%3Frgb%0A%3FedgeLabel%20%3Fo%20%3FoLabel%20where%20%7B%0A%20%20%0A%20%20BIND%20%28lwb%3AQ14281%20as%20%3Froot%29%20%23%20specify%20root%20node%20for%20the%20representation%20here%2C%20e.g.%20%22lwb%3AQ1%22%20for%20%22Lexicography%22%0A%20%20%0A%20%20%3Fo%20ldp%3AP5%20lwb%3AQ7.%0A%20%20%3Froot%20ldp%3AP73%2a%20%3Fo.%0A%20%20%3Fs%20%3Fp%20%3Fo%20.%0A%20%20%20%20%0A%20%20%7B%3Froot%20ldp%3AP73%2a%20%3Fs%20.%7D%20UNION%20%7B%3Fs%20ldp%3AP77%7Cldp%3AP5%20%3Fo.%7D%0A%20%0A%20%20%3Fs%20rdfs%3Alabel%20%3FsLabel%20.%20FILTER%20%28lang%28%3FsLabel%29%3D%22en%22%29%0A%20%20%3Fedge%20wikibase%3AdirectClaim%20%3Fp%20.%0A%20%20FILTER%20%28%3Fp%20%3D%20ldp%3AP73%20%7C%7C%20%3Fp%20%3D%20ldp%3AP77%20%7C%7C%20%3Fp%20%3D%20ldp%3AP5%29%20%23%20skos%3Anarrower%20%2F%20skos%3AcloseMatch%20%2F%20instance%20of%0A%20%20%3Fedge%20rdfs%3Alabel%20%3FedgeLabel%20.%0A%20%20%3Fo%20rdfs%3Alabel%20%3FoLabel%20.%20FILTER%20%28lang%28%3FoLabel%29%3D%22en%22%29%0A%20%0A%20%20%23%20distance%20from%20root%20node%20%22Lexicography%22%20%28number%20of%20broader%20concepts%29%0A%20%20%7B%20select%20%3Fs%20%28count%20%28%3Fbroader%29%20as%20%3Fdistance%29%20where%20%7B%0A%20%20%20%20%20%20OPTIONAL%20%7B%3Fs%20ldp%3AP72%2B%20%3Fbroader%20.%20%7D%7D%20GROUP%20BY%20%3Fs%20%3Fdistance%20%7D%0A%0A%20%20%23%20colouring%0A%20%20BIND%20%28%0A%20%20COALESCE%28%0A%20%20%20%20IF%28%3Fs%20%3D%20%3Froot%20%2C%20%220000CC%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28%3Fp%20%3D%20ldp%3AP77%2C%20%22CCFF99%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28%3Fp%20%3D%20ldp%3AP5%2C%20%22CCFFE5%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%221%22%20%2C%20%22FF9999%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%222%22%20%2C%20%22FFB266%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%223%22%20%2C%20%22FFFF99%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%224%22%20%2C%20%22CCFF99%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%225%22%20%2C%20%22CCFFE5%22%2C%201%2F0%29%2C%0A%20%20%20%20%22FFFF99%22%0A%20%20%29%20AS%20%3Frgb%0A%29%0A%20%20%7D%20GROUP%20BY%20%3Fs%20%3FsLabel%20%3Fdistance%20%3Frgb%20%3FedgeLabel%20%3Fo%20%3FoLabel%0A Graph]) | * Software <TBD> ([https://lexbib.elex.is/query/#%23defaultView%3AGraph%0A%23%20Graph%20view%20with%20specified%20root%20node%0APREFIX%20lwb%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fentity%2F%3E%0APREFIX%20ldp%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fdirect%2F%3E%0APREFIX%20lp%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2F%3E%0APREFIX%20lps%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fstatement%2F%3E%0APREFIX%20lpq%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fqualifier%2F%3E%0APREFIX%20lpr%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Freference%2F%3E%0APREFIX%20lno%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fnovalue%2F%3E%0A%0Aselect%20%3Fs%20%3FsLabel%0A%3Frgb%0A%3FedgeLabel%20%3Fo%20%3FoLabel%20where%20%7B%0A%20%20%0A%20%20BIND%20%28lwb%3AQ14281%20as%20%3Froot%29%20%23%20specify%20root%20node%20for%20the%20representation%20here%2C%20e.g.%20%22lwb%3AQ1%22%20for%20%22Lexicography%22%0A%20%20%0A%20%20%3Fo%20ldp%3AP5%20lwb%3AQ7.%0A%20%20%3Froot%20ldp%3AP73%2a%20%3Fo.%0A%20%20%3Fs%20%3Fp%20%3Fo%20.%0A%20%20%20%20%0A%20%20%7B%3Froot%20ldp%3AP73%2a%20%3Fs%20.%7D%20UNION%20%7B%3Fs%20ldp%3AP77%7Cldp%3AP5%20%3Fo.%7D%0A%20%0A%20%20%3Fs%20rdfs%3Alabel%20%3FsLabel%20.%20FILTER%20%28lang%28%3FsLabel%29%3D%22en%22%29%0A%20%20%3Fedge%20wikibase%3AdirectClaim%20%3Fp%20.%0A%20%20FILTER%20%28%3Fp%20%3D%20ldp%3AP73%20%7C%7C%20%3Fp%20%3D%20ldp%3AP77%20%7C%7C%20%3Fp%20%3D%20ldp%3AP5%29%20%23%20skos%3Anarrower%20%2F%20skos%3AcloseMatch%20%2F%20instance%20of%0A%20%20%3Fedge%20rdfs%3Alabel%20%3FedgeLabel%20.%0A%20%20%3Fo%20rdfs%3Alabel%20%3FoLabel%20.%20FILTER%20%28lang%28%3FoLabel%29%3D%22en%22%29%0A%20%0A%20%20%23%20distance%20from%20root%20node%20%22Lexicography%22%20%28number%20of%20broader%20concepts%29%0A%20%20%7B%20select%20%3Fs%20%28count%20%28%3Fbroader%29%20as%20%3Fdistance%29%20where%20%7B%0A%20%20%20%20%20%20OPTIONAL%20%7B%3Fs%20ldp%3AP72%2B%20%3Fbroader%20.%20%7D%7D%20GROUP%20BY%20%3Fs%20%3Fdistance%20%7D%0A%0A%20%20%23%20colouring%0A%20%20BIND%20%28%0A%20%20COALESCE%28%0A%20%20%20%20IF%28%3Fs%20%3D%20%3Froot%20%2C%20%220000CC%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28%3Fp%20%3D%20ldp%3AP77%2C%20%22CCFF99%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28%3Fp%20%3D%20ldp%3AP5%2C%20%22CCFFE5%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%221%22%20%2C%20%22FF9999%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%222%22%20%2C%20%22FFB266%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%223%22%20%2C%20%22FFFF99%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%224%22%20%2C%20%22CCFF99%22%2C%201%2F0%29%2C%0A%20%20%20%20IF%28str%28%3Fdistance%29%3D%225%22%20%2C%20%22CCFFE5%22%2C%201%2F0%29%2C%0A%20%20%20%20%22FFFF99%22%0A%20%20%29%20AS%20%3Frgb%0A%29%0A%20%20%7D%20GROUP%20BY%20%3Fs%20%3FsLabel%20%3Fdistance%20%3Frgb%20%3FedgeLabel%20%3Fo%20%3FoLabel%0A Graph]) | ||
== | == LexVoc development and full text indexation process == | ||
LexVoc, in its actual state, is by no means complete nor finished. It is ongoing work. On Elexis LexMeet, you can contribute to the [https://meet.elex.is/groups/lexicographic-concepts-vocabulary/ discussion about LexVoc structure], i.e. the definition of main branches, and the inner organization of these. We also call for collaboration [[LexVoc translation on Lexonomy]] (see our [https://lexbib.org/Kosem-Lindemann-fullpaper-printed.pdf 2021 Euralex paper]). | LexVoc, in its actual state, is by no means complete nor finished. It is ongoing work. On Elexis LexMeet, you can contribute to the [https://meet.elex.is/groups/lexicographic-concepts-vocabulary/ discussion about LexVoc structure], i.e. the definition of main branches, and the inner organization of these. We also call for collaboration for [[LexVoc translation on Lexonomy]] (see our [https://lexbib.org/Kosem-Lindemann-fullpaper-printed.pdf 2021 Euralex paper]). | ||
Regarding inclusion of new terms: As soon as a new term (i.e. a member of [[Item:Q7|Class "Term"]], and of at least one [[Item:Q33|skos:Collection]]) is linked to a member of the main SKOS graph using [[Property:P72|skos:broader]] or [[Property:P77|skos:closeMatch]], it is considered in subsequent iterations of article full text indexation. On the other hand, if a term has no [[Property:P72|skos:broader]] or [[Property:P77|skos:closeMatch]] relation to another term, it is not (any more) considered for article full text indexation. An article full text indexation iteration (performed locally by [[User:DavidL]]) thus reflects the state of LexVoc at that particular point of time. | Regarding inclusion of new terms: As soon as a new term (i.e. a member of [[Item:Q7|Class "Term"]], and of at least one [[Item:Q33|skos:Collection]]) is linked to a member of the main SKOS graph using [[Property:P72|skos:broader]] or [[Property:P77|skos:closeMatch]], it is considered in subsequent iterations of article full text indexation. On the other hand, if a term has no [[Property:P72|skos:broader]] or [[Property:P77|skos:closeMatch]] relation to another term, it is not (any more) considered for article full text indexation. An article full text indexation iteration (performed locally by [[User:DavidL]]) thus reflects the state of LexVoc at that particular point of time. | ||
Full text indexation with LexVoc terms, version 3, is currently in preparation. You can see single articles and associated LexVoc terms in version 2 (on a single bibliographical item's wikipage ([[Item:Q13936|example]]), click on the value of "LexBib v2 legacy ID".) | Full text indexation with LexVoc terms, version 3, is currently in preparation. In this version 3, at the moment, you can see in how many articles a term (i.e., at least one of the labels of that term) have been found in LexBib full texts (see below). You can see single articles and associated LexVoc terms in version 2 (on a single bibliographical item's wikipage ([[Item:Q13936|example]]), click on the value of "LexBib v2 legacy ID".) | ||
In how many articles have we found LexVoc terms (narrowers of "[[Item:Q1|Lexicography]]")? [https://lexbib.elex.is/query/#PREFIX%20lwb%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fentity%2F%3E%0APREFIX%20ldp%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fdirect%2F%3E%0APREFIX%20lp%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2F%3E%0APREFIX%20lps%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fstatement%2F%3E%0APREFIX%20lpq%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fqualifier%2F%3E%0APREFIX%20lpr%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Freference%2F%3E%0APREFIX%20lno%3A%20%3Chttp%3A%2F%2Flexbib.elex.is%2Fprop%2Fnovalue%2F%3E%0A%0Aselect%20%3FTerm%20%3FenPrefLabel%20%28group_concat%28%3FenAltLabel%3BSEPARATOR%3D%22%3B%22%29%20as%20%3FenAltLabels%29%20%3Fcorpus_hits%0A%0Awhere%20%7B%0A%20%20%3FTerm%20ldp%3AP5%20lwb%3AQ7%3B%0A%20%20rdfs%3Alabel%20%3FenPrefLabel.%20filter%28lang%28%3FenPrefLabel%29%3D%22en%22%29%0A%20%7B%20%3FTerm%20ldp%3AP72%2a%20lwb%3AQ1%20.%7D%20%23%20present%20in%20narrower-broader-tree%20with%20%22Lexicography%22%20as%20root%20node%0A%20%20%20UNION%0A%20%7B%20%3FTerm%20ldp%3AP77%20%3FcloseMatch.%20%3FcloseMatch%20ldp%3AP72%2a%20lwb%3AQ1%20.%20%7D%20%23%20includes%20closeMatch%20items%20without%20own%20broader-rels%0A%20%0A%20OPTIONAL%20%7B%20%3FTerm%20skos%3AaltLabel%20%3FenAltLabel.%0A%20%20%20%20%20%20%20%20%20%20%20%20filter%28lang%28%3FenAltLabel%29%3D%22en%22%29%20%7D%0A%20OPTIONAL%20%7B%20%3FTerm%20ldp%3AP109%20%3Fcorpus_hits.%7D%0A%0A%20%20%7D%20group%20by%20%3FTerm%20%3FenPrefLabel%20%3Fcorpus_hits%0A%20%20%20%20order%20by%20DESC%20%28xsd%3Ainteger%28%3Fcorpus_hits%29%29 Query]. | |||
== Stop-labels == | == Stop-labels == | ||
Some term lexicalizations are very ambiguous, and do not yield proper results (many false positives and/or too many distinct word senses relevant in the domain of Lexicography). At the time being, | Some term lexicalizations (SKOS ''labels'') are very ambiguous, and do not yield proper results (many false positives and/or too many distinct word senses relevant in the domain of Lexicography). At the time being, the approach towards that problem is a label stop-list, which is accessible [https://github.com/elexis-eu/elexifinder/blob/master/wikibase/stopterms.py here]. These term labels can be excluded in article full text indexation (if a term has several labels, the other labels are not affected); but currently, in order to review the stoplist, they aren't excluded. | ||
