koha:/usr/local/koha_cvs/dev_week/koha-new-upload/koha/misc/migration_toolsToday | Next day → | Search | Index
All times shown according to UTC.
| Time | Nick | Message |
|---|---|---|
| 08:30 | paul | kados around ? |
| 08:43 | kados | paul: yep |
| 08:43 | paul | hello kados |
| 08:43 | kados | hi :-) |
| 08:44 | paul | do you have a minut for a quick question ? |
| 08:44 | kados | yep |
| 08:44 | paul | I have indexed a large (50 000 biblios) with zebra |
| 08:44 | (& a new rebuild_zebra script i'll commit soon) | |
| 08:44 | but... | |
| 08:44 | kados | update_items already rebuilds the zebra index |
| 08:45 | if you pass it the -all parameter | |
| 08:45 | paul | when I search I get results only if I enter something that is in the title. |
| 08:45 | sample : | |
| 08:45 | keyword chaos => returns 10 biblios | |
| 08:45 | author chaos => returns the same 10 biblios | |
| 08:46 | keyword : <an author> => returns nothing | |
| 08:46 | author <an author> => nothing | |
| 08:46 | kados | there are so many possible reasons for that :-) |
| 08:46 | paul | stemming=0, field_weightiing=0 too |
| 08:46 | kados | it could be the zebra conf files |
| 08:46 | paul | yes, but I have a more precise question |
| 08:46 | kados | ok |
| 08:47 | paul | I uncommented buildQuery last lines : |
| 08:47 | warn "QUERY:".$koha_query; | |
| 08:47 | warn "SEARCHDESC:".$human_search_desc; | |
| 08:47 | warn "FEDERATED QUERY:".$federated_query; | |
| 08:47 | kados | it would be useful to see what the query is |
| 08:47 | paul | and get : |
| 08:47 | [Mon Oct 23 14:40:25 2006] [error] [client 192.168.0.2] QUERY:chaos at /home/paul/koha.dev/rel_3_0/C4/Search.pm line 3118., referer: http://i18.bureau.paulpoulain.[…]talogue/search.pl | |
| 08:47 | [Mon Oct 23 14:40:25 2006] [error] [client 192.168.0.2] SEARCHDESC:au: chaos at /home/paul/koha.dev/rel_3_0/C4/Search.pm line 3119., referer: http://i18.bureau.paulpoulain.[…]talogue/search.pl | |
| 08:47 | [Mon Oct 23 14:40:25 2006] [error] [client 192.168.0.2] FEDERATED QUERY:au= chaos at /home/paul/koha.dev/rel_3_0/C4/Search.pm line 3120., referer: http://i18.bureau.paulpoulain.[…]talogue/search.pl | |
| 08:47 | [Mon Oct 23 14:40:25 2006] [error] [client 192.168.0.2] QUERYING : chaos at /home/paul/koha.dev/rel_3_0/C4/Search.pm line 2776., referer: http://i18.bureau.paulpoulain.[…]talogue/search.pl | |
| 08:47 | QUERY:chaos seems wrong to me. | |
| 08:47 | kados | why? |
| 08:48 | it will use the default | |
| 08:48 | paul | should be related to author somewhere no ? |
| 08:48 | kados | ahh, yes |
| 08:48 | paul | (because is was "author : chaos" |
| 08:48 | kados | right |
| 08:48 | paul | what should I see here ? |
| 08:48 | kados | on dev_week with MARC21 |
| 08:48 | au:chaos | |
| 08:49 | paul | same as SEARCHDESC so ? |
| 08:49 | kados | (but do you have an author with 'chaos'? |
| 08:49 | yes | |
| 08:49 | owen | Hi everybody |
| 08:49 | kados | hi owen |
| 08:49 | paul | nope. but I get answers anyway. |
| 08:49 | kados | it should be au:chaos |
| 08:52 | paul: are you sure toins has not modified buildQuery? | |
| 08:52 | paul | unless he made something wrong no. |
| 08:53 | (could stemming=0 & field_weight=0 change sometihng ?) | |
| 08:53 | kados | I do see one possible problem |
| 08:54 | see this clause: | |
| 08:54 | else { | |
| 08:54 | $query.=" $operand"; | |
| 08:54 | if (!$index) { | |
| 08:54 | around like 3265 | |
| 08:54 | line even | |
| 08:55 | maybe $query.=" $operand"; should be: | |
| 08:55 | $query.="$index: $operand"; | |
| 08:55 | but I'm not 100% sure, I'd have to test it | |
| 08:55 | paul | i'll test & let you know |
| 08:56 | kados | (in fact, if it works, it should be modified slightly to test for the existance of $index first |
| 08:56 | $query.=" $index: $operand"; would be safer (adds a space) | |
| 09:05 | paul | kadose : about my script that rebuild zebra : it does many things : |
| 09:05 | s/kadose/kados/ | |
| 09:05 | it : | |
| 09:05 | - create the directory structure | |
| 09:05 | - copies the needed files from zebraplugin into the directory structure | |
| 09:06 | - creates the zebra-biblios.cfg with the dir structure too | |
| 09:06 | - dump the records | |
| 09:06 | - zebraidx | |
| 09:06 | so, once you have a koha.xml defined, just run it, it does what is needed where it is needed | |
| 09:06 | (& takes care of record.abs depending on your marc flavour) | |
| 09:07 | kados | cool |
| 09:07 | so you only run it once, on installation? | |
| 09:08 | or is it meant to be run every time you re-index? | |
| 09:09 | paul | you can do whatever you want, it creates what is needed. |
| 09:10 | there is even a parameter not to dump your file & preserve your export, when you want to test things | |
| 09:47 | kados : something is wrong with my setup : i can't do any search on something except title/subtitle : "find caroll" in yaz-client don't work, while "find chaos" works fine. | |
| 09:48 | this is not koha related, this is probably record.abs | |
| 09:50 | kados | yep, sounds like it |
| 09:50 | there are several places to check: | |
| 09:50 | paul | in yaz-client find "ti=chaos" should return records with title=chaos, right ? |
| 09:51 | kados | record.abs, bib1.att, ccl.properties |
| 09:51 | dewey | record.abs, bib1.att, ccl.properties are the most important files |
| 09:51 | paul | (because it don't work either. only "find chaos" returns something) |
| 09:51 | kados | paul: ti=chaos will work if you specified you're using ccl |
| 09:51 | paul | yep : yaz-client -c /home/zebradbs/rel_3_0/biblios/etc/ccl.properties @:9900/biblios is enough right ? |
| 09:51 | kados | yep |
| 09:52 | paul | ccl.properties contains : ti u=4 s=pw |
| 09:52 | kados | (and by the way, I've been starting zebra on a unix socket these days, much faster even) |
| 09:52 | paul | bib1.att : att 4 Title |
| 09:52 | kados | ok, and record.abs for Title? |
| 09:52 | paul | melm 200$atitle:w,title:p,title:s |
| 09:52 | melm 200$ctitle:w,title:p,title:s | |
| 09:52 | melm 200$dtitle:w,title:p,title:s | |
| 09:52 | # subtitle | |
| 09:53 | melm 200$etitle:w,title:p,title:s | |
| 09:53 | # authorities title | |
| 09:53 | melm 500title | |
| 09:53 | melm 501title | |
| 09:53 | melm 503title | |
| 09:53 | melm 510title | |
| 09:53 | melm 512title | |
| 09:53 | melm 513title | |
| 09:53 | melm 514title | |
| 09:53 | melm 515title | |
| 09:53 | melm 516title | |
| 09:53 | melm 517title | |
| 09:53 | melm 518title | |
| 09:53 | melm 519title | |
| 09:53 | melm 520title | |
| 09:54 | melm 530title | |
| 09:54 | melm 531title | |
| 09:54 | melm 532title | |
| 09:54 | melm 540title | |
| 09:54 | melm 541title | |
| 09:54 | melm 545title | |
| 09:54 | kados | looks fine |
| 09:54 | paul | oups. not kicked... |
| 09:54 | kados | hehe |
| 09:54 | paul | and for authors : |
| 09:54 | record.abs : melm 200$fauthor,author:p,author:s | |
| 09:54 | ccl.properties : au u=1003 s=pw | |
| 09:54 | bib1.att : att 1003 Author | |
| 09:54 | that's my feeling... | |
| 09:54 | hehe what ? | |
| 09:54 | kados | one thing |
| 09:54 | author vs Author | |
| 09:54 | not sure it matters | |
| 09:55 | paul | your record.abs (the marc21 one on CVS) is case-variant. |
| 09:55 | so I thought it did not matter | |
| 09:55 | kados | ok, then it doesn't :-) |
| 09:55 | hmmm | |
| 09:55 | paul | maybe i'm using a file i'm not looking at... |
| 09:56 | kados | can you confirm that zebra is indexing using the right config files? |
| 09:56 | check zebra-biblios.cfg tab file | |
| 09:56 | paul | profilePath:${srcdir:-.}:/home/zebradbs/rel_3_0/biblios/tab/:/usr/local/share/idzebra/tab/:${srcdir:-.}/tab/ |
| 09:57 | kados | I would limit it to just the one dir |
| 09:57 | I ahve: | |
| 09:57 | profilePath:/koha/zebradb/biblios/tab/ | |
| 10:03 | paul | kados : things are a little better : |
| 10:03 | find <title> works | |
| 10:03 | find <author> works | |
| 10:03 | but find "ti=<title>" does not work | |
| 10:05 | kados | do you have a line in ccl.properties: |
| 10:05 | ti title | |
| 10:05 | (ie, an alias ) | |
| 10:05 | paul | nop, just ti u=4 s=pw |
| 10:06 | kados | ahh |
| 10:06 | that'll work too | |
| 10:07 | paul | mmm... my ccl.properties is really different from the dev_week one (beeping toins) |
| 10:10 | something seems wrong with CVS ... | |
| 10:11 | kados | what's happening? |
| 10:12 | paul | my ccl.properties from dev_week is 1020 lines long, & toins one is less than 100. |
| 10:12 | both are said "uptodate" | |
| 10:13 | kados | paul: in the same repo? |
| 10:13 | cm: did I ever respond to your email last week? | |
| 10:14 | cm | hi kados. no, you didn't. :) |
| 10:14 | paul | hello cm |
| 10:14 | kados | ahh yes ... here it is :-) |
| 10:14 | cm: I didn't know you were using bulkmarcimport from dev_week | |
| 10:14 | cm | hi paul! |
| 10:14 | kados | cm: I will fix it immediately |
| 10:14 | cm | yeah, i think that's what I used the last time i did a full import. |
| 10:15 | thanks! | |
| 10:15 | lemme know when you're done. | |
| 10:15 | kados | cm: (I thought you were going through rel_2_2's bulkmarcimport ... has that changed?) |
| 10:16 | cm | i think the first time i installed dev_week I used it, but I redid it a few weeks ago and I think I forgot and used the one in dev_week. |
| 10:17 | I tried it with rel_2_2, which worked, but nothing was imported into biblioitems.marc. Is that data needed, or will it be? | |
| 10:17 | kados | well, since you were going by way of rel_2_2 before |
| 10:18 | the way to populate biblioitems.marc is with import_to_biblioitems script | |
| 10:18 | cm | is that in 2_2 or dev_week? |
| 10:18 | kados | lemme check |
| 10:19 | oops gave you the wrong name | |
| 10:19 | cm | for what? |
| 10:19 | kados | zebraplugin/utils/move_marc_to_biblioitems.pl |
| 10:19 | that's what you need to update a rel_2_2 system to dev_week | |
| 10:19 | ie, move from the marc-* tables to biblioitems.marc | |
| 10:20 | cm | ah. okay. |
| 10:20 | kados | cm: committed bulkmarcimport |
| 10:20 | cm | but I don't have marc-*tables anymore (except the structure ones) |
| 10:20 | i think. | |
| 10:20 | cool. I'll try it out. :D | |
| 10:22 | kados | cm: so is your plan to only use dev_week from the get-go now? |
| 10:22 | cm: for importing I mean | |
| 10:22 | cm | very likely. as long as it works. :) |
| 10:23 | i'm turning off zebra, though and uploading into zebra after import. | |
| 10:23 | otherwise it takes too long. | |
| 10:25 | kados | yep, makes sense |
| 10:26 | cm | i'm very glad! |
| 10:26 | kados | I've got a new version of update_items that will rebuild the zebra index automagically |
| 10:26 | I'll commit it now | |
| 10:26 | cm | interesting. |
| 10:26 | kados | oh, and cm, you should upgrade to the latest zebra version in the 1.3 series |
| 10:26 | released this morning | |
| 10:27 | cm | good to know. i'll do that too. |
| 10:27 | kados | it has a bugfix for a nasty multiple simultaneous updates over extended services |
| 10:27 | bug | |
| 10:27 | :-) | |
| 10:27 | discovered by yours truly :-) | |
| 10:30 | cm | congrats. bugfixes are good! |
| 10:30 | kados | misc/update_items.pl |
| 10:31 | will probably need some customization | |
| 10:31 | but i run that with the -today flag for NPL in cron every evening | |
| 10:31 | and if I need to rebuild the index with the -all flag | |
| 10:32 | let me know if the docs aren't clear in that file | |
| 10:32 | cm | okay. thanks. |
| 10:40 | uh oh. I got two errors: | |
| 10:40 | kados | bulkmarcimport errors? |
| 10:40 | cm | DBD::mysql::st execute failed: Unknown column 'ccode' in 'field list' at /usr/local/koha/intranet/modules/C4/Biblio.pm line 1957. problem with Zebra at /usr/local/koha/intranet/modules/C4/Context.pm line 490. |
| 10:40 | kados | right |
| 10:40 | cm | yeah. :/ |
| 10:40 | kados | that's an easy fix |
| 10:40 | cm | the other was: |
| 10:41 | ZOOM error 10000 "Connect failed" (addinfo: "tcp:@:9900") from diag-set 'ZOOM'rootkoha:/usr/local/koha_cvs/dev_week/koha-new-upload/koha/misc/migration_tools |
|
| 10:41 | kados | hmmm |
| 10:41 | cm | but I think that can probably be ignored. i remember seeing something like that before |
| 10:41 | kados | that doesn't look good |
| 10:41 | but if you're not using zebra it shouldn't matter | |
| 10:41 | ok ... so lemme check the wiki | |
| 10:42 | I'll add the sql you need | |
| 10:43 | cm | good. thanks. |
| 10:44 | kados | cm: can you check your biblioframework.sql file? |
| 10:44 | make sure it has: alter table kohanew.biblioitems add ccode varchar(4); | |
| 10:45 | cm | ok. let me look. |
| 10:46 | paul | kados : is there a reason to have, in koha.xml : |
| 10:47 | <ccl2rpn>/home/zebradbs/rel_3_0/biblios/etc/ccl.properties</ccl2rpn> in <serverinfo id="biblioserver"> section | |
| 10:47 | kados | yes |
| 10:47 | paul | while <cql2rpn>/home/zebradbs/rel_3_0/biblios/etc/pqf.properties</cql2rpn> is in <server id="..."> |
| 10:47 | kados | because zebra complains if there are unrecognized elements in <server element |
| 10:47 | so we need one for zebra, and one for extra stuff | |
| 10:48 | (it's not ideal, I agree) | |
| 10:48 | paul | zebra undestand cql2rpn but NOT ccl2rpn ? |
| 10:48 | strange | |
| 10:48 | kados | right |
| 10:48 | maybe there is a way to tell it about ccl2rpn, but I don't know it | |
| 10:48 | cm | do you mean utils/biblio_framework.sql? that doesn't have it. the latest cvs I just downloaded doesn't have it in there either. |
| 10:48 | kados | cm: ok, I'll update it |
| 10:48 | paul | another question : |
| 10:49 | yaz-client -c /home/zebradbs/rel_3_0/biblios/etc/ccl.properties @:9900/biblios | |
| 10:49 | cm | zehr gut. :) |
| 10:49 | paul | querytype ccl |
| 10:49 | find "ti=dogmatique" | |
| 10:49 | gives me a : | |
| 10:49 | [107] Query type not supported -- v2 addinfo 'type-2' | |
| 10:49 | which let me think he does not support ccl | |
| 10:49 | kados | nope |
| 10:50 | that means that attribute type 2 isn't supported | |
| 10:50 | it should be specified in your record.abs | |
| 10:50 | paul: just a sec ... lemme commit this file | |
| 10:51 | paul | no prob |
| 10:52 | kados | cm: ok, update and profit ;-) |
| 10:52 | paul: so lets trace what zebra is doing here | |
| 10:52 | paul: first, you say 'ti=dogmatique' | |
| 10:52 | and I assume you're using a ccl specification, right? | |
| 10:52 | so check ccl.properties | |
| 10:53 | for the ti line and tell me what it is | |
| 10:53 | paul | ti u=4 |
| 10:53 | title ti | |
| 10:53 | (i don't have reindexed after adding title ti alias, but i don't think it matters) | |
| 10:53 | kados | you changed it! |
| 10:53 | before you had ti u=4 s=something | |
| 10:54 | right? | |
| 10:54 | paul | right. |
| 10:54 | I tried both, without any difference | |
| 10:54 | ti u=4 s=pw | |
| 10:54 | kados | ahh, I missread your error |
| 10:55 | you must start yaz liek this; | |
| 10:55 | yaz-client -c /path/to/ccl.properties | |
| 10:55 | paul | yes : |
| 10:55 | [16:49] <paul> yaz-client -c /home/zebradbs/rel_3_0/biblios/etc/ccl.properties @:9900/biblios | |
| 10:56 | kados | try this: |
| 10:56 | yaz-client -c /home/zebradbs/rel_3_0/biblios/etc/ccl.properties | |
| 10:56 | > open localhost:9900/biblios | |
| 10:56 | cm | kados: thanks! I let you know how it works shortly. |
| 10:56 | kados | > f ti=something |
| 10:57 | paul | Z> open localhost:9900/biblios |
| 10:57 | Connecting...OK. | |
| 10:57 | Sent initrequest. | |
| 10:57 | Connection accepted by v3 target. | |
| 10:57 | ID : 81 | |
| 10:57 | Name : Zebra Information Server/GFS/YAZ | |
| 10:57 | Version: Zebra 1.3.32/1.66/2.1.12 | |
| 10:57 | Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets | |
| 10:57 | Elapsed: 0.002776 | |
| 10:57 | Z> f ti=dogmatique | |
| 10:57 | Sent searchRequest. | |
| 10:57 | Received SearchResponse. | |
| 10:57 | Search was a success. | |
| 10:57 | Number of hits: 0, setno 1 | |
| 10:57 | SearchResult-1: term=ti dogmatique cnt=0 | |
| 10:57 | records returned: 0 | |
| 10:57 | Elapsed: 0.006193 | |
| 10:57 | (note that I have zebra 1.3, not 2.0) | |
| 10:58 | kados | I have the same prob |
| 10:58 | just a sec | |
| 10:59 | maybe we need to say: | |
| 10:59 | querytype ccl | |
| 10:59 | or better | |
| 10:59 | querytype ccl2rpn | |
| 10:59 | yep, that works |
Today | Next day → | Search | Index
