← Previous day | Today | Next day → | Search | Index
All times shown according to UTC.
| Time | Nick | Message |
|---|---|---|
| 12:05 | paul | wow... segfault... |
| 12:05 | [paul portable zebradb]$ zebraidx -g iso2709 -c /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg -d kohaplugin update ../export/ |
|
| 12:05 | 19:05:59-12/07 zebraidx(30255) [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36 | |
| 12:05 | Erreur de segmentation | |
| 12:06 | in which directory am I supposed to be when I run this command ? | |
| 12:06 | I was in : /home/paul/koha.dev/dev_week/zebraplugin/zebradb | |
| 12:06 | 7PM, maybe it's time to leave for dinner... | |
| 12:06 | i'll investigate tomorrow... | |
| 12:07 | tumer | paul: you have to be one folder above rtecords foldfer |
| 12:07 | kados | paul: maybe missing 090s :-) |
| 12:07 | tumer | where zebra.cfg resides |
| 12:08 | kados | paul: also maybe need to make sure no CVS dir is in records? |
| 12:08 | paul | you have to be one folder above rtecords foldfer => done, no change |
| 12:09 | also maybe need to make sure no CVS dir is in records => checked, no CVS dir in my fresh export/ | |
| 12:09 | maybe missing 090s => missing090.pl run just before exporting. | |
| 12:09 | kados | version of zebra? |
| 12:09 | paul | +the problem occurs immediatly |
| 12:09 | a fresh 1.3.36 | |
| 12:10 | kados | hmmm |
| 12:10 | paul | + a fresh yaz install too |
| 12:10 | kados | what is the exact error? |
| 12:10 | tumer | paul run zebraidx with no parameters and see what it saya |
| 12:10 | paul | the exact error is just before : |
| 12:10 | 19:05:59-12/07 zebraidx(30255) [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36 | |
| 12:10 | Erreur de segmentation | |
| 12:11 | (segmentation fault in french) | |
| 12:11 | zebraidx without parameters give me the help page | |
| 12:11 | zebraidx | |
| 12:11 | zebraidx [options] command <dir> ... | |
| 12:11 | Commands: | |
| 12:11 | update <dir> Update index with files below <dir>. | |
| 12:11 | If <dir> is empty filenames are read from stdin. | |
| 12:11 | delete <dir> Delete index with files below <dir>. | |
| 12:11 | create <db> Create database <db> | |
| 12:11 | tumer | so its not zebra |
| 12:11 | paul | ... |
| 12:11 | kados | paul: paste in your exact command |
| 12:11 | that segfautls | |
| 12:11 | paul | zebraidx -g iso2709 -c /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg -d kohaplugin update export/ |
| 12:12 | i'm in /home/paul/koha.dev/dev_week/zebraplugin directory | |
| 12:12 | kados | how many records? |
| 12:12 | tumer | -d kohaplugin update export/ --- kados what is that? |
| 12:12 | paul | 14 000 (around) |
| 12:13 | kados | -d is database |
| 12:13 | update is the command | |
| 12:13 | export/ is where the iso2709 file is | |
| 12:13 | tumer | i know what does pauls code saying |
| 12:13 | paul | mmm... the DB is not kohaplugin |
| 12:13 | kados | paul: marcdump export/marc.iso2709 |
| 12:13 | paul: make sure your MARC is valid | |
| 12:14 | paul | oups... |
| 12:14 | utf8 "\xE9" does not map to Unicode at /usr/lib/perl5/5.8.7/i386-linux/Encode.pm line 166. | |
| 12:14 | welcome back, unicode problems... | |
| 12:17 | any idea ? | |
| 12:17 | tumer & kados ? | |
| 12:17 | kados | it seems you don't have utf8 data :-) |
| 12:17 | paul | (otherwise I leave for dinner...) |
| 12:17 | sh.... | |
| 12:17 | kados | wait |
| 12:17 | I'm wrong | |
| 12:18 | paul | ok, I wait ... |
| 12:18 | kados | that means that \xE9 is not valid MARC8 |
| 12:18 | it must be latin1 | |
| 12:18 | so maybe marcdump can't handle unimarc | |
| 12:18 | paul | iirc, it's the é, a very common accent in french |
| 12:18 | kados | it think you have marc8 data |
| 12:19 | paul | i get a segfault without any more interesting information. |
| 12:19 | can I send you my file to see if you get the same segfault ? | |
| 12:19 | tumer | well i think he still has latin1 |
| 12:19 | paul | maybe it's a yaz/zebra/??? version problem |
| 12:20 | tumer is probably right | |
| 12:20 | tumer | paul:use one record only and index with flag -s |
| 12:20 | if it works at all you will see lots of dump | |
| 12:21 | paul | why 1 record only ? |
| 12:21 | tumer | with -s you get the debugger showing you all the steps veeery long process |
| 12:22 | for zebra your encoding is no problem it does not check that | |
| 12:23 | paul | still segfault, without any message. |
| 12:23 | just | |
| 12:23 | 19:23:17-12/07 zebraidx(30360) [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36 | |
| 12:23 | segfault | |
| 12:24 | I updated my copy of dev_week, and the path to my zebradb are /koha... | |
| 12:24 | which is invalid ! | |
| 12:24 | tumer | i do not understand the linux line but if you are in the right folder -c shows the .cfg file than your zebra is corrupt |
| 12:26 | paul | tumer: will you be here tomorrow ? |
| 12:26 | tumer | i am around |
| 12:26 | paul | bye tumer & kados |
| 12:26 | tumer | bonapetit |
| 12:26 | paul | thanks |
| 02:17 | btoumi | hi all |
| 03:35 | somebody can explain me what is $DEBUG ? is it system variable? | |
| 03:49 | chris: are u around | |
| 03:49 | ? | |
| 04:27 | paul | kados : segfault solved : some directories where missing (key & tmp, under biblios/) |
| 07:20 | btoumi | paul are u around? |
| 07:22 | paul2 are u around? | |
| 07:24 | paul | hello btoumi |
| 07:24 | btoumi | hi paul |
| 07:24 | ca va? | |
| 07:24 | paul | il fait un peu froid à MArseille. Je vais mettre une petite laine... |
| 07:25 | btoumi | lol a istres aussi a -35° ca crain un peu |
| 07:25 | g une petite question pour toi au sujet de ce qui m'occupe en ce moment ,les amandes | |
| 07:26 | as tu le temps ou pas? | |
| 07:26 | paul | le amandes ? miam, c'est bon ca, les amandes... |
| 07:26 | je préfère les noisettes remarque. | |
| 07:26 | btoumi | lol |
| 07:26 | paul | mais bon, zyva pour les amandes |
| 07:27 | btoumi | a quoi correspond maxfine |
| 07:27 | ? | |
| 07:28 | paul | ou tu vois ca ? |
| 07:28 | btoumi | dans fines2.pl line 68 |
| 07:29 | je penses a un montant maximum mais pas sur | |
| 07:30 | paul | fines2.pl est dans quel répertoire ? |
| 07:30 | ok, trouvé | |
| 07:31 | btoumi | misc |
| 07:31 | paul | ligne 93 |
| 07:31 | c'est donc bien un maximum | |
| 07:31 | n'hésites pas à en faire une préférence système ! | |
| 07:31 | btoumi | a la ligne 68 il y a my maxFine=5; |
| 07:32 | a quoi sert t 'elle exactement? | |
| 07:32 | je suis d'accord pour la prefence systeme | |
| 07:32 | paul | à affecter la variable en début de script pour permettre une personnalisation + simple |
| 07:32 | rustique, mais efficace ! | |
| 07:33 | btoumi | est ce que je peut t'appeler? |
| 07:34 | paul | c'est difficile : je suis chez Antoine pour avoir internet, mais SFR passe très mal ou il habite sur la colinne de la bonne mère pourtant ! |
| 07:34 | mais tu peux tenter le coup si tu veux (sur mon N° portable donc) | |
| 07:35 | btoumi | ok j'essaye |
| 07:41 | meric paul: | |
| 07:41 | merci pardon | |
| 08:06 | paul | kados around ? |
| 08:06 | kados | yep |
| 08:06 | hia ll | |
| 08:06 | hi all even | |
| 08:07 | :-) | |
| 08:07 | paul | hello & good morning |
| 08:07 | some good news : | |
| 08:07 | * no more seg fault | |
| 08:07 | kados | w00t |
| 08:07 | paul | * no more encoding problems (utf8 valid) |
| 08:07 | kados | yay! |
| 08:07 | paul | * zebraidx works fine |
| 08:07 | now, I just have 2 questions : | |
| 08:07 | zebrasrv is running : | |
| 08:07 | 15:08:25-13/07 [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36 | |
| 08:07 | 15:08:25-13/07 [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-authorities.cfg 1.3.36 | |
| 08:07 | 15:08:25-13/07 [server] Adding dynamic listener on tcp:@:9900 id=1 | |
| 08:08 | 15:08:25-13/07 [server] Adding dynamic listener on tcp:@:2100 id=2 | |
| 08:08 | 15:08:25-13/07 [server] Starting server zebrasrv pid=8156 | |
| 08:08 | kados | right |
| 08:08 | paul | iiuc, the biblio are on 9900 |
| 08:08 | kados | correct |
| 08:08 | paul | I tried yaz-client @9900, then "find delmas" |
| 08:08 | but got : | |
| 08:08 | [109] Database unavailable -- v2 addinfo 'Default' | |
| 08:08 | kados | you need to do: |
| 08:08 | yaz-client | |
| 08:08 | dewey | hmmm... yaz-client is returning results |
| 08:08 | kados | open localhost:9900/biblios |
| 08:09 | paul | why biblios ? |
| 08:09 | kados | ie, specify the database name |
| 08:09 | biblios is the db name | |
| 08:09 | paul | i tried with "biblioserver" |
| 08:09 | kados | in the koha.xml file |
| 08:09 | biblioserver is the 'type' | |
| 08:09 | paul | <server id="biblioserver" listenref="biblioserver"> |
| 08:09 | ? | |
| 08:09 | type ??? | |
| 08:09 | kados | well ... |
| 08:09 | paul | I don't see any "biblios" |
| 08:09 | kados | we have different types of zebra servers |
| 08:09 | no? | |
| 08:10 | <biblioserver>biblios</biblioserver> | |
| 08:10 | paul | oups... |
| 08:10 | sorry. i missed it ! | |
| 08:10 | strange to have the name here and not in <server> tag | |
| 08:10 | kados | well ... |
| 08:11 | you should have seen how complicated it was before we switched to an xml conf file :-) | |
| 08:11 | paul | still |
| 08:11 | kados | it had to be put in 4 places :-) |
| 08:11 | paul | [109] Database unavailable -- v2 addinfo 'biblios' |
| 08:11 | kados | hmmm |
| 08:11 | you started zebra how? | |
| 08:11 | paul | xml is great, I just have to learn how it is written |
| 08:11 | kados | zebrasrv -f /path/to/koha.xml |
| 08:11 | ? | |
| 08:11 | paul | zebrasrv -f etc/koha.xml |
| 08:11 | kados | hmmm |
| 08:11 | when you indexed your records? | |
| 08:11 | you specified -d biblios ? | |
| 08:12 | maybe the docs are wrong here | |
| 08:12 | paul | zebraidx -g iso2709 -c /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg update export.unique/ |
| 08:12 | ok, I didn't understand the -d biblioplubin | |
| 08:14 | kados | updated |
| 08:15 | paul | how can I drop the DB to rebuild it ? |
| 08:15 | kados | zebraidx init |
| 08:15 | paul | zebraidx -c etc/koha.xml drop seems useless |
| 08:15 | kados | but you don't need to |
| 08:15 | koha.xml isn't the right config for -c | |
| 08:15 | zebra-biblios.cfg is | |
| 08:15 | anyway I don't think the drop command works | |
| 08:16 | but init does the same thing | |
| 08:16 | but you don't need to do that | |
| 08:16 | if you simply update again it will fix everything | |
| 08:16 | (I think) | |
| 08:16 | (though you may still have the old db there ...) | |
| 08:16 | anyway, init will do it | |
| 08:17 | it clears out the registers | |
| 08:17 | paul | zebraidx -g iso2709 -d biblios -c /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg update export.unique/ |
| 08:17 | kados | looks good |
| 08:19 | paul | 15:20:16-13/07 zebraidx(8221) [log] Records: 1 i/u/d 0/1/0 |
| 08:20 | means 1 record has been inserted, right ? | |
| 08:21 | kados | yes I think so |
| 08:21 | -s will give you all the gritty details while you're indexing | |
| 08:23 | paul | then : |
| 08:23 | [paulportable zebraplugin]$ zebrasrv -f etc/koha.xml |
|
| 08:23 | 15:23:36-13/07 [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36 | |
| 08:23 | 15:23:36-13/07 [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-authorities.cfg 1.3.36 | |
| 08:23 | 15:23:36-13/07 [server] Adding dynamic listener on tcp:@:9900 id=1 | |
| 08:23 | 15:23:36-13/07 [server] Adding dynamic listener on tcp:@:2100 id=2 | |
| 08:23 | 15:23:36-13/07 [server] Starting server zebrasrv pid=8240 | |
| 08:23 | zebrasrv is running fine | |
| 08:23 | then yaz-client to query : | |
| 08:23 | yaz-client localhost:9900/biblios | |
| 08:23 | Connecting...OK. | |
| 08:24 | Sent initrequest. | |
| 08:24 | Connection accepted by v3 target. | |
| 08:24 | ID : 81 | |
| 08:24 | Name : Zebra Information Server/GFS/YAZ | |
| 08:24 | Version: Zebra 1.3.36/1.86/2.1.22 | |
| 08:24 | Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets | |
| 08:24 | Elapsed: 0.001979 | |
| 08:24 | Z> find delmas | |
| 08:24 | Sent searchRequest. | |
| 08:24 | Received SearchResponse. | |
| 08:24 | Search was a bloomin' failure. | |
| 08:24 | Number of hits: 0, setno 1 | |
| 08:24 | records returned: 0 | |
| 08:24 | Diagnostic message(s) from database: | |
| 08:24 | [109] Database unavailable -- v2 addinfo 'biblios' | |
| 08:24 | Elapsed: 0.002820 | |
| 08:24 | in zebrasrv logs : | |
| 08:25 | 15:23:56-13/07 zebrasrv(1) [session] Starting session 1 from tcp:127.0.0.1 (pid=8245) | |
| 08:25 | 15:23:56-13/07 zebrasrv(1) [log] User perm for perm.anonymous: r | |
| 08:25 | 15:23:56-13/07 zebrasrv(1) [request] Auth none | |
| 08:25 | 15:23:56-13/07 zebrasrv(1) [request] Init OK - ID:81 Name:YAZ Version:2.1.22 | |
| 08:25 | 15:24:00-13/07 zebrasrv(1) [log] ResultSet '1' | |
| 08:25 | 15:24:00-13/07 zebrasrv(1) [log][app2] zebra_register_open rw = 0 useshadow=0 p=0x8197448,n=,rp=(none) | |
| 08:25 | 15:24:00-13/07 zebrasrv(1) [log] profilePath=.:/usr/local/share/idzebra/tab/:/home/paul/koha.dev/dev_week/zebraplugin/zebradb/biblios/tab/:./tab/ cwd=/home/paul/koha.dev/dev_week/zebraplugin/zebradb/biblios | |
| 08:25 | 15:24:00-13/07 zebrasrv(1) [log] 0 keys, 0 sysnos, rank | |
| 08:25 | 15:24:00-13/07 zebrasrv(1) [log] term="delmas" nn=0 type=void count=0 | |
| 08:25 | 15:24:00-13/07 zebrasrv(1) [log] user/system: 0/0 | |
| 08:25 | 15:24:00-13/07 zebrasrv(1) [request] Search ERROR 109 1 1+0 RPN: @attrset Bib-1 delmas | |
| 08:26 | kados | [109] Database unavailable -- v2 addinfo 'biblios' |
| 08:26 | that's the error | |
| 08:28 | hmmm | |
| 08:28 | paul: in yaz-client try: | |
| 08:28 | base IR-Explain-1 | |
| 08:29 | form sutrs | |
| 08:29 | f @attr exp1 1=1 databaseinfo | |
| 08:30 | s | |
| 08:31 | what is in the 'recordCountActual' line? | |
| 08:31 | paul | databaseinfos with a s ? |
| 08:31 | kados | s eq show |
| 08:31 | paul | hmm... |
| 08:31 | Z> s | |
| 08:31 | Sent presentRequest (1+1). | |
| 08:31 | Diagnostic message(s) from database: | |
| 08:31 | [13] Present request out of range -- v2 addinfo '1' | |
| 08:31 | nextResultSetPosition = 2 | |
| 08:31 | Elapsed: 0.000614 | |
| 08:31 | kados | so you don't have a db |
| 08:32 | did you do a commit ? | |
| 08:32 | (if you have shadows enabled) | |
| 08:32 | paul | no i forgot this too |
| 08:34 | [paulportable zebraplugin]$ zebraidx -g iso2709 -c etc/zebra-biblios.cfg -d biblios commit |
|
| 08:34 | 15:34:25-13/07 zebraidx(8301) [log] zebra_start etc/zebra-biblios.cfg 1.3.36 | |
| 08:34 | 15:34:25-13/07 zebraidx(8301) [log] enabling cache spec=/home/paul/koha.dev/dev_week/zebraplugin/zebradb/biblios/shadow:4G | |
| 08:34 | 15:34:25-13/07 zebraidx(8301) [log] cache_fname = /home/paul/koha.dev/dev_week/zebraplugin/zebradb/biblios/shadow/cache | |
| 08:34 | 15:34:25-13/07 zebraidx(8301) [log] commit start | |
| 08:34 | 15:34:25-13/07 zebraidx(8301) [log][app2] zebra_stop | |
| 08:34 | alleluja !!!! | |
| 08:34 | kados | :-) |
| 08:34 | paul | Z> find delmas |
| 08:34 | Sent searchRequest. | |
| 08:34 | Received SearchResponse. | |
| 08:34 | Search was a success. | |
| 08:34 | Number of hits: 1, setno 1 | |
| 08:34 | SearchResult-1: term=delmas cnt=1 | |
| 08:34 | records returned: 0 | |
| 08:34 | Elapsed: 0.003601 | |
| 08:34 | kados | use s to see the record |
| 08:34 | paul | I see : |
| 08:34 | kados | and opac-zoomsearch.pl should be working right now |
| 08:35 | paul | [biblios]Record type: USmarc |
| 08:35 | 00743 a2200217 4500 | |
| 08:35 | 001 2 | |
| 08:35 | 010 $a 2729843132 $d 190 F | |
| 08:35 | 090 $9 1 $a 1 | |
| 08:35 | 100 $a 1995 y0fre 0103 ba | |
| 08:35 | 101 $a fre | |
| 08:35 | 105 $a y 00 y | |
| 08:35 | 200 1 $a Introduction aux probabilités $b LIVR $f Jean-Pierre DELMAS | |
| 08:35 | 210 $5 Edition Marketing $a Paris $c Edition Marketing $d 1993 | |
| 08:35 | 215 $a 316 p | |
| 08:35 | 225 $a Collection Pédagogique de Télécommunication | |
| 08:35 | 410 $t Collection Pedagogique de Telecommunication | |
| 08:35 | 606 $9 3339 $9 3321 $a MATHEMATIQUES | |
| 08:35 | 700 $9 6659 $a DELMAS, Jean-Pierre | |
| 08:35 | 995 $b CDI $c CDI $e SL $f Non inventorie $j 000498 $o 2 $9 1 | |
| 08:35 | 995 $b CDI $c CDI $e SL $f Non inventorie $j 005643 $o 2 $9 2 | |
| 08:35 | 995 $b CDI $c CDI $e SL $f Non inventorie $j 000527 $o 2 $9 3 | |
| 08:35 | nextResultSetPosition = 2 | |
| 08:35 | which is what I expected. | |
| 08:35 | kados | w00t! |
| 08:35 | paul | e Télé |
| 08:35 | being correct utf8 | |
| 08:35 | (the famous é) | |
| 08:35 | kados | so now we have three zebra experts in #koha :-) |
| 08:35 | paul | I was thinking to unimarc support. |
| 08:36 | kados | yes, it's much needed |
| 08:36 | paul | if I don't mind, the only file I have to modify is record.abs |
| 08:36 | kados | also, I need to put the search subs in opac-zoomsearch.pl into Search.pm |
| 08:36 | bib1.att also | |
| 08:36 | paul | to say melm 200$a author |
| 08:36 | kados | bib1.att has indexing rules for some special searches |
| 08:36 | paul | why bib1.att ? |
| 08:37 | kados | like for searching 008 in marc21 |
| 08:37 | necessary to search by 'date' in marc21 | |
| 08:37 | as it's the only normalized place that date exists | |
| 08:37 | also for format/content/audience | |
| 08:37 | must search fixed fields as well | |
| 08:40 | paul | ok that was my 2nd question ;-) |
| 08:40 | : | |
| 08:40 | I had problems with zebraidx until I commented the 2 following lines : | |
| 08:40 | # melm 008 date-entered-on-file:n:range(data,0,6),date-entered-on-file:s:range(data,0,6),date1:n:range(data,7,4),date1:s:range(data,7,4),date2:n:range(data,11,4),date2:s:range(data,11,4),ff8-22:w:range(data,22,1),language:w:range(data,35,3),ff8-33:w:range(data,33,1),ff8-34:w:range(data,34,1) | |
| 08:40 | # melm 007ff7-00:w:range(data,0,1),ff7-01:w:range(data,1,1),ff7-02:w:range(data,2,1) | |
| 08:40 | it was "Couldn't find att 'date-entered-on-file' in attset" | |
| 08:40 | any explanations ? | |
| 08:40 | kados ? | |
| 08:40 | dewey | hmmm... kados is becoming a true Perl Monger... |
| 08:40 | paul | I had a small disconnect... |
| 08:40 | kados | paul: bib1.att in /usr/share/idzebra/tab does not contain everything that's in record.abs |
| 08:41 | paul | so you modified it ? |
| 08:41 | kados | it should be committed |
| 08:42 | it's in tab dir in zebradb/biblios/ | |
| 08:42 | paul | I must remove the /usr/local/share/idzebra/tab one ? |
| 08:43 | kados | well ... |
| 08:43 | in zebra docs | |
| 08:43 | you can specify location of tab dirs | |
| 08:43 | in the zebra.cfg | |
| 08:43 | (we call it zebra-biblios.cfg) | |
| 08:43 | but I can't get it to work | |
| 08:43 | so I just copy my bib1.att to /usr/local/share/idzebra/tab one | |
| 08:44 | I wrote ID about this but haven't heard back yet | |
| 08:45 | ToinS | kados: how do you spell "recieve" : recieve or receive ? |
| 08:45 | kados | receive |
| 08:45 | ToinS: i before e except after c | |
| 08:45 | ToinS | kados: ok... |
| 08:46 | kados | ToinS: a rule like all others that is broken sometimes in english :-) |
| 08:46 | ToinS: but correct in this case :-) | |
| 08:46 | paul | in french we say "it's the exception that confirms the rule" |
| 08:46 | kados | :-) |
| 08:47 | paul | hello tumer |
| 08:47 | tumer | hi paul how did it go? |
| 08:47 | paul | i'm proud to announce that I just did my 1st zebra search 10mn ago !!! |
| 08:47 | tumer | zebraing? |
| 08:47 | paul | (thx to kados, once again) |
| 08:48 | tumer | what was the sagmentaion error? |
| 08:48 | paul | some missing directories |
| 08:48 | it's a shame that zebra don't exit with any message | |
| 08:49 | tumer | along the way they fixed some other bugs as well |
| 08:49 | what also annoying that it stops indexing in the middle of 150k records just besuase 1 does not have biblionumber | |
| 08:50 | rather tahn saying that it does not and continue with others | |
| 08:50 | well it takes a while to get used to this animal | |
| 08:51 | paul | yep. |
| 08:51 | I also solved some issues with utf8. I think i understand how it works in perl now... | |
| 08:52 | tumer | paul:the additems.pl mentiomns theasurus_category instead of Authority do you still use the thaerus.pl? in france |
| 08:52 | paul | ??? |
| 08:53 | tumer | does the french libraries use the old thasurus structure or the new authorities? |
| 08:54 | paul | no old structure, only the marc authorities |
| 08:55 | tumer | s the line Dopop('../thesaurus_popup.pl?category= ... is obsolete is it? |
| 08:55 | i found this line in additem.pl and dont know what it does | |
| 08:56 | paul | it's useless I think. there is no authorities in items. So it's a very very old copy/paste that hasn't been fixed because it's unused... |
| 08:57 | tumer | k about utf8 you were saying? |
| 08:57 | paul | I have exported one of my base, and it's correct utf8. |
| 08:57 | tumer | do you use the new M:F:X from sourceforge with no problems? |
| 08:57 | paul | I think I finally have understood how it works... |
| 08:58 | I have 0.83 | |
| 08:58 | tumer | the latest version 2.0 MARC::Record |
| 08:59 | yes that version | |
| 08:59 | paul | $VERSION = '2.0'; |
| 08:59 | yes that's what I have. | |
| 08:59 | and I read marcxml from biblioitems, that is utf8 already. | |
| 08:59 | tumer | when i do a search with accented characters in it that serach term and my template gets funny chars in it |
| 09:00 | paul | with a new_from_xml($marcxml,'utf8'), it produces a MARC::Record that seems valid |
| 09:00 | I still haven't tried a search with something accented. | |
| 09:00 | I have to do unimarc.abs 1st | |
| 09:00 | tumer | you know that it still has to be named record.abs |
| 09:02 | so the base could you convert it with alter table? | |
| 09:08 | kados | tumer: how's the authorities integration going? |
| 09:09 | tumer | kados:more involved with items now.I will look into that later |
| 09:09 | kados | k |
| 09:09 | items as in holdings? | |
| 09:09 | tumer | yep |
| 09:09 | kados | you looking at separating bib records and holdings? |
| 09:09 | tumer | and its working |
| 09:10 | kados | wow! |
| 09:10 | that's fantastic news | |
| 09:10 | thd will be thrilled :-) | |
| 09:10 | tumer: what do you think of my original idea to have just one framework system and one editor? | |
| 09:11 | tumer: ie, instead of maintaining three separate table structures and three separate code bases, just create one capable of doing it all? | |
| 09:11 | tumer | well similar framework sytem for all but differnt frameworks offcourse |
| 09:11 | kados | right |
| 09:11 | tumer | i now have a forth |
| 09:11 | kados | yea? |
| 09:12 | tumer | koha_attr |
| 09:12 | kados | what is it for? |
| 09:12 | tumer | maps koha field names to @attr, also needed for a database with no fields in it |
| 09:13 | no more @attr 1=1007 in koha code | |
| 09:13 | that is bound to change since we all use or may use differnt attributes to index | |
| 09:14 | my record.abs is very differnt to yours | |
| 09:14 | but now whatever yo index you can search | |
| 09:14 | kados | hmmm |
| 09:14 | we could do the same thing with ccl.properties | |
| 09:14 | have you looked at the ccl searching? | |
| 09:14 | tumer | you can simply add virtual field names to everything |
| 09:15 | yes but koha specific terms like datelastseen, subscriptionid etc has to be defined for KOHA | |
| 09:16 | now its very flexible | |
| 09:16 | kados | you can just add datelastseen to ccl.properties |
| 09:16 | and define it in terms of an @attr | |
| 09:16 | I added one for series for instance | |
| 09:16 | tumer | but we still need these names in koha |
| 09:17 | kados | series u=5 s=pw |
| 09:17 | u=5 eq 1=5 | |
| 09:17 | tumer | search terms is not my main concern |
| 09:17 | kados | ahh |
| 09:17 | so explain it again? | |
| 09:18 | tumer | mappin marc to koha we use koha database field names |
| 09:18 | kados | ok |
| 09:18 | tumer | author,title,biblionumber etc |
| 09:18 | kados | right |
| 09:18 | tumer | these field now do not exist |
| 09:18 | kados | (though I have some question why we do this now except for special cases) |
| 09:19 | do we need the koha fields except for some like itemtype and biblionumber? | |
| 09:19 | tumer | we well i do not have those fields now |
| 09:19 | but all of koha needs them | |
| 09:19 | kados | hmmm |
| 09:20 | tumer | where to show title where the biblionumber etc |
| 09:20 | if you remember we are multimarc compliant | |
| 09:21 | i cannot say field245 for title | |
| 09:21 | kados | right |
| 09:21 | ok, I see | |
| 09:21 | tumer | i have to say title so that whatever your marc title is you map it |
| 09:22 | kados | I see |
| 09:22 | but in MARC, it's not just one field :-) | |
| 09:22 | tumer | zebra also supports 1=title |
| 09:22 | so we do not need ccl | |
| 09:23 | kados | it's a syntax language that is more user-friendly than cql |
| 09:23 | tumer | well as a preference ok |
| 09:23 | but behind screens pqf | |
| 09:24 | kados | not sure about that |
| 09:24 | display is different than search | |
| 09:24 | we need one way to specify how to display 'title' | |
| 09:24 | and another to specify how to search it | |
| 09:25 | for searching, it is easier to maintain the ccl.properties file IMO | |
| 09:25 | tumer | well what i am descrbing combines them and leaves it to your choice |
| 09:25 | kados | I see |
| 09:25 | tumer | you will see it when i commit |
| 09:25 | kados | but 1=title is quite different than ccl's ti=something |
| 09:25 | ok | |
| 09:25 | tumer | it was just another example of searching |
| 09:26 | koha search engine should support all | |
| 09:26 | kados | agreed |
| 09:27 | tumer | but currently for internal works of koha i need to get record with biblionumber xxx |
| 09:27 | you mapped it to identifier standart | |
| 09:28 | i mapped it to Number-db | |
| 09:28 | what i wrote is flexible to find both | |
| 09:28 | kados | gotcha |
| 09:29 | tumer | so koha language all mapped like ccl |
| 09:29 | kados | tumer: can you start committing your 3.0 stuff to head? I'm concerned that you'll be going in directions that aren't keeping up with what other folks are doing in head |
| 09:30 | tumer: if you keep your stuff in sync at all times it will be much easier in the long run | |
| 09:30 | ie, the goal is for everyone to run off of stock CVS | |
| 09:30 | tumer | that was my concern when i mentioned about all this |
| 09:31 | current head and all this -- very dubious | |
| 09:31 | kados | well you can ignore the zebra stuff chris and I wrote for head |
| 09:31 | in Biblio and Search modules | |
| 09:31 | we've got much better methods now | |
| 09:32 | tumer | well biblio is complete re-write |
| 09:32 | kados | tumer: do you know how to 'merge'? |
| 09:32 | tumer | no |
| 09:32 | kados | tumer: perfect, that's what we need |
| 09:32 | tumer: a complete rewrite was my goal for 3.0 | |
| 09:32 | tumer: with a new API | |
| 09:32 | tumer: but if you don't commit I wont' be able to review what you've done :-) | |
| 09:32 | tumer | weekend |
| 09:33 | kados | tumer: and it will be difficult later to merge your stuff with head |
| 09:33 | fair enough | |
| 09:33 | tumer | i will also write a detailed explanation of it all |
| 09:33 | how do i merge? | |
| 09:34 | kados | in linux: |
| 09:34 | hmmm | |
| 09:35 | not sure if you're not working with two different cvs trees | |
| 09:35 | maybe ask chris | |
| 09:35 | you could use diff/patch | |
| 09:35 | but with a complete rewrite that could get messy | |
| 09:36 | yay | |
| 09:36 | paul | oops... |
| 09:36 | 16:36:23-13/07 zebraidx(8760) [warn] Record didn't contain match fields in (bib1,Identifier-standard) | |
| 09:36 | 16:36:23-13/07 zebraidx(8760) [warn] Bad match criteria | |
| 09:36 | kados | missing 090 :-) |
| 09:36 | paul | means 1 090 is missing right ? |
| 09:36 | tumer | right or reading wrong file |
| 09:36 | kados | for me, the missing090 script doesn't always work 100% |
| 09:37 | because i think export.pl sometimes doesn't assign a 090 field | |
| 09:37 | in rel_2_2 code | |
| 09:37 | I haven't investigated | |
| 09:37 | paul | I probably have done things in a wrong order. |
| 09:37 | kados | what I do is pre-process records |
| 09:37 | ahh, that might do it too :-) | |
| 09:37 | paul | + i'm reading from marcxml |
| 09:37 | and not from iso2709 | |
| 09:37 | kados | hmmm |
| 09:37 | why? | |
| 09:37 | tumer | paul wont work |
| 09:38 | paul | (i mean i've modified MARCgetbiblio to read biblioitems.marcxml instead of biblioitems.marc |
| 09:38 | kados | why? |
| 09:38 | paul | because biblioitems.marc is not correctly encoded |
| 09:38 | tumer | they all have to be single xml records then |
| 09:38 | paul | where marcxml is OK |
| 09:38 | kados | hmmm |
| 09:39 | tumer | paul are they single records each? |
| 09:39 | kados | maybe we need a syspref for this |
| 09:39 | paul | tumer: yes they are. |
| 09:39 | kados | 'whether to store natively in xml or marc' |
| 09:39 | tumer | then you do not use -g flag then |
| 09:39 | paul | but biblioitems.marc is a blob, so the move to utf8 in mysql changed nothing |
| 09:40 | where marcxml is text, so the move iso => utf8 worked fine | |
| 09:40 | (my db was latin1, declared as latin1) | |
| 09:41 | kados | interesting |
| 09:41 | tumer | i was trying to say that latin1 to utf8 is up the ladder and should work |
| 09:41 | kados | how did you populate biblioitems.marc? |
| 09:42 | paul | with updatedatabase |
| 09:42 | tumer | paul your zebraidx line is it correct? no -g flag? |
| 09:42 | kados | tumer: he's not using zebraidx with xml |
| 09:42 | paul | that reads marc_subfield_table |
| 09:44 | tumer: I use zebraidx with iso2709 datas. but the iso2709 comes from export/export.pl, that uses Biblio.pm/MARCgetbiblio. and I have modified MARCgetbiblio to read data from biblioitems.marcxml instead of biblioitems.marc | |
| 09:44 | that solves my encoding problems, but it seems MARCmodbiblio don't handle marcxml | |
| 09:44 | tumer | paul:if you index like that it will take hours |
| 09:44 | paul | ??? |
| 09:44 | tumer | export your xml as iso |
| 09:45 | paul | I create a single iso2709 file, that can't be faster isn't it ? |
| 09:45 | tumer | is01709 and zebraidx on command line |
| 09:45 | 1000 times faster | |
| 09:45 | paul | yep, that's what I do. |
| 09:45 | tumer | great |
| 09:45 | paul | but I explain how I generate my iso2709 |
| 09:47 | tumer | if you have ugraded your latin1 base to utf8 when all the data was text then created a marc blob and put them in it it should |
| 09:47 | do the same thing | |
| 09:48 | paul | yes, but I ran missing090 AFTER |
| 09:48 | so, marcxml has not been updated I suspect | |
| 09:48 | tumer | why doe that chane encoding? |
| 09:49 | paul | ??? |
| 09:49 | tumer | why does it change encoding? |
| 09:49 | all that does is to chech 090c | |
| 09:49 | paul | it does not. it's just that I now have utf8 recognized as utf8 |
| 10:26 | kados | morning owen |
| 10:26 | owen | Hi kados |
| 10:26 | How's it going? | |
| 10:28 | kados | pretty good |
| 10:28 | having trouble focusing today :-) | |
| 10:29 | paul: still there? | |
| 10:29 | dewey | there is a minor diff in <div>s, that I missed |
| 10:29 | owen | dewey, what on earth are you talking about? |
| 10:29 | dewey | owen: i haven't a clue |
| 10:30 | owen | That's what I thought |
| 10:30 | kados | paul: I took a look at adding a wrapper for the old search API yesterday |
| 10:30 | paul: before I delve deeply into it, I wanted to give you the opportunity | |
| 10:30 | paul: since you are more familiar with the old API :-) | |
| 10:31 | owen | kados: is that why searching isn't working in the intranet? |
| 10:31 | kados | there is a new search API in dev_week |
| 10:31 | opac-zoomsearch.pl uses it | |
| 10:32 | but the old search API is used everywhere else | |
| 10:32 | owen | So the old search doesn't work at all anymore in dev_week |
| 10:32 | kados | so yes, that's why :-) |
| 10:32 | not currently | |
| 10:32 | but it would be pretty simple to create a wrapper | |
| 10:32 | for the new search | |
| 10:32 | so that the old search did work | |
| 10:32 | all it would have to do is translate the old API to ccl (very simple) | |
| 10:33 | and retrieve the results in the old style | |
| 10:33 | (also pretty simple) | |
| 10:33 | I could probably get that done today if paul doesn't want to do it | |
| 10:33 | the other option we have | |
| 10:33 | is to completely overhaul all the old search references in dev_week | |
| 10:34 | update them to use ccl | |
| 10:34 | that would probably be more work | |
| 10:34 | before fully integrating the search | |
| 10:35 | to avoid extra wrok :-) | |
| 10:35 | work even | |
| 10:37 | paul | kados : it's almost 6PM here, and i'm still investigating zbera migration. |
| 10:37 | so if you have enough time to work on a wrapper, feel free. | |
| 10:38 | kados | paul: ok, I'll do it and commit this afternoon |
| 10:38 | paul | kados & tumer : is there a way to say to M:F:X something like "ignore this if it is illegal". |
| 10:38 | kados | paul: and release 2.3. |
| 10:38 | 1 | |
| 10:38 | paul | because i've some records with a subfieldcode='>' |
| 10:38 | kados | yes |
| 10:38 | paul | that is illegal : |
| 10:38 | ! | |
| 10:38 | kados | MARC::Charset->ignore_errors(1); |
| 10:38 | ahh | |
| 10:38 | might not work for those cases | |
| 10:39 | tumer | not that i know only ignore encoding errrors |
| 10:39 | paul | yep, it's not an encoding problem |
| 10:39 | kados | you need to preprocess the records |
| 10:39 | and fix those codes | |
| 10:39 | paul | the problem being i have sometimes a capital letter too... |
| 10:39 | kados | ouch |
| 10:40 | paul | but OK, i'll try a regexp to remove everything that is not a..z0..9 |
| 10:41 | kados | owen: have we tested opacnav recently? is it working? |
| 10:41 | owen | I don't know... did you work on it? |
| 10:41 | kados | I think it was a db prob, it should be fixed |
| 10:48 | owen | Still broken. But let's not mess with it right now on the slim chance that someone's trying it out |
| 10:50 | kados: got a question for you about members. | |
| 10:51 | kados | sure |
| 10:51 | owen | Previously we'd tried to mess with how we used the members database. |
| 10:51 | We decided we'd use the street address field as "mailing address line 2" | |
| 10:51 | I think it was working for a long time and got broken in the last upgrade. | |
| 10:51 | kados | hmmm |
| 10:51 | owen | Doing it that way requires the template to be customized |
| 10:52 | What we /really/ need is an actual field for "mailing address line 2" and "street address line 2" | |
| 10:52 | kados | I'm not sure exactly what the requirements are |
| 10:53 | we have a mailing address currently | |
| 10:54 | tnb: ok I see Address Line 1 and Address Line 2 | |
| 10:55 | woops :-) | |
| 10:55 | in the liblime demo | |
| 10:55 | I see it differently | |
| 10:55 | http://koha.liblime.com/cgi-bi[…]=9068&modify=edit | |
| 10:56 | there are two address options there | |
| 10:56 | mailing and street | |
| 10:56 | owen: so what is npl trying to do? | |
| 10:57 | owen | I'd like to see 2 lines available for each address: Address Line 1 and Address Line 2. |
| 10:57 | So you could do: Address Line 1: P.O. Box 235, Address Line 2: 5 Jacobs Street | |
| 10:57 | kados | I see |
| 10:58 | is it the first line from each? | |
| 10:58 | is that the customization? | |
| 10:58 | owen | NPL's Address Line 2 is actually mapped to Street Address Address, I think. |
| 10:58 | At least it's supposed to be. | |
| 10:58 | kados | yea, I figured |
| 10:58 | hmmm | |
| 10:59 | we'd have to add new columns to the db | |
| 11:00 | owen | Yeah, I know |
| 11:00 | paul | owen/kados : do you want me to summarize what is in head about members management ? |
| 11:00 | (from SAN) | |
| 11:00 | owen | sure, paul |
| 11:00 | paul | SAN (& me) have modified the table structure to have 2 addresses. |
| 11:01 | they are called : | |
| 11:02 | streetnumber / streetype / address / address_2 / city / zipcode / email / phone / | |
| 11:02 | and the 2nd one is called : B_... | |
| 11:03 | if you look in head/updater/updatedatabase you'll see which field has been renamed as which field | |
| 11:03 | we tried to do things with care, so no data should be lost in the process. | |
| 11:03 | does that make sense ? | |
| 11:04 | owen | Yes |
| 11:04 | paul | streetnumber & streettype appears only if the (new in head) streettype table is filled. |
| 11:05 | SAN want to have a list for rue, avenue, boulevard, chemin,... | |
| 11:05 | I don't think most libraries will be interested by this feature. | |
| 11:07 | owen | So kados it sounds like it's something I should just wait for. |
| 11:13 | kados | owen: ok :-) |
| 11:17 | owen | I guess if my hack hasn't even been working for at least a year there's no sense in piling more on... We'll just have to try to grab the existing data properly when 3.0 rolls around. |
| 11:17 | kados | sounds like a plan |
| 11:17 | lets add it to a list somewhere so I don't forget about it :-) | |
| 11:18 | paul | zebraidx on my 1x000 biblios, seems to work fine with my updated xml datas... |
| 11:18 | kados | owen: I didn't see any commits to dev_week yesterday |
| 11:18 | paul: great! | |
| 11:19 | owen: did you make some changes to the tempates/scripts? | |
| 11:19 | owen | kados: I have several things saved locally that I need to commit today |
| 11:19 | paul | what kind of commits can I do on dew_week ? |
| 11:19 | (what I modified may break some things...) | |
| 11:19 | kados | what did you modify paul? |
| 11:20 | paul | in Biblio.pm, I did some modifs to support marcxml instead of marc (in biblioitems) |
| 11:20 | kados | ok, lets wrap that in a syspref |
| 11:20 | paul | do you confirm you also have a biblioitems.marcxml field ? |
| 11:20 | kados | yes |
| 11:20 | paul | 18:20:43-13/07 zebraidx(9415) [log] Records: 10867 i/u/d 9885/982/0 |
| 11:20 | 18:20:43-13/07 zebraidx(9415) [log] user/system: 10839/115 | |
| 11:20 | kados | but I don't use it |
| 11:20 | paul | 18:20:43-13/07 zebraidx(9415) [log][app2] zebra_stop |
| 11:20 | 18:20:43-13/07 zebraidx(9415) [log] zebraidx times: 122.41 108.39 1.15 | |
| 11:20 | done ;-) | |
| 11:20 | kados | :-) |
| 11:20 | very fast :-) | |
| 11:20 | I don't use marcxml | |
| 11:21 | paul | very fast ? I work on my laptop, so it's a 5400TPM ide disk. |
| 11:21 | will be faster on my SCSI ;-) | |
| 11:21 | kados | :-) |
| 11:21 | don't forget to commit too :-) | |
| 11:23 | tumer[A] | the only reason i had mzrcxml in dev week was to be able to do keyword seraches in SQL as backup |
| 11:23 | i am dropping marcxml in 3 is that ok | |
| 11:23 | paul | for example, removing invalid subfieldcodes was really easy |
| 11:24 | just 1 regexp you can't do with iso2709 | |
| 11:24 | thd | paul: are your records with subfield code = '<' corrupted or had they been originally encoded somewhere using punctuation codes for subfield names? |
| 11:24 | tumer[A] | yes but the db grows very fat with them while iso footprint is small |
| 11:25 | paul | kados, your opinion ? |
| 11:25 | hello thd. | |
| 11:25 | kados | I think we make a syspref :-) |
| 11:25 | thd | hello paul |
| 11:25 | tumer[A] | one advantage is that you dont have to convert to xml when updating zebra |
| 11:25 | paul | It may have been a very old bug, from previous ILS of the library |
| 11:25 | or there is a bug in additem.pl | |
| 11:26 | tumer[A] | since we create marc from xml |
| 11:26 | kados | bbl |
| 11:27 | paul | kados : tomorrow is our national day |
| 11:27 | so we won't be here. | |
| 11:27 | thd | paul: was that your answer about punctuation symbols used for subfield codes? |
| 11:27 | kados | paul: ok |
| 11:27 | paul | so see you on monday. |
| 11:27 | thd : yep | |
| 11:27 | kados | have a good weekend paul |
| 11:27 | owen | paul: will you have fireworks? |
| 11:27 | paul | (on tuesday/wednesday, i'll be in nantes) |
| 11:27 | owen: of course. | |
| 11:28 | (and it's very very dangerous, as marseille is so wet) | |
| 11:28 | a big risk of fire inside the city. | |
| 11:28 | every year the fire dept does many many work to prevent any major problem | |
| 11:31 | thd | paul: what problem did punctuation symbols for subfield codes create? I ask especially because some large union catalogues and other systems which would be good copy cataloguing targets use punctuation to add extra subfields to extend the standard. The large union catalogues are big enough to have their practises create an effective standard. |
| 11:37 | paul | thd : MARC::File::XML die miserabily if you have an invalid subfield. |
| 11:37 | and thus the script that uses M::F::X | |
| 11:37 | it seems there is no option to continue even with an error. | |
| 11:38 | thd | paul: we need M::F::X to work as well as MARC::Record so that it is safe to use. |
| 11:40 | paul: I am going to ask Ed Summers to consider making some code changes in the next few days. | |
| 11:40 | paul | I just wrote something on perl4lib ;-) |
| 11:42 | ok, time to leave for me | |
| 11:42 | bye bye guys, i'm very happy with my day ;-) | |
| 11:42 | (although very hot...) | |
| 11:44 | thd | goodbye paul |
← Previous day | Today | Next day → | Search | Index
