IRC log for #koha, 2006-08-06

← Previous day | Today | Search | Index

All times shown according to UTC.

Time Nick Message
12:00 kados so there are two unreleated problems it seems:
12:00 one is with your local computer
12:00 one is with your zebra/koha system
12:01 paul if you could connect to my box, it would be useful
12:01 kados what is the link again?
12:02 paul http://o19.bureau.paulpoulain.[…]RCdetail.pl?bib=5
12:02 (opac)
12:02 or :
12:02 http://i19.bureau.paulpoulain.[…]RCdetail.pl?bib=1
12:02 (login test/test)
12:02 kados server not found
12:02 :(
12:02 Firefox can't find the server at o19.bureau.paulpoulain.com.
12:03 paul does that one work better :
12:03 http://i5.bureau.paulpoulain.c[…]/koha/mainpage.pl
12:03 ?
12:03 kados no
12:03 paul if yes, then i've a dns problem is o19
12:03 kados in traceroute I get 'host name not found'
12:03 so it's definitely dns prob
12:03 paul bureau.paulpoulain.com is better ?
12:03 kados no
12:04 paul wow... same thing for me... host unknown
12:04 (when I try to connect from a distant server
12:04 )
12:04 www.koha-fr.org works ?
12:05 kados yes
12:05 paul mmm... hdl around ?
12:09 ok, it's time to leave for dinner. + i've a headhacke.
12:09 have a good day kados & see you tomorrow.
12:09 kados you too
12:09 paul (+ a last note : when displaying result list from zebra, i don't have any problem)
12:10 kados in the browser?
12:10 paul yep
12:10 kados with the same font?
12:10 paul yep, of course
12:10 kados strange
12:10 paul (still dev_week from CVS)
12:10 kados then maybe it's mysql again :-)
12:10 (where can you see results list from zebra in dev-week?
12:11 (afaik it's all display from mysql)
12:11 paul opac-zoomsearch => search for something with X results
12:11 nope, this one is zebra, if i don't mind
12:11 kados ahh ... you're right
12:12 paul_away this time, bye bye
13:05 kados owen: you around?
13:05 owen: adding a new subscription, I don't seem to be able to receive issues for it
13:05 owen I'm getting full-serial-issues squared away finally I think.
13:05 kados cool
13:05 http://wipokoha.liblime.com/cg[…]?subscriptionid=1
13:08 owen No errors or anything, I take it?
13:08 kados nope ... the form is just blank
13:08 hang on, looks like you committed something to statecollection this morning
13:09 nope ...
13:09 only the Manual Issue option is showing up
13:09 there are no auto-generated issues showing up :(
13:10 :-)
13:10 owen I'll finish with full-serial-issues.tmpl and then tackle that.
13:10 kados k
13:11 location not getting filled in either
13:11 on that page
13:54 owen Okay kados, I committed an updated full-serial-issues.tmpl as well as updated intranet.css and colors.css. That page is still giving a javascript error, but looks and fucntions much better than before
13:55 kados owen: looks like my problem with the statecollection was due to not updateing the datbase
13:55 http://wipokoha.liblime.com/cg[…]?subscriptionid=1
14:01 owen Great
14:04 kados: did you ever get a chance to work on the OPAC facets so that the template could output stuff as a list?
17:24 thd kados: are you there?
17:25 kados thd: yes
17:25 thd: alwys :-)
17:25 thd kados: are you always awake too?
17:26 kados thd: almost always :-)
17:26 thd kados: I hope my reorganised koha-zebra list question is clearer
17:27 kados: I was expecting some answer of just do it like we told paul to do it in the thread you quoted from February
17:28 kados you mean expecting from ID?
17:28 thd yes
17:29 kados: paul's character display problems seem like X-windows problems with some fonts if he had a different result when changing fonts in CSS
17:31 kados: Even if MS Windows and OSX work fine we should not have paul and I wondering if something else is wrong every time the characters do not look correct.
17:32 kados: all the standard CSS should use only the fonts which have the fewest problems in the least capable environment
17:34 kados: paul should be worrying about real encoding issues instead of chasing CSS phantoms
17:40 kados: are you unconcerned because it does not happen on your system and X-windows is a minority system that most of your prospective customers are not running?
17:43 kados: for proportional fonts I trust nothing but Arial, Helvetica, Geneva, sans-serif on X-windows
17:44 kados I was having trouble with Arial on OSX
17:44 using firefox
17:44 (safari was ok)
17:49 thd kados: well then Helvetica, Geneva, sans-serif, Arial
17:49 kados: if we put the somewhat problematic ones last, that will be an improvement
17:51 in case the environment is MS Windows where at least fonts work but maybe some fonts will not be present
17:56 kados: if I remember, only 2 monospaced fonts worked for me in X-Windows.  and none was in the Koha list
18:05 kados thd: we should at some point, try to compile a scratchpad of which fonts work well with which language scripts and encodings on which operating systems and browsers :-)
18:06 thd kados: meanwhile paul is chasing phantom problems instead of real ones
18:06 kados phantom problems?
18:06 I think the problem he was having was actually a font problem
18:06 because it was only happening for utf8 combining characters
18:07 thd kados: exactly, a phantom problem, not a real underlying problem
18:07 kados ahh, right
18:08 thd kados: paul and I should not wonder if something is wrong every time the characters do display correctly
18:08 kados: if the characters do not display correctly we should know that something is wrong
18:09 at a lower level than fonts
18:09 Burgwork somebody was looking for me?
18:09 kados not me I don't think
18:10 thd maybe in another channel Burgwork
18:21 kados: testing monospaced fonts again, the 2 named Courier fonts in Koha CSS do not work for me
18:25 Burgwork thd, kados hmm, ok. Might have been a mistaken ping, or somebody looking for me at home
18:27 thd kados: I think that monospaced should be FreeMono, "Courier New", Courier, monospace   None of those but FreeMono works for me
05:34 slef can someone boot From: "KISS Madeleine \(OPOCE\)" <Madeleine.Kiss@cec.eu.int>
05:34 off the lists until next month... there's a dumb auto-reply replying to the From address of every list email
06:38 osmoze hello #koha
06:44 just one question, who are the designer for koha ? (tee-shirt and logo ?)
07:02 paul hello osmoze. It's katipo
07:02 kados morning paul
07:02 paul hello kados
07:02 seems I still have DNS problems.
07:03 kados strange
07:03 paul but we can solve them easily for you :
07:03 in etc/hosts, add
07:03 213.41.245.208 o19.bureau.paulpoulain.com
07:03 and you should be able to reach
07:04 http://o19.bureau.paulpoulain.[…]RCdetail.pl?bib=1
07:04 and tell me if conbined chars are OK for you
07:05 kados 200$a is not ok
07:05 a titre propre  Dogmatique chr�etienne
07:05 paul ???
07:05 kados same with 606 $a
07:05 paul I just see dogmatique ch,tienne
07:05 kados I see question marks
07:05 I see:
07:05 osmoze hi paul and kados
07:06 kados sujet Th?eologie dogmatique
07:06 osmoze paul, c est katipo, mais il y a une personne en particulier ? c est pour une suggestion de logo
07:06 paul kados : who wrote koha logo at katipo ? do you know ?
07:06 kados rachel
07:07 paul I see the same thing for chr,tienne & th,ologie
07:07 a kind of comma
07:07 you see different things ?
07:07 (maybe you could screen copy your page & show it to me)
07:08 osmoze rachel is rach ?
07:09 paul you can see mine at 213.41.245.208/kados.png
07:09 osmoze: yes.
07:10 kados http://kados.org/desktop.png
07:10 paul ok, we have the same thing, except that i see a kind of comma instead of a ?
07:10 a question :
07:11 when you pasted the "titre propre", I got many A+square as é/?/,
07:11 osmoze ok, in fact, i ll suggest a logo like http://www.debian.org/logos/button-mini.png for liking in a blog, so purhups it's already exist
07:11 paul how did you do your copy/paste ?
07:11 (I have 11 A+square)
07:11 kados just with copy paste of OSX :-)
07:12 right
07:12 it's because this characters is a combining character
07:12 paul but it should appear correctly on your osX isn't it ?
07:12 (or should I update the stylesheet ?
07:12 )
07:12 kados well ... not always
07:13 let me check with safari
07:13 ok, I can verify, it's not OK
07:13 in safari I also get a question mark
07:13 paul which means what ?
07:14 kados it means that perhaps the encoding is still wrong
07:14 and it might not just be the font
07:17 paul very nice to read :-(
07:17 kados paul: one thing that troubles me about this
07:17 in UTF8, combining characters are written:
07:17 e/
07:17 but it seems in these exmples, the ? is _before_ the base character
07:17 as in /e
07:18 / standing for acute above the e
07:18 in MARC8 it would be written /e
07:18 so I wonder did you run these records through MARC::* before displaying them?
07:19 paul yep, as MARCgetrecord read mySQL & uses MARC::Record
07:20 (note that this database comes from Koha 2.2, and previously it was a marc21 database on a proprietary software, so maybe encoding is wrong since the migration to koha, but it worked correctly until now)
07:21 kados paul: sometimes a character can be correctly encoded and still show up as a ?
07:21 paul: http://www.mezzoblue.com/archi[…]005/07/25/glyphs/
07:21 paul: it was marc21 and you converted to unimarc?
07:21 paul yep
07:21 kados why?
07:21 paul (but I did nothing specific for encoding
07:21 because the library resquested it ;-)
07:22 kados very strange
07:22 paul (it was specified in RFP, I did not suggest anything)
07:22 kados it's a french library?
07:22 paul of course
07:22 (Institut Protestant de Théologie)
07:22 kados wow, I thought most french libraries were anticipating moving from unimarc to usmarc :-)
07:22 paul where did you get this idea ?
07:23 kados from dev week :-)
07:23 so you've done nothing specific for encoding?
07:23 of these records?
07:24 did you at least convert to UTF8 using MARC::Charset?
07:24 (otherwise, how could it be done ... marc21 only has two encodings, MARC8 and UTF8)
07:24 paul no, I strictly did nothing.
07:25 just read the iso file, put in in MARC::Record, get subfield by subfield a rearrange them in an UNIMARC way
07:25 kados ok, so you have MARC8 data of course :-)
07:25 paul (but i don't know what was the real encoding in previous ils)
07:26 but why does it works with koha 2.2 ?
07:26 kados what is leader / 09?
07:26 paul mmm... i rebuilded the leader, it's no more the marc21 one.
07:27 kados if leader / 09 is 'a', it means UNICODE, otherwise it means MARC8
07:27 paul but for instance, it's :  005767nam 22001813  4500
07:27 kados so , you have MARC8 ... or else you don't have USMARC records to start with :-)
07:28 paul (it's the space between m and 2 right ?)
07:28 kados no, that's position 7
07:28 paul the 'a' ?
07:28 kados ahh ... yes
07:28 sorry ... between m and 2
07:29 marc8 and latin1 share some codepoints
07:29 paul ubt it's unimarc record, so MARC::Record should read :
07:29 100    _a        d        u  y0frey50
07:29 the 50 meaning it's utf8 data
07:29 (it's 100$a)
07:30 kados but it's _not_ utf8 data in the marc21 record!
07:30 it can't be ... unless it's not MARC21
07:30 it's really quite simple
07:31 with MARC21, you either have MARC8 or UTF8
07:31 and you look in the leader to see which one you have
07:31 it's MARC8 in your case
07:32 we know this for two reasons:
07:32 1. leader position 9 is ' '
07:32 2. your combining characters are structured as /e instead of e/
07:32 dewey Hmm.  No matches for that, kados.
07:33 kados paul: so you must convert from MARC8 to UTF8 using (probably) MARC::*
07:33 or ... MARCEdit does it too
07:33 hey owen
07:33 owen Hi
07:33 dewey what's up, owen
07:33 owen dewey: you're chatty this morning!
07:33 dewey owen: huh?
07:33 paul and how do I that ? (convert from marc8 to utf8) ?
07:34 (MARCedit seems to require windows, which I don't have :-( )
07:34 kados here is one example: http://liblime.com/public/roundtrip.pl
07:35 but ... it's just an example
07:35 sometimes you need to also check the leader length
07:36 also check Opening Files here: http://wiki.koha.org/doku.php?[…]pad#opening_files
07:36 paul strange, wiki.koha.org still don't work for me :(
07:37 mmm... really strange ...
07:37 works on konqueror but not on firefox !
07:37 kados hehe
07:37 I bet firefox has cached something wrong
07:38 paul I wrote the trick for opening_file, so I knew it ;-)
07:38 kados hehe
07:38 paul my problem with your script is that I don't have a iso file, I just have a koha database.
07:39 kados so you have to use export.pl
07:39 paul but export with or without :utf8 when opening the file ???
07:40 kados in fact, I don't think you can do it properly
07:40 it's the fault of Koha
07:41 I tried many hours to fix NPL's encoding probs
07:41 but mysql doesn't understand MARC8
07:41 and if your table defs are set to latin1
07:42 the characters will be mangled when you export I think
07:42 especially the combining characters
07:45 paul OK, i've exported the 1st biblio only. once with utf8, once without
07:45 the .utf8 is 578 bytes long, the leader says 578 bytes and I can see \xc3\xa2 for é
07:46 kados Dogmatique chr<e2>etienne
07:46 paul the .latin is 576 bytes long, the leader still says 578, and I see \xe2\x65 (âe)
07:46 kados that is the code of the character
07:46 00e2
07:49 paul:                      <code>
07:49                                <isCombining>true</isCombining>
07:49                                <marc>E2</marc>
07:49                                <ucs>0301</ucs>
07:49                                <utf-8>CC81</utf-8>
07:49                                <name>ACUTE / COMBINING ACUTE ACCENT (Oxia)</name>
07:49 so the MARC is E2 for Acute / Combining Acute Accent
07:49 meaning you have MARC8 data :-)
07:50 paul but why is it shown correctly with koha 2.2 ???
07:50 (unimarc, default templates, french, iso8859-1)
07:51 kados where is the catalog?
07:51 I can look at the same record there?
07:52 paul you mean the real life catalogue ?
07:52 kados yes
07:52 paul http://catalogue.iptheologie.f[…]koha/opac-main.pl
07:55 kados it's the same code point there
07:55 paul which means it's marc21 ?
07:55 kados yes, it seems so
07:55 paul so why the hell does it work ???
07:55 kados :-)
07:56 very interesting question :-)
07:57 wait ...
07:57 it's not the same
07:57 before it was E2, now it's E9
07:57 and it's not a combining character!
07:58 latin small letter e with acute, U+00E9 ISOlat1
07:58 paul so, something between my 2.2. and my dev_week transformed the E9 in something else...
07:58 kados so it's not MARC8 :-) and not MARC21 either :-)
07:58 yes, it seems so
07:58 paul what I did :
07:59 - copy the 2.2 database (mysqldump => import)
07:59 kados do you have the mysqldump?
07:59 paul - alter table biblioitems ... collate utf8
07:59 kados check the codepoint in there
07:59 paul mmm... I can do it again, as i have the 2.2 on my computer
08:01 kados what is very strange to me
08:01 paul in the dump of marc_subfield_table, I can see :
08:01 kados is how did latin1 data turn into valid marc8 data
08:01 paul \xc3\xa9
08:02 but that does not mean anything, as the dump alway produces utf8...
08:02 so, how can I check ?
08:02 kados you can view the hex in mysql itself
08:02 paul how ?
08:02 kados http://mysql.he.net/doc/refman[…]ng-functions.html
08:03 something like:
08:03 paul BIN(subfield_value) ?
08:03 no
08:03 kados SELECT subfieldvalue, HEX(subfieldvalue) from marc_subfield_table where ...;
08:04 maybe even use substr
08:04 to isolate a specific place in subfielfvalue
08:06 paul é = E9
08:06 kados E9 = latin small letter e with acute
08:12 paul ok, i've select hex() on marcxml from dev_week database, and I have C3A9
08:12 for é
08:12 which means something added a C3.
08:12 the question being : who !
08:12 kados ?
08:12 paul oups...
08:12 kados A9?
08:13 paul yes, A9 (and not E9
08:13 kados that's the (c) symbol
08:13 paul it's C3A9, not A9 alone
08:13 kados C3 is latin capital letter A with tilde
08:13 in latin1
08:14 ahh
08:14 paul (how do you know so quickly what means what ? you learned tables by heart ?)
08:14 kados in utf8 C3A9 is LATIN SMALL LETTER E WITH ACUTE
08:14 google :-)
08:14 paul ah, so it's correct utf8 ?
08:14 kados yes
08:15 paul which is a good news isn't it ?
08:16 as we just have to find why it's no more correct utf8 in my browser.
08:16 kados I think so
08:16 paul so it can be : mySQL, perl DBI, MARC::Record, MARC::File::XML
08:16 kados but how did it get to UTF8 from MARC8?
08:17 paul (or my browser, but we eliminated it through your test)
08:17 maybe it was utf8 data already ?
08:17 (in marc21 I mean)
08:17 kados no ... we checked original catalog
08:17 paul mmm... good point...
08:17 kados it's MARC8 to start with
08:17 paul I just ran updatedatabase from head (which translate tables to utf8)
08:18 kados so maybe mysql is MARC8 aware?
08:18 and we just didn't know it?
08:18 paul (and yesterday, i did the text => blob => text manip that you requested)
08:23 kados http://dev.mysql.com/doc/refma[…]set-charsets.html
08:23 mysql doesn't know marc8
08:27 paul mmm... i've dumped in a file what is read from marcxml => it seems that I still get c3a9
08:27 then i've dumped the MARC record made from the xml => it's transformed to e265
08:37 kados paul: here's the problem:
08:37 you have:
08:37 Dogmatique chr<e1>etienne
08:38 s/e1/e2/
08:38 paul <e2> you mean ?
08:38 kados you should have:
08:38 Dogmatique chre<e2>tienne
08:39 paul ???
08:40 kados so this was never touched by MARC::*?
08:40 paul sorry, I didn't understand what you said previously :
08:41 I have <e1> and I should have <e2> ?
08:41 kados no
08:41 paul i have <e2> and I should have <c3><a9>
08:41 right ?
08:41 kados you have <e2>e and you should have e<e2>
08:42 in MARC8, combining characters are written as /e but in UTF8 as e/
08:42 dewey Hmm.  No matches for that, kados.
08:42 paul ??? I thought c3a9 was e with acute ?
08:42 kados hmmm
08:42 paul we said some lines ago that I had correct utf8 in mySQL ?
08:43 kados yes, in mysql
08:43 paul and I still have c3a9 when retreiving the marcxml field.
08:43 kados but I'm looking at the webpage
08:43 paul but when I do new_as_xml, i get e265
08:43 kados 65 is e
08:44 becaues you didn't tell MARC::* that you have UTF-8
08:44 there are two ways to do it:
08:44 1. leader position 9 is a
08:44 2. call as:
08:44 paul ah, OK, I think I begin to understand : you're saying my MARC::Record has been low endian encoded instead of hi endian
08:44 (or something like that)
08:44 kados new_from_xml('UTF-8','UNIMARC');
08:44 yes
08:45 paul you want the bad news ? that's already what I do...
08:45    $record = MARC::Record::new_from_xml( $marcxml,'utf8','UNIMARC' ) if $marcxml;
08:45 kados hmm
08:45 which version of MARC::File::XML ?
08:45 and MARC::Record
08:45 (not tumer's version I hope)
08:46 paul what it tumer version ?
08:46 kados one he posted to koha-devel
08:46 paul mmm... iirc, yes, I commented the decode_utf8 line...
08:46 kados I think only SF version knows about UNIMARC
08:46 paul i've MARC::Record from sf
08:46 $VERSION = '2.0';
08:47 kados MARC::File::Xml from SF?
08:47 paul $VERSION = '0.83';
08:47 kados mike rylander asked you to test many months ago, unimarc support
08:47 because he added this function
08:47 to not touch the encoding if unimarc
08:47 ??
08:48 that's how we got the ,UNIMARC flag in the first place
08:48 so if I were you, I'd post to perl4lib, explain that:
08:48 1. you are using MARC::* from SF
08:49 2. you have a UNIMARC record, encoded as UTF-8 but with no position 9=a
08:49 3. the new_from_xml($record,UTF-8,UNIMARC); isn't working
08:49 paul you mean that with position9=a it should work ?
08:49 (because I could force position 9 to check)
08:49 kados because the reverses the code points for combining characters
08:50 (yes, with position 9=a it will work fine)
08:50 (not that leader/09 is actually 10th position)
08:50 s/not/note/
08:50 paul of course, I remember
08:50 kados I suspect that
08:51 mike rylander's solution for unimarc works perfectly
08:51 except for combining characters
08:52 paul mmm... strange : even if I force a, it is replaced by a space
08:53 (i putted 4 'a', and only 3 are shown, the last one, the 9 being replaced by a space !
08:53 kados did you turn off the unicode flag?
08:53 I mean unimarc flag
08:54 paul I tried without unimarc & without unimarc and utf8
08:54 and none of them work
08:55 kados what are you working with, a file?
08:55 or data from mysql?
08:55 paul data from mysql
08:55 kados and export.pl?
08:55 paul export.pl ? what do you want with export.pl ?
08:55 kados ahh ...
08:55 you are just refreshing your browser page?
08:56 paul yep
08:56 kados try to change in mysql
08:56 the leader
08:56 paul that's what I did ;-)
08:56 kados hehe
08:57 wow, it' quite strange
08:58 are you 100% sure you don't have tumer's MARC::Record?
09:00 paul: did you comment out line 171 in USMARC.pm?
09:00 because I think this is probably the problem :-)
09:01 I have suspected that tumer was wrong about this from the beginning
09:01 paul I commented it, but even without commenting, i have the problem
09:01 kados hmmm
09:01 paul now i have a 100% official MARC::Record package
09:01 (unless i'm missing something)
09:03 kados and you checked the code points in the webpage?
09:03 paul ??
09:03 kados it's still ?e instead of e?
09:03 yes, it is
09:03 strange
09:04 but your encoding is still wrong in leader
09:04 someone is setting it to ' ' ... but who?
09:04 ahh ...
09:04 paul yes, and what is strange is that I abs assure I putted 'a' in mySQL !
09:04 kados you need to tell new_from_xml that you want UTF-8
09:04 paul ahh...
09:05 kados new_from_xml($record,'UTF-8');
09:05 otherwise, it will give you MARC8
09:05 paul reminder :
09:05    $record = MARC::Record::new_from_xml( $marcxml,'UTF-8','UNIMARC' ) if $marcxml;
09:06 kados can you 'warn marcxml'?
09:06 maybe it is converted to marc8 before it becomes $marcxml
09:06 paul I already have saved it in a file.
09:06 kados and?
09:06 paul (/tmplxmldump.iso & /tmp/xmldump.utf)
09:06 .iso being opened without anything
09:07 .utf being open with :utf8
09:07 xmldump.utf has reencoded utf8 : c383c2a9
09:08 xmldump.iso is unchanged : c3a9
09:09 + if I warn the XML, I see in my logs : \xc3\xa9
09:09 so I would vote : $marcxml is correct & new_from_xml did something unexpected
09:10 kados I would remove MARC::File::XMl
09:10 and install new one from SF
09:10 paul OK, let's try it
09:14 kados it's from cvs
09:14 paul OK, that's what is wrong : I never compiled this package from cvs.
09:14 kados cvs -z3 -d:pserver:anonymous@marcpm.cvs.sourceforge.net:/cvsroot/marcpm co -P marc-xml
09:14 paul 100% sure
09:15 kados maybe you can test ,UNIMARC flag also now :-)
09:15 and tell Mike if it works :-)
09:16 ok, I've got to get some breakfast ...
09:16 I will be back in about an hour or so
09:17 paul i'll left in 30mn around
09:17 I may be back in some hours.
09:17 but it seems that this does not fix the problem
09:17 kados ok ... tell me if it works for you
09:17 :(
09:17 paul (& that I already had the same version,even if not from SF)
09:18 kados (the versions are not maintained in SF, only on CPAN)
09:18 (so 0.83 can be several versions)
09:18 so the problem seems to be that:
09:19 koha is turning 'a' in leader to ' '
09:19 before printing to the browser
09:19 and as a result, MARC::File::XMl isn't preserving the order of the combining characters
09:20 just to be safe, try installing MARC::Record from SF too:
09:20 cvs -z3 -d:pserver:anonymous@marcpm.cvs.sourceforge.net:/cvsroot/marcpm co -P marc-record
09:20 cvs -z3 -d:pserver:anonymous@marcpm.cvs.sourceforge.net:/cvsroot/marcpm co -P marc-charset
09:20 for the heck of it too :-)
09:20 ok, now I must go
09:20 good luck paul
09:20 paul OK, bye
09:20 and many many thanks
09:20 kados np
09:21 paul++ :-)
09:26 slef hello

← Previous day | Today | Search | Index

koha1