← Previous day | Today | Next day → | Search | Index
All times shown according to UTC.
Time | Nick | Message |
---|---|---|
15:48 | hdl | kados |
15:49 | I checked his problem and has the same on 226 version for utf-8 management | |
15:49 | taht is, | |
15:51 | if I choose an accented term for 700 or 650 in a bibliographic record from an authority (say égypte for instance), it wil result into a strange encoding when blind-detail-biblio occurs | |
16:27 | kados | hdl: yes? |
16:28 | interesting | |
16:28 | what is blind-detail-biblio? | |
16:30 | hdl | kados : it is the page that fills the bibliographic datas with the Authority input. |
16:31 | kados | hmmm |
16:38 | another encoding problem has turned up | |
16:38 | with the rel_2_2 version | |
16:38 | if you attempt to mysqldump your utf8 records out | |
16:38 | and then import them | |
16:38 | characters are mangled | |
16:38 | accented characters outside the ascii range | |
16:39 | it makes me question the chosen platform ! | |
16:40 | thd | hdl: do you have an example of the problem and an example of what the 'blind detail' page is itself with and without encoding problems? |
16:40 | chris | that wont be a big thing, it'll be the commandline client not supporting utf-8 |
16:40 | thd | kados: you did not seem to have any shortage of hair to me |
16:41 | chris: when would you eve need to use UTF( on the command line? | |
16:41 | chris | Using MySQLDump, make sure to use the '-r <filename>' option, instead of redirecting command-line output to a file ('> <filename>'). Redirection does not capture UTF characters properly, while the -r option does. |
16:41 | Example: | |
16:41 | >mysqldump -r dbdump.sql | |
16:41 | Using MySQL, source the sql script. | |
16:41 | Example: | |
16:41 | MySQL>source dbdump.sql | |
16:41 | thd | s/UTF(/UTF8/ |
16:42 | chris: but I see no UTF8 characters in your command | |
16:43 | chris | there arent |
16:44 | im saying that the redirect to a file doesnt work with utf8 | |
16:44 | use -r instead | |
16:44 | thd | chris: I see, I missed your -r option posting while I was typing |
16:44 | hdl | thd: Try on i3.hdlaurent.paulpoulain.com |
16:45 | and you will be able to see a problem when creating a new Biblio and getting accented authorities | |
16:47 | chris | mysql -e 'source /patch-to-backup/backup-file.sql' |
16:47 | to load it back in | |
16:48 | thd | hdl: I am uncertain just what I should do to observe this problem |
16:49 | hdl | try and click on the ... on field 632 then search egypt* |
16:50 | and select the first accented value. | |
16:50 | kados | chris: thx for the tips, I'll try that |
16:50 | hdl | chris : thx too |
16:51 | chris | no problem ive done it with maori data and its worked .. so hopefully it should fix the problems you are having with import/export too |
16:55 | kados | chris: I tried the -r option, but when I do 'cat koha.sql |more' I still see mangled characters |
16:55 | chris | i wouldnt expect that to work |
16:56 | is it mangled if you import it back in | |
16:56 | kados | I'll give it a shot |
16:56 | chris | or if you open it in jed? |
16:58 | or some editor we know parses utf8 ok | |
16:59 | kados | mangled in vi |
16:59 | which I know does utf8 ok | |
16:59 | because I can read raw MARC with it | |
17:01 | chris | hmm |
17:02 | kados | diff -q kohawipo.utf8.sql kohawipo.utf8.sql2 |
17:02 | there is no diff between > and -r | |
17:02 | :( | |
17:03 | chris | how odd |
17:03 | what version of mysql do you have? | |
17:03 | kados | debian sarge's : mysql Ver 14.7 Distrib 4.1.11, for pc-linux-gnu (i386) |
17:06 | chris | -compatible=mysql40 make any difference? |
17:06 | kados | in mysqldump? |
17:07 | chris | yeah |
17:07 | kados | I woudln't think it would since 4.0 didn't support _any_ character sets :-) |
17:07 | chris | ok |
17:07 | i have a theory, can you try this for me please | |
17:07 | kados | btw: |
17:07 | l$ echo $LANG | |
17:07 | en_US.UTF-8 | |
17:08 | chris | --default-character-set=latin1 |
17:08 | when you mysql dump it | |
17:08 | and just take a look what it looks like | |
17:08 | kados | I've tried that too :-) |
17:09 | chris | and still mangled? |
17:09 | kados | it just mangles it in a different way :-) |
17:09 | I also tried --default-character-set=utf8 | |
17:09 | chris | yeah thats its default |
17:09 | kados | well ... I've got to get going |
17:09 | thanks for the help | |
17:10 | I'll see if I can lose some more hair on this tomorrow :-) | |
17:10 | chris | you are aware changing a table to utf8 doesnt change the data in the table eh? |
17:11 | this data was loaded in after the table was set to utf8 ? | |
17:11 | kados | yes |
17:11 | chris | cool |
17:11 | there will be something we are missing | |
17:11 | kados | maybe that mysql can't handle charsets :-) |
17:12 | chris | no it can |
17:12 | ppl all over the world are using it to do just that | |
17:15 | one thing to try would be in | |
17:15 | in mysql | |
17:16 | select * from tablename into outfile '/tmp/somefile'; | |
17:16 | then take a look at that file | |
17:32 | interesting, i just mysqldumped some, and loaded them back in .. and they appear ok ... ill check in with you tomorrow joshua and see what we are doing different | |
17:40 | thd | hdl: everything looks correct to me except that on my ISO 8859-1 environment system I do not see the accented character represented as a composed single character glyph but I see the correct characters for each of the two UTF8 bytes which will form the character. |
17:43 | hdl | thd: IMHO, you should see the accented character and not "as a composed single character glyph but I see the correct characters for each of the two UTF8 bytes which will form the character." |
17:44 | thd | hdl: I cannot save a test record on your test system to see that the end result is fine because hidden MARC tag for the record ID is not being automatically filled in the default framework. |
17:45 | hdl: are you saying that I ought to see the single UTF8 character in the record editor even if my environment is not UTF8? | |
17:47 | hdl | Normally, the display is UTF-8 encoded (sent by apache.) And even with MY environment full UTF-8, it is ugly displayed. |
17:47 | bob | i've a question about the value_builder plugins do they work for repeatable fields? |
17:48 | eg when editing a record does the popup get info from just one field or many? | |
17:50 | thd | hdl: I thought tumer had reported that it was not a problem for him using Internet Explorer set to UTF8 preferred but only for the rest of us using Firefox or whatever else. |
17:51 | bob: the value builder is often filling repeatable fields and filling them individually. | |
17:52 | bob: had you asked a question on koha-devel to day relating to value builder pop-up? | |
17:52 | bob | so if you have a bunch of repeatable fields for 650a with info in them will clicking the ... grap the info from all or just the one |
17:52 | ah ype | |
17:52 | yep | |
17:53 | thd | bob: I answered above for the record editor. Is your context the record editor? |
17:53 | bob | yep |
17:54 | thd | bob: I had difficulty understanding your question as asked on koha-devel |
17:54 | bob | i understand the popups can take a bunch of info and place it in right places in the marc record form |
17:55 | but i'm wondering if you then edit the record using the popup | |
17:55 | will the popup be repopulated with the right info | |
17:55 | if you are using repeatable fields | |
17:57 | thd | bob: so one confusion I had was how you were naming fields or subfields in asking your question. |
17:58 | bob | ah right |
17:58 | i'm meaning tag 650 subfield a | |
17:59 | thd | bob: I answered above about repeatable fields such as repeatable MARC 21 field 650 for topical subjects |
18:00 | bob: so you mean the repeatable 650 of which the first subfield is $a and $a as we know for this case is non-repeatable while the 650 is repeatable? | |
18:01 | bob | umm not sure |
18:02 | thd | bob: so you should be able to fill one 650 after another with different values and if you have a good version of the record editor you can fill all the subfields in the field at once and not just $a. |
18:06 | bob: I think in future, multiple value builders will function so that the free floating subdivisions $z, $x, $y, and $v can be assigned using the value builder in context separately from $a if needed. | |
18:07 | bob | right |
18:08 | thd | bob: this is a problem especially for subjects, where the value builder does not work as well as it might for filling subdivided subject headings in most cases.. |
18:09 | bob | ta |
18:09 | thd | s/most cases/most cases with real authority records from real authority files/ |
18:11 | bob: the current value builder design which expects one value for all the subfields in the field works best for names in 1XX, 7XX, etc. | |
18:12 | tumer: have you come to save us from the scary UTF8 monsters? | |
18:20 | tumer: the scary UTF8 monsters dragged kados away. | |
18:24 | bob: what version of the record editor are you using? | |
18:24 | bob | 2.2.5 i think |
18:25 | tumer | hi thd |
18:26 | bob:what u are asking i belive is using authorities not value builder. | |
18:27 | bob:thats how we fill tag 650 with data from somehere else | |
18:27 | thd | bob: the best version is rel_2_2 just before paul's recent commits to fix a bug reverted things somewhat |
18:28 | bob | hmm, i looking at the files in the /value_builder/ sub dir |
18:28 | and playing with them | |
18:28 | but not sure what i'm doing is right | |
18:28 | tumer | thd:regarding utf8 i believe we have a bigger problem than anticipated. Revert to earliear version of MARC::Record |
18:29 | as long as you dont need MARC8 to UTF8 conversion you will see that all problems go away | |
18:29 | thd | bob: yes are you using a set of authority records or are you just creating a few token values in an array? |
18:30 | tumer: what is wrong with the current version of MARC::Record? | |
18:31 | tumer: I mean what is wrong that you had not fixed? | |
18:31 | tumer | that is what is causing all this havoc of utf8 display problems |
18:31 | thd:i believe i have fixed it but kados does not | |
18:32 | thd | tumer: where does kados find that the problem still exists which you do not see? |
18:33 | tumer | thd:well i think he did not still try it so i cannot see the difference on his server |
18:34 | thd | tumer: so if he does not have your fix installed he cannot test it :) |
18:35 | tumer | well i have posted it to devel expecting people to use and comment on it |
18:36 | chris | tumer: im working on getting some space to test it |
18:36 | hopefully in the next few days | |
18:36 | thd | tumer: he had another suggested solution today using Encode but he found problems persisted and I was uncertain why that code would solve anything. |
18:36 | chris | we had a public library go live on saturday, so ive been pretty busy |
18:36 | tumer | hi chris btw i hear wife expecting a baby? |
18:37 | chris | yep, december 19th |
18:37 | tumer | so is he/she started kicking yet? |
18:37 | chris | anytime in the next week or so, the kicking should start apparently :) |
18:38 | tumer | chris congragulations |
18:38 | thd | chris: and in December the screaming starts |
18:38 | chris | thank you |
18:38 | :-) dont remind me | |
18:38 | rach | howdy |
18:38 | tumer | well sleepless nights help koha:-) |
18:38 | chris | :-) |
18:39 | thd | chris: only pleasant screaming of course :) |
18:39 | tumer | rach:library.neu.edu.tr should be working |
18:39 | thd | tumer: yes, what records bring up the animated walking path to the book? |
18:40 | rach | thanks very much |
18:40 | tumer | click on the LC call number |
18:40 | thd | tumer: any call number? how complete is the map? |
18:41 | tumer | chris:what kados is proposing of using Encode for the query dows not work ( i tried them all) |
18:41 | what that does is to create the problem of query | |
18:41 | chris | right |
18:41 | thd | tumer: It did make some difference in the display |
18:42 | tumer | but i already have utf8 characters on the template and they get broken |
18:42 | Encoding the whole Output.pm does not work either | |
18:43 | i am currently using the fix i reported in production | |
18:44 | library.neu.edu.tr uses this fix | |
18:44 | chris | its looking ok to me |
18:44 | tumer | changing the language to turkish from the top of the page do you see any anomalies |
18:45 | chris | nope |
18:45 | if i change encoding from utf-8 to anythign else i do | |
18:45 | but if im viewing the page as utf8 it looks fine | |
18:45 | tumer | so i say we bug the Marc::Record maintainer and see what he says |
18:46 | chris | cant hurt |
18:46 | tumer | chris we are supposed to look and see in utf8 dont we? |
18:46 | chris | yes thats right tumer |
18:47 | i was just checking that i was viewing it in utf8 by toggling off and on | |
18:47 | thd | tumer: does the JavaScript for the call number map pop-up only work on Internet Explorer? |
18:48 | tumer | chris:i have checked characters hex by hex since this problem started used to have white hair now no hair at all |
18:49 | thd:is there a bug i only look at it in IE? | |
18:49 | thd | tumer: kados and I still have extra hair |
18:50 | tumer | thd: can you see the little popup plan? |
18:50 | thd | tumer: I cannot get the map to pop-up on Firefox and I do not have Opera installed at the moment because Opera prevented me from upgrading x-windows libraries. |
18:51 | tumer | thd: ihave to check that and see what i can do.Firefox popups do not popup in IE so i change them |
18:51 | chris | i get a javascript error in firefox |
18:52 | tumer | same javascript supposedly! |
18:52 | thd | tumer: Is there not one set of JavaScript code to make the same function work for any browser? |
18:52 | tumer | chris:does it say what error is that |
18:53 | chris | it doesnt like the location function |
18:53 | uncaught security exception | |
18:54 | tumer | chris:i will look into that. its a simple localion.pl Perl script supposed to run in a window |
18:55 | bob | i found the problems with most javascript popups that work in firefox but not IE were due to people trying to give the popup window a name with a space in it |
18:55 | which firefox can handle but IE could not | |
18:55 | thd | tumer: you would never do that would you? |
18:56 | bob | eg window.open(url,name,...) |
18:56 | where name was like 'koha popup' | |
18:56 | chris | http://library.neu.edu.tr/cgi-[…]xact&op=do_search |
18:56 | take a look at that page bob, thats where it happens | |
18:57 | thd | tumer: do any of your names have holes in them? |
18:57 | tumer | thd: i delete all the names the templates have in them as well otherwise it does not popup in IE |
18:58 | chris | line 232 it seems to be moaning about |
18:59 | its the javascript its complaining about | |
18:59 | thd | tumer: are you using a variable 'General Collection', just the sort of name with a space that bob cautioned about? |
18:59 | bob | says location is not a function |
18:59 | chris | yeah and before that do you get the error |
19:00 | on page load | |
19:00 | uncaught exception bit bob? | |
19:00 | i think thats why location isnt a function | |
19:00 | bob | me reloads |
19:01 | chris | the help popup works |
19:01 | bob | says it may not load or link to file:///C:/Program%20Files/Microsoft%20Office/Office10/fpclass/animate.js |
19:02 | chris | function help(help_topic) { |
19:02 | var help_window = window.open('/cgi-bin/koha/opac_help.pl?help_template=' + help_topic ,'_blanck','height=400,width=400,status=no,toolbar=yes,menubar=no,location=no,resizable=yes,top=0,left=0,scrollbars=1'); | |
19:02 | help_window.focus(); | |
19:02 | function location(locationc,lib) { | |
19:02 | var help_window = window.open('/cgi-bin/koha/opac-location.pl?location=' + locationc +'&lib='+lib,'_blanck','height=300,width=640,status=no,toolbar=no,menubar=no,location=no,resizable=no,top=0,left=0,scrollbars=1'); | |
19:02 | help_window.focus(); | |
19:02 | } | |
19:02 | the help one seems to work, the location one doesnt | |
19:02 | tumer | it just calls href="javascript: location('<!-- TMPL_VAR NAME="classification" -->','<!-- TMPL_VAR name="location-only" -->');"> |
19:03 | and then.. | |
19:03 | function location(locationc,lib) { | |
19:03 | var help_window = window.open('/cgi-bin/koha/opac-location.pl?location=' + locationc +'&lib='+lib,'_blanck','height=300,width=640,status=no,toolbar=no,menubar=no,location=no,resizable=no,top=0,left=0,scrollbars=1'); | |
19:03 | help_window.focus(); | |
19:03 | } | |
19:03 | chris | yep |
19:03 | and for some reason that bit doesnt work .. the help function above it .. works just fine | |
19:03 | the only difference i can see | |
19:04 | is one is resizable and the other isnt | |
19:04 | but i might be missing something | |
19:04 | bob | is location a reserved word? |
19:04 | i know there one can call window.location | |
19:04 | chris | hmm that might be it |
19:04 | bob | eg window.location.href |
19:05 | tumer | bob:good point |
19:06 | ii have renamed it to locationonly | |
19:06 | any luck anyone? | |
19:08 | chris | im still seeing location tumer |
19:08 | tumer | oops sorry wrong server |
19:08 | chris | :) |
19:09 | thd | :) |
19:10 | tumer: you never modified the record detail display to support the map? | |
19:11 | tumer | correct server now i hope |
19:12 | chris | sweet works now |
19:12 | thats cool as hell tumer | |
19:12 | thd | fantastic it works |
19:12 | tumer | thanks guys |
19:13 | bob | woo that is fancy |
19:13 | tumer | end of october it will take you to the correct face of the shelf- i hope |
19:13 | thd | rach: click on a call number in an OPAC search result set to see the best Koha demo ever :) |
19:14 | chris | very cool |
19:14 | bob | this line in the code though looks wrong -> <script language="JavaScript1.2" fptype="dynamicanimation" src="file:///C:/Program%20Files/Microsoft%20Office/Office10/fpclass/animate.js"> |
19:14 | tumer | bob: i am cleaning those. left owers of previous anymation we tried |
19:15 | chris | ahh |
19:15 | bob | cool |
19:15 | thd | tumer: you had not added this to the record detail display it is only in the multi-record result set |
19:16 | tumer | thd: currently yes |
19:16 | its just a line of code which currently is too much | |
19:16 | thd | :0 |
19:17 | tumer | i am currently deeply modifying intranet |
19:17 | working on version 3.5! | |
19:18 | thd | :) |
19:18 | tumer: kados says that most of my suggestions are for version 4 | |
19:18 | tumer | well so you are ahead of me |
19:20 | thd | tumer: you can see the result of what kados had done with encode earlier |
19:20 | tumer | thd: just checking |
19:21 | chris | not me |
19:22 | whats the problem? | |
19:22 | dewey | the problem is probably if there is no data in the sort field zebra throws and error and does not even return the unsorted results |
19:22 | tumer | thd:ok this resolves the problem of query displaying correctly |
19:22 | kados | chris: you there? |
19:22 | tumer | dewey:upgrade zebra |
19:22 | dewey | tumer: i'm not following you... |
19:22 | kados | chris: you're right about the problem |
19:23 | thd | tumer: I find that the canonisation rules are not supporting Altug\x{0306} Osman, |
19:23 | kados | chris: just emailed you what the mysqldump looks like |
19:23 | chris: now, the trick is, what's the solution? :-) | |
19:23 | chris | ahh so i suspect that there is the problem eh |
19:23 | yeah how to fix it | |
19:23 | i think what we have done is told the tabel to be utf8 | |
19:24 | thd | tumer: http://zoomopac.liblime.com/cg[…]=O%CC%88rneklerle |
19:24 | tumer | kados: if you had accented characters in your template you will see that they break as well |
19:24 | chris | but all the columns are still latin1 |
19:24 | kados | right |
19:24 | so we need to convert them then, eh? | |
19:24 | chris | hmmm i suspect so |
19:25 | kados | though it would probably be better to create them as utf8 |
19:25 | then import the data | |
19:25 | chris | i suspect out convet_utf8.pl needs to be fixed |
19:25 | tumer | kados:your page looks correct on IE |
19:25 | chris | yeah that would be best joshua |
19:25 | thd | kados: yes if you create the table new it is not a problem but then you do not have your data |
19:26 | kados | chris: so what do I need to do to create all the Koha tables and columns correctly? |
19:27 | chris: setting aside for the moment how to fix the databases of already existing Koha users | |
19:27 | chris | good question |
19:27 | ill have to google | |
19:27 | thd | tumer: if everything is correct why does my posted author search finding nothing and failing to use the correct characters |
19:27 | tumer | thd:? |
19:28 | thd | tumer: http://zoomopac.liblime.com/cg[…]=O%CC%88rneklerle |
19:28 | kados | tumer: in safari the O in that page works, but the s doesn't ... in firefox, neither one does :( |
19:29 | tumer | kados:i know and in IE they will. If you use my corrected version of USMARC.pm thay all will look correct |
19:29 | its a matter of firefox and safari beeing poorer in displaying decomposed characters | |
19:30 | kados | tumer: so is it necessary to import using your modified USMARC.pm? |
19:30 | thd | kados: are you not referring to a mere browser problem if it is browser specific? |
19:30 | kados | tumer: or is it just the display? |
19:30 | tumer: (ie, are they stored wrong in the db, or just being mangled on the way out?) | |
19:31 | tumer | kados:XML definition and web character definition requires them to be composed |
19:31 | some browsers like IE can handle them ddecomposed | |
19:31 | kados | tumer: does that mean we should entity encode them? |
19:31 | tumer | yes |
19:31 | kados | tumer: i have code that does that in Record.pm in HEAD |
19:31 | tumer: _entity_encode() subroutine | |
19:31 | tumer: should do the trick | |
19:32 | tumer | otherwise you will not be able to serach them from the keyboard |
19:32 | kados | ok ... |
19:32 | so do we entity encode them before we import? | |
19:32 | or just while they are being pulled out for display? | |
19:32 | tumer | our keyboards produce composed characters |
19:32 | thd | kados: we can find them in most cases by copying the string and pasting the query so they must be stored correctly. |
19:33 | tumer | we put them in zebra composed |
19:33 | i have to check your _entity_encode | |
19:33 | i have to read entity_encode | |
19:34 | thd | kados: yet my posted author search fails, why? |
19:34 | kados | tumer: it's quit short :-) |
19:34 | tumer: quite even | |
19:35 | tumer | is it NFC'ing the results? |
19:35 | kados | yep |
19:35 | tumer | then it should work |
19:36 | thd | kados: why is there nothing displayed for this record outside of the result set? http://zoomopac.liblime.com/cg[…]l.pl?bib=99999999 |
19:36 | kados | thd: I've no idea |
19:36 | tumer | thd:copying and pasting you can search but not from keyboard |
19:37 | thd | kados: that is the record found by searching for yasa |
19:38 | tumer | thd:this record seems to exist only in zebra |
19:39 | thd | tumer: so if it is only in Zebra then there is no code to display the record detail or support author searching by clicking on the author link? |
19:39 | tumer | kados:still all my records are NFC'ed both in zebra and in Mysql |
19:40 | and all my templates do break with the new Record.pm but mine | |
19:40 | thd | tumer: what is NFC? |
19:40 | tumer | s/Record/USMARC |
19:41 | composed character format | |
19:41 | thd:from now on i use NFC for composed and NFD for decomposed characters | |
19:42 | thd | tumer: what does the 'N' represent? |
19:44 | tumer | thd: these are Unicode terms i am not very familiar with |
19:45 | thd | tumer: must have been introduced with Klingon :) |
19:53 | kados: do I understand correctly that Perl cannot manipulate the byte string correctly unless Encode::decode has been used? | |
20:04 | chis: do you understand when and where Encode::decode should be needed and when it would be unnecessary? | |
20:24 | N | |
20:25 | I want a different universe or at least a different OS | |
20:26 | Burgundavia | thd: what do you use? |
20:29 | thd | Burgunavia: the ordinary version of reality viewed through Debian GNU/Linux etch with more font and encoding problems than if I were using some system running Internet Explorer. Yet I do not want any of the Internet Explorer OS's or Internet Explorer even. |
20:36 | kados | thd: I'm not entirely convinced it's a browser problem |
20:38 | thd | kados: what differences do you find between browsers if anything other than display? |
20:40 | kados: why would Encode::decode be needed to pass the content of a string if there is no need to parse the actual characters in the string? | |
01:59 | ToinS | hello world |
02:59 | hdl | hello world |
03:46 | chris | hi all |
03:46 | big game today eh arnaud? | |
04:26 | alaurin | hi all, yes, it's a big day today, for a lot of french |
04:30 | russ | see you tomorrow morning |
04:31 | chris | yep, i have to be up at 7am to see kickoff, so im going to sleep too |
10:21 | slef | Has anyone seen .lex files before? |
10:23 | it's some sort of book data | |
10:23 | but trying to less it just ate my session :-/ | |
10:23 | so I'm guessing binary and not nice | |
10:23 | owen | http://filext.com/detaillist.php?extdetail=LEX |
10:24 | slef | yeah, it's none of those AFAICT |
← Previous day | Today | Next day → | Search | Index