IRC log for #koha, 2012-01-07

← Previous day | Today | Next day → | Search | Index

All times shown according to UTC.

Time Nick Message
00:04 melia joined #koha
00:35 jcamins_away rangi: any idea why MARC::Record uses expat?
00:38 rangi hmm I think it uses whatever the xml parser is
00:38 you could try messing with parserdetails.ini
00:38 try libxml::xml instead or something
00:39 jcamins_away That's what I would've thought, but it looks like we're wrong.
00:39 rangi maybe you can tell it what to use
00:39 jcamins_away The configured parser is LibXML, and it's using that, but it also uses Expat, which is incredibly slow in comparison.
00:39 rangi its marc::file::xml eh?
00:40 * rangi is on phone so can't look
00:40 jcamins_away I'll poke around.
00:41 I was just hoping you knew off the top of your head. :)
00:42 Wow. We use every single parser module.
00:42 rangi heh
00:43 I suspect you can tell it what to use somehow
00:44 jcamins_away According to cfouts, expat is faster anyway.
00:44 rangi expat_xs maybe
00:45 jcamins_away Oh, right. They're different. Good point.
00:45 rangi xs means its a wrapper for compiled c binaries
01:02 melia left #koha
02:04 ebegin joined #koha
02:36 Soupermanito joined #koha
03:01 druthb joined #koha
03:01 druthb o/
03:37 wizzyrea joined #koha
03:49 stuartyeates joined #koha
03:54 ebegin joined #koha
05:53 jcamins_away @later tell rangi In case you were wondering, ZOOM doesn't store IDs.
05:53 huginn jcamins_away: The operation succeeded.
06:47 trea joined #koha
07:02 cait joined #koha
07:07 rangi hey trea and cait
07:07 trea hey rangi
07:07 cait hi rangi :) hi trea
07:07 trea hi cait
07:08 rangi ur up late trea
07:08 trea yea, just noodlin around
07:23 * rangi has to head off in aminute to buy some groceries
07:25 cait don't forget the cookies :)
07:25 rangi :)
07:25 * rangi will bbiab
07:35 Oak joined #koha
07:55 rangi back
07:55 cait wb rangi
08:02 bag hey there rangi and cait
08:02 cait heya bag
08:02 bag alright I'm off to bed
08:02 :D
08:05 rangi sleep well
08:40 magnuse kia ora #koha
08:41 bbl
08:41 nick magnus_afk
08:47 cait hi magnuse :)
08:48 rangi hi magnuse
09:38 magnuse hiya cait and rangi
09:45 stuartyeates left #koha
11:57 druthb joined #koha
11:57 druthb o/
11:57 \o
11:57 \o/
11:59 magnuse o/
11:59 jcamins_away o/
12:00 * druthb is bringing another quarter-million library patrons into the world of being served by a Koha-powered library.  :)
12:01 * magnuse is wowed
12:01 magnuse lucky quarter-million!
12:02 druthb :)
12:02 jcamins Well, eww. Zebra has some major issues, doesn't it?
12:03 magnuse it has been rumoured...
12:03 druthb wahanui:  druthb?
12:03 wahanui She finally snapped, like we all knew she would.
12:03 * druthb nods.
12:03 magnuse oops?
12:07 druthb hm.
12:07 wahanui:  magnuse?
12:07 wahanui magnuse is afraid that we added another 10000 bugs while he was eating pizza.
12:07 * druthb giggles.
12:08 druthb wahanui:  Brooke?
12:08 wahanui I'M TAKING THE PISS OUT OF 'EM. or a ratbag.
12:08 magnuse hehe
12:08 @wunder boo
12:08 huginn magnuse: The current temperature in Bodo Vi, Norway is -7.0°C (1:00 PM CET on January 07, 2012). Conditions: Partly Cloudy. Humidity: 67%. Dew Point: -10.0°C. Pressure: 29.45 in 997 hPa (Falling).
12:08 druthb cait?
12:08 wahanui That's Ms. Cait to you! or in a very good mood today :)
12:08 cait heya
12:08 druthb @wunder lawrence, ks
12:08 huginn druthb: The current temperature in Channel 6 Downtown, Lawrence, Kansas is 0.9°C (6:08 AM CST on January 07, 2012). Conditions: Clear. Humidity: 63%. Dew Point: -5.0°C. Windchill: 1.0°C. Pressure: 30.12 in 1019.9 hPa (Steady).
12:08 magnuse guten tag cait
12:08 cait @wunder Konstanz
12:08 huginn cait: The current temperature in Konstanz, Germany is 2.0°C (1:00 PM CET on January 07, 2012). Conditions: Light Snow. Humidity: 77%. Dew Point: 0.0°C. Pressure: 30.24 in 1024 hPa (Falling).
12:08 druthb guten tag, cait.  :)  *hug*
12:08 magnuse +2 and light snow = slush ?
12:09 cait it melts when hitting the grond
12:09 ground
12:09 magnuse yay
12:12 jcamins Welp, looks like using mod_perl with Zebra Might Not Be A Good Idea.
12:13 cait perhaps it's not zebra
12:13 but how we use it?
12:13 and hi jared
12:13 jcamins Alas, this is Zebra.
12:14 magnuse ouch?
12:14 would it be the same for plack? i don't know either of them...
12:15 pastebot "jcamins" at 98.14.120.176 pasted "Yikes! This is NOT O(n)" (31 lines) at http://paste.koha-community.org/192
12:16 jcamins The important bits are columns 1 and 4.
12:16 For 5k routines, Net::Z3950::ZOOM::event took 32735 seconds to run 95441 times. For 1k records, 71.1s to run 19181 times.
12:18 magnuse 32735s is quite a long time...
12:18 jcamins It is.
12:19 It is *100* times as long per call.
12:19 magnuse what kind of operation are you profiling there?
12:19 jcamins I am currently telling myself "surely there's something I can do to make it happy."
12:19 link_bibs_to_authorities.pl.
12:22 magnuse ah, and you are running that with mod_perl?
12:22 jcamins Nope, running it from the command line.
12:23 magnuse ah, so where does mod_perl come into it?
12:23 * magnuse is confused
12:24 jcamins With CGI, Zebra connections are short-lived.
12:24 With mod_perl, not so much.
12:25 It's unfortunate that I can't check whether my changes help quickly.
12:28 To be clear, the problem is not Zebra, it's the Perl interface to it.
12:31 Woohoo!
12:31 Wait, maybe I didn't solve it.
12:31 Drat.
12:31 * magnuse knows the feeling ;-)
12:32 * druthb enjoys watching jcamins sometimes.
12:32 jcamins Hrm. Microseconds are smaller than milliseconds.
12:32 This is good.
12:32 druthb lol
12:33 cait heh
12:33 :)
12:33 magnuse yay
12:33 jcamins Wait...
12:33 * jcamins does some math.
12:33 jcamins 14.6s < 71.1s.
12:33 Ladies and germs, we have solved the problem!
12:33 druthb yes, yes it is.
12:33 magnuse woohoo!
12:33 cait woohoo
12:34 druthb "we."  You and Myshkin?
12:34 jcamins Actually, Myshkin has abandoned me to cuddle with Shari.
12:34 He doesn't like Zebras.
12:34 druthb At this hour on a Saturday morning...he's trying to set an example for you.
12:35 jcamins The problem, in case you were wondering, is that you have to destroy your Zebra recordSets when you're done with them.
12:35 cait destroy!
12:35 sounds radical :)
12:36 * magnuse hums "anarchy in the uk"
12:36 jcamins I didn't recognize that was the issue at first because we already had the destroy command.
12:36 But it was commented out.
12:36 Whoops.
12:38 cait so zebra is not so bad? ;)
12:38 jcamins No, thank goodness.
12:38 fredericd jcamins: what are you trying to do exactly?
12:39 * druthb finishes loading 400K+ bibs and the quarter-million patrons, and runs some touch-up scripts.
12:39 jcamins fredericd: run link_bibs_to_authorities.pl on 167k records.
12:39 Without dying of boredom.
12:39 cait hehe
12:39 how long does it take now?
12:39 I think you began with an hour?
12:40 jcamins Being impatient, that last is problematic.
12:40 cait: well, 3k records just took... 308s.
12:40 cait: previously 5k records took...
12:40 33238s.
12:40 cait oh
12:40 so it should be faster now :)
12:40 jcamins No typo there.
12:40 Yes.
12:41 fredericd what have you modified?
12:41 jcamins fredericd: for the performance change I just uncommented the oAResultSet->destroy().
12:41 fredericd: more generally, I completely rewrote all the linking code.
12:43 fredericd intersting...
12:43 jcamins Bug 7284
12:43 wahanui Bug 7284 is the first step to making that happen.
12:43 huginn 04Bug http://bugs.koha-community.org[…]w_bug.cgi?id=7284 enhancement, P5 - low, ---, jcamins, ASSIGNED , Authority matching algorithm improvements
12:43 jcamins wahanui: interesting things? Yes indeed.
12:43 wahanui jcamins: excuse me?
12:45 * jcamins has another idea for optimization.
12:45 cait heh
12:45 go jcamins go
12:47 jcamins I'll profile 10k, then try my modification.
12:52 magnuse bbl
12:55 kmkale joined #koha
12:57 druthb Namaskar, kmkale!
12:58 kmkale Namaste druthb :-D
12:58 LTNC
12:59 druthb Indeed.  It's been a briskly busy fall for me, so I haven't been on here much.
12:59 jcamins I should make BiblioAddsAuthorities use the new working linking.
13:01 kmkale Namaste jcamins :)
13:01 jcamins Namaste, kmkale.
13:09 Woohoo! MySQL now takes more time than Zebra!
13:20 cait um yay? :)
13:20 jcamins cait: yes, that's good.
13:20 I mean, the fact that updating a record is hugely inefficient is bad, but it's progress.
13:22 fredericd jcamins: is your code somewhere?
13:22 patch for bug 6990 could help
13:22 huginn 04Bug http://bugs.koha-community.org[…]w_bug.cgi?id=6990 enhancement, PATCH-Sent, ---, frederic, NEW , TransformKohaToMarc enhancement
13:22 jcamins fredericd: https://github.com/jcamins/koha/commits/bug_7284
13:23 Cool!
13:23 That's exactly what I was just thinking we needed!
13:23 fredericd++
13:24 * jcamins may add an additional sign-off on that.
13:24 kmkale joined #koha
13:25 fredericd jcamins: A C4::Heading::UNIMARC class is required...
13:26 jcamins fredericd: we've never had one in the past.
13:26 fredericd: I'll add a simple one, though.
13:26 fredericd: but that means someone will have to test it. :P
13:26 fredericd++ # for volunteering to test linking with UNIMARC.
13:27 :P
13:27 fredericd I don't say no :-)
13:30 jcamins What does TransformHtmlToMarc do?
13:30 It returns a MARC::Record object...
13:30 right?
13:31 Seems to.
13:34 fredericd jcamins: I got your branch. Could you rebase it to HEAD? It would help tracking your modifications.
13:37 jcamins fredericd: I'll rebase when I'm done.
13:37 fredericd: it is based on head as of two days ago.
13:37 I can't push branches after rebasing.
13:40 (I guess that's the disadvantage of frequent upstream changes... it makes sharing repositories difficult)
13:43 fredericd: what if I used a different branch?
13:43 That should work... right?
13:44 fredericd Yes. You could have a branch bug_7284_v2 for example
13:44 But I don't need it anymore
13:44 I was trying to figure out you modification
13:45 jcamins fredericd: does it make sense?
13:45 fredericd I can with git diff jcamins/master, where jcamins is the name of your remote repo
13:45 jcamins Ah. Cool! I didn't know that.
13:46 The "master" branch on there is spectacularly out of date.
13:46 GitHub won't let me push to it anymore.
13:46 I guess I should figure out why and fix that at some point.
13:46 fredericd you can do a git merge
13:47 I can't test your patch without having a C4::Heading::UNIMARC class!
13:48 jcamins I will add one. :)
13:48 fredericd and I don't like that much how C4::Heading::MARC21 hard code authorities definitions...
13:49 jcamins It's certainly not ideal, but it's been in Koha for years, so it can stay a bit longer.
13:50 I've been thinking of making it use the framework, though.
13:50 Thoughts?
13:52 fredericd The framework must contain info required to build $bib_heading_fields data structure
13:52 jcamins Yeah, and I've been thinking that I'd prefer to use that.
13:53 Since someone might not want to link 655.
13:56 fredericd I have libraries who use specific authorities types located in the 69X biblio tags. For them, C4::Heading::MARC should have to be modified by hand ($bib_heading_fields)
13:58 jcamins So, the question is, how do I cache the results of SELECT authtypecode,tagfield FROM ...
14:00 Ah, got it.
14:02 Wait, no I don't.
14:05 Soupermanito joined #koha
14:05 fredericd This way, we even could merge MARC21 and UNIMARC classes...
14:06 jcamins Yes, I like that idea.
14:06 Except for one thing. How do I cache the results of the database access?
14:08 memoize valid_bib_heading_tag, and have it take $tag and $framework?
14:09 Oh, there's the problem. Subdivisions.
14:10 fredericd: do headings currently link with authorities in UNIMARC at all?
14:10 fredericd Yes, but they could stay hardcoded. Not absolutely sure, but I think that MARC21 and UNIMARC share the same letters to identify subdivisons
14:10 was replying to your subdiv question
14:11 jcamins The form subdivision in UNIMARC is $j.
14:11 Actually, bigger problem: UNIMARC uses different indexing.
14:11 GRS-1.
14:13 fredericd we don't use Zebra as we should/could
14:13 * jcamins is not writing UNIMARC indexing.
14:13 jcamins I'm getting paid to fix linking for MARC21, and I can add a UNIMARC class easily enough.
14:14 fredericd I have a pending patch for improving Zebra indexing, and facets building: bug 3216
14:14 huginn 04Bug http://bugs.koha-community.org[…]w_bug.cgi?id=3216 normal, PATCH-Sent, ---, frederic, NEW , UNIMARC author facets
14:17 fredericd jcamins: C4::Heading::MARC21::$bib_heading_fields has something to distinguish subject/series/main_entry
14:18 it doesn't exist in auth_tag_structure and auth_subfield_structure tables...
14:18 jcamins Yes, but that doesn't serve any useful purpose.
14:18 fredericd so, code using that should be removed before
14:18 jcamins Yeah, I was just thinking of doing that.
14:22 cait 3216 looks really interesting
14:22 will it work for marc21 too?
14:22 fredericd yes
14:22 cait and hi fredericd :)
14:22 we use icu for one library with hebrew
14:22 is this only icu for facets or will it use it in general then?
14:23 fredericd general
14:23 and you can choose during installation
14:27 cait sounds really good
14:28 hope to find time to look at it... but probably not for a while :(
14:31 fredericd cait: This patch is essential for UNIMARC libraries. Without it, authors facets don't include firstname
14:31 cait oh!
14:32 I think a sign off from someone knowing unimarc would be good then
14:32 and another for marc21
14:32 there are some system preferences that don't work well with icu
14:33 I think it was stemming, fuzzy or/and field weighting
14:33 we never really figured it out, but there is some weirdness about them and icu
14:34 fredericd Yes. But it may have been fixed in Zebra/YAZ itself
14:36 cait worth a test :)
14:40 perhaps link that to the diacritics patch?
14:40 hm bug not patch
14:40 I have to run to catch my bus - bye all
15:25 jcamins AAAAAAAHHHHH!!!!!!!
15:26 We do the stupidest things!
15:33 * druthb occasionally worries about jcamins, you know.
15:33 jcamins druthb: ModZebra is part of C4::Biblio.
15:34 druthb yes....?
15:34 jcamins ModZebra.
15:34 wahanui it has been said that modzebra is part of C4::Biblio.
15:34 jcamins The function used to modify...
15:34 authorities.
15:35 druthb it's also used to trigger reindexing of a biblio.
15:35 jcamins Yeah, I know.
15:35 druthb Properly, that probably ought to be two separate functions.  If you check out an item, you need to update the index---you *don't* need to tinker with its' attached authorities at that point.
15:36 jcamins Ah, no, you misunderstand me.
15:36 In order to modify an authority record, you must call ModZebra.
15:36 In C4::Biblio.
15:36 Providing a "biblionumber" that looks suspiciously like the authid.
15:37 druthb ahhh...
15:37 probably oughta put that in C4::Search::Zebra or some such, then, in the long run.
15:37 jcamins Exactly.
15:37 * druthb is a little slow sometimes, but she usually gets it, eventually.
15:41 jcamins :)
15:42 Having exhausted members of his own family...
15:42 * jcamins moves on to cataloging imaginary books by members of druthb's family.
15:42 druthb lol
15:43 jcamins Excellent, this works.
15:43 druthb "Thick as mud", by D Ruth Bavousett.  "Living with a Crazy Transsexual", by Lindsey Bavousett.  "I'm a Teenager, Therefore My Life Sucks," by Libby Bavousett.  "Meh, Whatever." by Captain Midnight Bavousett
15:43 * jcamins grumbles about circular dependencies.
15:45 jcamins druthb: lol
16:18 fredericd: I'm still having trouble figuring out how to cache the results of retrieving a list of tags linked to authorities.
16:18 fredericd: any thoughts?
16:19 It's too early for NZ still.
16:27 ktk joined #koha
16:28 ktk hw to get rid of an error for patrons with message getting an error as patron has had overdue items and is blocked for 833 days We are getting this error for many patrons
16:30 jcamins ktk: I think I recall someone mentioning a bug that caused this, but I don't recall what it was.
16:30 (I mean, very recently someone mentioned it)
16:33 ktk joined #koha
16:45 ktk joined #koha
16:48 ktk joined #koha
16:49 ktk is there any patch available to this bug then "to get rid of an error for patrons with message getting an error as patron has had overdue items and is blocked for x days" we are getting this error for several of our borrowers and we cant do any transactions
16:50 does it sort out issue with any latest version of koha? or is there any way to make changes in the database directly and get this error sorted out.
16:54 NateC joined #koha
17:10 NateC joined #koha
18:00 fredericd jcamins: how to cache what?
18:01 jcamins fredericd: which fields are associated with authority records.
18:02 fredericd I don't understand
18:02 jcamins Wait, never mind.
18:02 As I wrote out my question in more detail I found the answer.
18:03 fredericd good for you
18:03 jcamins Oh, no, I didn't.
18:03 Un-never mind.
18:04 We discussed looking up in the framework which fields were authority-controlled, rather than hardcoding it.
18:04 So I need to figure out whether a given field is authority-controlled.
18:05 I do not want to query MySQL *every* time I am processing a heading.
18:07 fredericd jcamins: I suppose you have during initialization and populate the appropriate data structure
18:08 jcamins Yeah, my difficulty is that I really need the equivalent of the C++ "static".
18:08 Maybe I'll try memoize.
18:09 fredericd In C4::Heading::MARC21 ?
18:10 jcamins Well, just in C4::Heading.
18:10 That doesn't have to be specific to the MARC flavor.
18:11 Wait, maybe it does.
18:11 Drat.
18:11 fredericd Have you checked that: http://perldoc.perl.org/perlmod.html#BEGIN,-UNITCHECK,-CHECK,-INIT-and-END
18:12 jcamins I have not.
18:12 Cool!
18:12 Thanks.
18:25 bigbrovar_ joined #koha
18:28 jcamins Okay... it doesn't like variables declared with my.
18:29 our?
18:30 Yup.
18:33 rangi and make sure you clear/reset that variable whenever the db changes, or it will cause a big mess when running in a persistent environment like mod_perl or plack
18:35 jcamins rangi: how do I do that?
18:36 rangi find the routine that sets the value in the db, and make that clear the variable
18:36 jcamins Hm.
18:36 That's in a different file.
18:37 rangi right so that will now need to call a sub in the other one called something like clear_cached_heading or something
18:39 for cgi it doesnt matter so much, the variable dies when the script does
18:39 in mod_perl .. it will last forever
18:40 i think preferences in Context does it
18:40 jcamins Hm.
18:42 Okay... what's plan B?
18:43 rangi either way, if you cache something, you have to clear it when you change the value
18:43 jcamins Right, it used to be hardcoded.
18:43 rangi yep
18:43 jcamins That has not bothered anyone in 3 years.
18:44 rangi yep, but now its not
18:44 jcamins My plan C is "put it back to hardcoded because this really isn't the issue that I'm trying to deal with."
18:44 rangi so caching it will not be the same
18:44 jcamins Well, it was up until I decided fifteen minutes ago not to hardcode it.
18:44 rangi ah
18:44 id probably do the dynamic bit as a separate thing then
18:45 jcamins Right, that's the plan.
18:46 "No regressions" = good enough for me.
18:46 rangi yup
18:49 jcamins Sorry, fredericd.
18:49 rangi it can be a separate patch
18:50 but i wouldnt hold up your matching work trying to do it
18:50 kaizen :)
18:50 jcamins Right.
18:51 I was only doing it 'cause fredericd asked and it looked easy.
18:51 I was wrong about that last bit.
18:53 Good news! I remembered my alphabet after a few tries!
18:56 fredericd: UNIMARC pushed.
18:57 Argh!
18:57 I just pushed a typo.
18:57 fredericd: UNIMARC pushed without typo, now.
18:58 140k records matched today.
18:59 Several orders of magnitude faster.
18:59 rangi :)
18:59 awesome
19:02 jcamins rangi: incidentally, having ModZebra in C4::Biblio is horrifying.
19:02 Just thought I'd point that out.
19:04 Also, the AuthoritiesMarc<->Biblio dependency cycle is very vexing.
19:06 rangi yes
19:06 thats the main idea behind Koha:: in my mind
19:06 is make new modules there, more carefully and transition to them
19:06 but not to try to do it all in one go
19:06 jcamins If we had Koha::Search::Engine already, I could've mostly resolved that problem.
19:07 (AuthoritiesMarc<->Biblio, not cycles in general)
19:07 rangi and i dont think we need to create the whole module
19:07 jcamins No, just the part that does searching. :P
19:07 rangi no in general
19:07 jcamins Ah.
19:07 rangi what i dont want to see
19:07 jcamins Right.
19:07 Agreed.
19:08 rangi is one huge patch that shifts a whole module over
19:08 unless that module is already clean
19:08 but start with Koha::Biblio for example
19:08 and redo one subroutine to start
19:10 the nice thing with a new namespace is you can do it carefully without busting anything already existing :)
19:10 in theory
19:10 :)
19:10 jcamins Right.
19:12 rangi if i were redoing biblio, i think its totally a candidate for OO
19:12 using Class::Accessor as base
19:12 so i can do
19:12 $biblio->biblionumber();
19:13 thats as far as ive thought :)
19:13 jcamins :)
19:21 wizzyrea joined #koha
19:22 rangi oh look its wizzyrea
19:22 * wizzyrea waves
19:22 * jcamins waves back.
19:22 wizzyrea whassup
19:22 * rangi use to wave then he took an arrow to the kneww
19:22 rangi knee
19:23 joke fail
19:23 * wizzyrea laughs
19:23 wizzyrea naw, not a fail - still pretty funny.
19:24 * jcamins didn't get the joke.
19:24 bag hi there wizzyrea
19:24 wizzyrea see: skyrim memes
19:24 cait joined #koha
19:24 wizzyrea heya bag
19:24 cait heya #koha
19:24 bag yo cait
19:24 wizzyrea wow!
19:24 jcamins Ah. Yeah, I won't get those.
19:24 Hi bag and cait.
19:24 rangi hey cait!
19:24 bag left #koha
19:24 bag joined #koha
19:25 wizzyrea i am installing ancient windows games in wind
19:25 wine*
19:25 cait morning rangi, hi wizzyrea and jcamins :)
19:25 jcamins wizzyrea: excellent!
19:25 wizzyrea: you could play Starcraft!
19:26 cait oh starcract
19:26 starcraft
19:26 wizzyrea hehe, I love starcrack
19:26 but i've really only played SC2
19:26 rangi http://soundcloud.com/dubstep/[…]knee-by-rameses-b
19:26 jcamins Morrowind?
19:26 rangi skyrim is elder scrolls jcamins
19:27 wizzyrea ^^
19:27 rangi the latest one
19:27 jcamins rangi: I know, but Morrowind is old enough it might work in wine.
19:27 rangi yup
19:27 * rangi hasnt played it, my dad has it tho
19:27 jcamins Wait...
19:27 wizzyrea I think skyrim will, but I don't have my super good video card in linux
19:27 jcamins Skyrim runs under Wine?
19:27 Wow!
19:27 wizzyrea would be better to run it in windows
19:28 jcamins That's incredible!
19:28 wizzyrea *I think*
19:28 I got Rift to play in Wine
19:28 and WoW
19:28 rangi wine is pretty sweet these days
19:28 wizzyrea it really is, quite impressive
19:29 of course they keep thinking of new, novel ways to break it
19:29 but usually 6-8 months and they'll have a tweak for it
19:29 jcamins Morrowind is the only game I've played since Myst 3.
19:29 Wow... there have been two more Mysts.
19:29 rangi heh
19:30 i only play games at my parents house
19:30 * wizzyrea finds this funny
19:30 * jcamins wasn't so impressed with Myst III.
19:31 jcamins It took a week to win. :(
19:31 Myst took *years*
19:31 Okay, that might be because I was... errr... not very old... when it came out.
19:32 Apparently Myst came out almost 20 years ago.
19:33 rangi lords of midnight
19:33 cait has anyone played day of the tentacle?
19:33 * cait loved that
19:33 cait but was young too :)
19:33 jcamins Never played either of those.
19:33 rangi http://en.wikipedia.org/wiki/T[…]Lords_of_Midnight
19:33 i was 11 when we got that
19:33 i played that for years
19:34 drew maps on graph paper
19:34 cait heroes of might and magic - I drew maps for that
19:34 rangi course it took about 17 minutes to load each time
19:34 cait hm not heroes
19:34 only might and magic
19:34 rangi on my zx spectrum
19:34 * jcamins didn't have one of those.
19:34 rangi so actually i spent years listening to a tape squeal
19:34 cait http://en.wikipedia.org/wiki/Might_%26_Magic
19:35 rangi ah yeah i played that one
19:37 cait i got soo scared in those dungeions
19:37 and when I looked at it years later... it seemed quite silly
19:38 rangi hehe
19:38 anyone play ultima ?
19:38 jcamins Ooh... almost done with the 167k linking.
19:38 Not me.
19:38 I know about it, though. A bit of a cult classic, I think.
19:38 rangi http://en.wikipedia.org/wiki/Ultima_(series)
19:38 yeah
19:38 it was fun
19:39 i played 1 - 5
19:40 cait :)
19:41 rangi my nephew (who is 4) was running around yesterday yelling
19:41 "where's my sword, brigands are attacking"
19:41 jcamins lol
19:41 rangi i suspect he has been watching his dad play games
19:42 it was at the market, was quite funny
19:43 jcamins Bib 167198.
19:43 167313
19:43 ^^ do you see that? In just a few seconds it did over a hundred!
19:43 rangi nice
19:44 and because it doesnt need to rebuild everything in one go thats totally good enough
19:44 jcamins Yup.
19:45 Heh. Producing the linking report takes a long time.
19:45 Wow. It's a 21MB file.
19:45 rangi whoa
19:45 jcamins It includes a list of every single bib it processed.
19:46 rangi ahhh
19:46 jcamins Right... guess I need to write a script that will remove that list.
19:46 rangi that'd do it, well too much info beats not enough
19:47 jcamins Exactly.
19:47 First time running it on the entire database.
19:47 cait jcamins++ :)
19:48 jcamins 12.5MB for the report only.
19:54 340k unlinked; 495k linked; 5877 fuzzily linked.
19:54 rangi oh thats pretty darn good
19:55 jcamins Now, how to improve it?
19:56 rangi well i think thats a good improvement from the current one
19:56 jcamins Yeah, certainly.
19:56 cait what does fuzzy link mean?
19:56 rangi and worth getting others to test
19:56 like cait
19:56 hehe
19:56 jcamins :)
19:56 * cait hides
19:56 cait I gave him data!
19:56 * jcamins will be trying with cait's data. :)
19:56 cait how long did it take for those now?
19:57 jcamins I didn't try your data yet.
19:57 cait and you know that our data could be perfectly linked... if the script understood the $0
19:57 jcamins Right.
19:57 cait :P
19:57 rangi hmm
19:58 i wonder if that paul guy would give you data
19:58 jcamins cait: you can write your own linker for that, you know. Very easy with my new architecture.
19:58 I'm sure he would.
19:58 * jcamins would probably prefer to beg MDAH for data, though.
19:59 rangi yep
19:59 i just thought might be a nice PR
19:59 jcamins Not that it's likely they'd refuse to give it to me.
19:59 True.
19:59 * jcamins will consider.
20:00 cait jcamins: I should have seen that coming...
20:00 rangi :)
20:01 jcamins Ohh.
20:01 Figured out why these aren't linking.
20:01 No auth record.
20:02 The cataloger is going to be *pissed*.
20:02 trea joined #koha
20:03 jcamins (but, when you don't have an authority record, you're kind of SOL on the whole "automatic linking" thing.
20:04 cait: remind me where to find your data?
20:04 cait ah
20:04 jcamins Wait, found it.
20:04 cait thought you had downloaded it
20:04 jcamins I did.
20:04 But I want to test it on a server, and I have a lousy home connection.
20:04 cait want me to find the link?
20:04 jcamins No, I have it.
20:05 What's the difference between _1 and _2?
20:05 Just different parts of the data?
20:05 cait I think so
20:05 sliced up a bit
20:06 yep
20:06 https://wiki.bsz-bw.de/doku.ph[…]n:openaccess:maub
20:06 says 2 parts
20:07 jcamins cait: you'll be glad to know that writing your linker module will be super-easy.
20:08 :)
20:09 wizzyrea bbiab
20:11 cait jcamins: will be even happier when you help me doing it :)
20:13 jcamins Can I softlink to a file which is under a softlinked directory?
20:15 Answer: yes.
20:15 * jcamins refuses to ask the follow up question: "Should I?"
20:16 cait don't look at me
20:16 i don't know
20:17 * cait offers dried cherries
20:23 jcamins Wow!
20:23 cait: can I please use your data center?
20:24 Pretty please? :)
20:24 cait my what?
20:24 jcamins BSZ-BW's data center.
20:24 I've never gotten 5MB/sec download before.
20:24 cait oh
20:24 jcamins :)
20:25 cait perhaps has something to do with belwue?
20:25 jcamins It's nice and fast, whoever hosts it. :)
20:25 cait and you are probably downloading at a good time
20:26 our servers are at the data center of university of konstanz
20:26 jcamins Oh, maybe they have Internet2.
20:26 cait http://www.belwue.de/  - perhaps that's the reason?
20:28 jcamins Ah. Maybe.
20:29 cait: aut = authorities, tit = biblios?
20:30 cait: oh, in answer to your question earlier, "fuzzy" is whatever the linker defines it as
20:30 cait yep
20:30 sounds right
20:30 jcamins In this case, it means "more than one authority matched."
20:33 Importing Mannheim's data.
20:33 1.4 million titles?
20:33 Eek.
20:33 * jcamins will try just importing _1.
20:34 jcamins Hopefully that will be enough.
20:34 cait are there 2 auth files?
20:34 jcamins Yup.
20:35 20k imported...
20:38 50k imported...
20:40 cait: you have this in a Koha instance?
20:40 cait nope
20:40 jcamins It seems to me that this must be one of the larger installations.
20:40 Oh.
20:41 Right.
20:41 I guess it's not, then.
20:41 cait it's data from the union catalog
20:41 oh
20:41 and it's not converted... with our koha converter
20:41 so you will have wrong normalization perhaps
20:41 jcamins What does your Koha converter do?
20:41 Ah.
20:41 Hm.
20:41 cait unicode normalization
20:41 and some other evil things
20:41 jcamins Well, I'll bug you if it doesn't work. :P
20:41 cait like moving the subject chains
20:42 jcamins Moving the subject chains?
20:42 cait breaking them up into single subjects and move them to the proper marc21 fields
20:42 689
20:42 look at that
20:42 jcamins Uh-oh. So there aren't subjects in 650s?
20:43 cait hm
20:43 there might be some from loc
20:43 oh right
20:43 jcamins That won't help me test your subject headings. :P
20:43 cait I should have told you that...
20:43 jcamins Yeah...
20:43 cait sorry
20:43 didn't think about it
20:43 jcamins That's okay.
20:43 cait about that part... the other things should not make a big difference
20:43 stuartyeates joined #koha
20:43 jcamins Well, I have 90k authorities from BSZ-BW imported now.
20:44 cait there will be
20:44 corporate authors
20:44 personal names
20:44 no meeting
20:44 topical, geographic
20:45 jcamins Eh. It's running on a server. We'll see what happens.
20:45 :D
20:45 Better increase the size limits on the Zebra directories...
20:46 40GB each better be enough, 'cause I don't have infinite disk space on this server.
20:46 cait the structure of the 689 is ... not so nice I think
20:46 jcamins That's very diplomatic.
20:47 Perhaps Uli will be interested in testing the linking, to see if it can simplify things for you at all.
20:47 :)
20:48 cait at the moment we don't link then
20:48 them
20:48 we import them linked
20:48 but it would be nice to not rely so much on our authority number
20:49 but use koha internal numbers for linking
20:49 jcamins Yes, but maybe you can find a use for it. :)
20:49 cait and it would be cool to figure that out until may
20:49 because they change the fomrat
20:49 for the authorities
20:49 jcamins In May?
20:49 Why?
20:49 cait it's a real big project
20:49 jcamins Uh-oh. Big projects are always bad news. ;)
20:49 140k imported.
20:49 This is blazing fast.
20:49 That worries me.
20:52 cait nice data? :P
20:52 jcamins I suppose that's possible.
20:52 cait it's not bad news, but it's really big
20:53 all union catalogs closing, converting data to new format
20:53 training catalogers
20:53 ...
20:53 jcamins Exciting times. :)
20:53 cait yep
20:54 and we will reload all koha libraries with union catalog data
20:54 authorities especially
20:54 but also the bibliographic records
20:54 jcamins Well...
20:54 This is weird.
20:54 cait the advantages of a central cataloging database :)
20:54 so it's a good time to make changes to the converter and improve things
20:54 jcamins I have 171k authority records in my database, but none show up in the staff client.
20:55 Actually, I'm just having issues figuring out how to access authority records.
20:56 cait hmm
20:56 so now they show?
20:57 jcamins Oh, I just don't know how importing authorities works, apparently.
20:57 Ignore me.
20:57 cait that wouldn't be very nice... to ignore yo
20:57 jcamins Authorities are working fine.
20:58 Strangely it's starting at 16million.
20:58 cait hm
20:58 what does it look at?
20:58 the number in 001?
20:58 jcamins 001.
20:58 Yeah.
20:58 cait ah
20:58 that' scorrect
20:58 jcamins Something else you should've warned me about? :P
20:58 cait always 9 digits
20:59 hehe
20:59 this is a test
20:59 if I did tell you everything... would not be a lot of fun
20:59 jcamins What are you testing?
20:59 lol
21:02 * magnuse waves
21:03 cait hi magnuse :)
21:03 magnuse guten abend cait
21:03 cait :)
21:05 jcamins: not you if you are worried about that :)
21:06 jcamins lol
21:19 Oh dear.
21:20 So...
21:20 Don't load Mannheim into a Koha database.
21:20 cait oh
21:20 what happened?
21:20 jcamins The tablespace was filled.
21:21 cait ?
21:21 jcamins DBD::mysql::st execute failed: The table 'auth_header' is full
21:21 cait oh
21:21 full??
21:22 * jcamins shrugs.
21:22 cait what doest hat mean full?
21:22 jcamins I have no idea.
21:22 cait that's kinda.. scary
21:23 jcamins Agreed.
21:28 I think I'll just drop that database and pretend it didn't happen.
21:28 cait oh :(
21:28 * magnuse shares some equinoxe from valrhona with the weekend/late night crew
21:28 druthb joined #koha
21:29 cait hi druthb
21:29 druthb hi cait
21:29 magnuse "If a table-full error occurs, it may be that the disk is full or that the table has reached its maximum size." http://dev.mysql.com/doc/refma[…]n/full-table.html
21:29 * jcamins will beg you for a smaller dataset that you use for testing.
21:29 magnuse kia ora druthb!
21:29 druthb kia ora, magnuse.  :)
21:29 cait jcamins: I was interested in the results of that!
21:29 :)
21:29 but remind me to dig for some data on monday
21:30 jcamins cait: me too, but it'd take days to reindex, too.
21:30 cait hm k
21:32 I have created a small dataset for testing
21:32 quite small... but might be interesting for you, lots of links and things
21:32 jcamins Perfect!
21:32 cait waiting for the 'converted' version
21:33 so then the subjects should be correct, but it's quite small
21:33 wanted to it for testing links in the opac
21:36 jcamins Okay, how to I rebase a whole bunch of patches onto a new branch?
21:36 *do I
21:37 magnuse jcamins: are they patch files, or commits on a branch?
21:37 jcamins Commits on a branch.
21:38 Usually I'd use git format-patch and apply that, but I'd prefer to do it the right way.
21:38 magnuse "just" rebase that branch onto the branch where you want the commits, then, i would guess?
21:38 jcamins But won't that eliminate the original branch?
21:38 * magnuse was doing it the wrong way a minute ago
21:38 magnuse the branch that you want to get the commits from?
21:39 hm, i'm no expert, but i think it will
21:39 but would you want to have the commits in two places?
21:39 jcamins Yeah, that's what I'm trying not to do.
21:39 magnuse maybe cherry-picking?
21:40 jcamins Yes, because on the second branch I'm going to squash commits, and I want to have a working branch if I screw stuff up.
21:40 I thought that might be the answer.
21:42 Wait, I have an idea.
21:42 * magnuse is waiting
21:42 jcamins I can checkout a copy of my branch, then rebase the copy onto kc/master.
21:43 magnuse yeah, a copy sounds like what you want
21:45 jcamins That worked.
21:45 magnuse yay!
21:59 * magnuse has tried to rebase catalyst's l10n work on the 3.6.x branch and is now trying to build packages with translations off the result...
21:59 jcamins Woohoo!
21:59 magnuse++
22:00 magnuse well, i did say *try to*!
22:02 cait magnuse++
22:02 we believe in you :)
22:03 magnuse thanks
22:03 "debian/list-deps: unknown package for CGI::Session::Driver::memcached" does not look too promising, perhaps...
22:04 but i think i might know the solution, at least in theory
22:09 rangi yep
22:09 that module is at debian.kc
22:09 ull just need that in ur sources list
22:09 magnuse yeah, i think eythian told me to do:
22:09 sudo pbuilder create --othermirror 'deb http://debian.koha-community.org/koha [^] squeeze main'
22:10 oops, without the "[^]" of course
22:10 rangi should do it
22:10 jcamins That sounds right.
22:10 Wow. I really didn't expect this to involve quite so many lines.
22:11 +1709, -578, after squashing a bunch of small commits together.
22:13 Any idea how to sum up *all* the values in a hash?
22:17 rangi foreach loop?
22:17 * rangi attempts to burn down the house
22:17 rangi making popcorn
22:17 jcamins rangi: I was hoping there was a nicer way.
22:17 cait lol
22:18 rangi jcamins: there might be, but can't think of it
22:20 magnuse ooh, it's generating nb-NO templates from the translations...
22:20 jcamins Hooray!
22:20 magnuse still no guarantee of ultimate success...
22:21 cait crossing fingers
22:22 magnuse no luck...
22:22 dh_install: koha-common missing files (debian/tmp_docbook/*.8), aborting
22:22 make: *** [binary] Error 2
22:22 dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
22:22 E: Failed autobuilding of package
22:23 rangi looks like man page stuff
22:23 magnuse yeah, weird
22:24 rangi hmmm
22:25 magnuse well, that's a headache for another day, i'm off to watch the last half hour of "Lewis", season 5 episode 3!
22:25 rangi :)
22:25 magnuse see ya tomorrow, folks!
22:25 rangi make sure u write it up :)
22:26 cait bye magnus_away
22:26 magnus_away rangi: will do
22:26 c ya cait!
22:34 jcamins Is there a way to get git-bz to attach multiple patches with one command?
22:34 rangi not without hackery I think
22:34 jcamins That's what I figured.
22:35 cait jcamins: i do something like HEAD^^^
22:35 and only -e for the first
22:35 and then just delete one ^ after the other
22:35 but not really elegant...
22:35 jcamins cait: that's what I do, too.
22:36 rangi read up on git, how to specify a range
22:36 and try that
22:37 it might work :)
22:37 jcamins Hm.
22:37 If I remember I'll try next time.
22:37 Already posted the patches.
22:37 Bug 7284
22:37 wahanui Bug 7284 is probably the first step to making that happen.
22:37 huginn 04Bug http://bugs.koha-community.org[…]w_bug.cgi?id=7284 enhancement, P2, ---, jcamins, ASSIGNED , Authority matching algorithm improvements
22:37 rangi cool!
22:38 cait very cool :)
22:39 jcamins So, if anyone was looking for something *incredibly absurdly totally and utterly* awesome, today is your lucky day.
22:39 rangi Hehe
22:39 I do need to do authority linking
22:40 jcamins Well, give it a whirl.
22:40 rangi but not for a week or so
22:40 jcamins Grr.
22:40 :)
22:40 rangi first I have to make the authorities
22:40 from a csv file
22:40 jcamins Hmm...
22:40 Yeah, might be a good idea.
22:40 rangi on leave next week
22:41 taking the family to rotorua
22:41 jcamins Cool!
22:41 rangi to see geysers and boiling mud etc
22:42 jcamins Mmm. Boiling mud.
22:42 rangi cait and magnuse have been there
22:42 and paul, hdl and slef too
22:43 cait :)
22:43 and geysir
22:43 huginn New commit(s) needsignoff: [Bug 7284] Authority matching algorithm improvements <http://bugs.koha-community.org[…]w_bug.cgi?id=7284>
22:43 cait I have heard they have 'mud for men' at the shop there :P
22:43 maybe you should get some
22:44 rangi heh
22:44 ill pass
22:44 cait oh
22:44 and they have kiwis
22:44 rangi yep
22:44 we have some here in the zoo
22:45 u can see them at zoodoo
22:45 jcamins Kiwis? I guess this is something other than the fruit?
22:45 cait :)
22:45 rangi the fruit is kiwifruit
22:46 kiwi is the bird it takes it name from
22:46 small brown and hairy looking
22:46 jcamins Ah.
22:46 I didn't know that.
22:47 rangi I think the us drops the fruit bit off the name
22:48 maybe germany too cait?
22:48 cait yep
22:48 only kiwi here
22:58 rangi: going to take a hot bath too?... forgot how they are called
22:59 rangi hot pool
22:59 cait yes hat
22:59 that
23:00 rangi yep I think we will
23:00 cait :)
23:19 * jcamins has been evicted.
23:19 rangi oh?
23:19 jcamins A higher life form has preempted my chair.
23:19 rangi ahh
23:19 jcamins He is currently *loudly* washing his paws.
23:19 * cait waves at myshkin
23:20 jcamins When I put my face near his, he licks my nose. It's very cute.
23:20 I'm not sure exactly why.
23:23 I guess it doesn't meet his standards of cleanliness.
23:24 rangi heh
23:24 cait hehe :)
23:25 magnus_away rotorua was fun!
23:26 rangi heh you woke up to say that?
23:26 * magnus_away saw a wild kiwi on kapiti
23:26 magnus_away sure!
23:26 rangi ohh a wild one
23:27 you a literally one in 100 million or so
23:27 to have done that
23:27 magnus_away yay!
23:36 rangi @wunder nzwn
23:36 huginn rangi: The current temperature in Wellington, New Zealand is 14.0°C (12:00 PM NZDT on January 08, 2012). Conditions: Rain. Humidity: 100%. Dew Point: 14.0°C. Pressure: 29.86 in 1011 hPa (Steady).
23:36 rangi hope that stops before tomorrow
23:39 * magnus_away goes to sleep
23:39 cait good night magnuse :)
23:39 rangi sleep well
23:39 cait @wunder Konstanz
23:39 huginn cait: The current temperature in Taegerwilen, Taegerwilen, Germany is 3.8°C (12:35 AM CET on January 08, 2012). Conditions: Light Rain. Humidity: 91%. Dew Point: 2.0°C. Windchill: 4.0°C. Pressure: 30.22 in 1023.2 hPa (Steady).
23:40 jcamins Good night, cait.
23:40 cait oh
23:40 that was for magnuse :)
23:40 jcamins Yes, and that was for you. :)
23:40 cait but actually... not a bad idea
23:41 ohok
23:53 cait left #koha

← Previous day | Today | Next day → | Search | Index

koha1