← Previous day | Today | Next day → | Search | Index
All times shown according to UTC.
| Time | Nick | Message |
|---|---|---|
| 11:22 | kados | thd: are you there? |
| 11:48 | thd | kados: yes, I am here |
| 11:48 | kados | thd: have you ever imported authorities records into Koha? |
| 11:50 | thd | kados: Koha 2.X has no provision for importuning externally created authority files |
| 11:50 | s/importun/import/ | |
| 11:51 | kados | bulkauthimport ? |
| 11:52 | thd | kados: hdl had confused the English distinction between building and importing |
| 11:52 | kados | ahh |
| 11:52 | well ... I suppose it's just as well | |
| 11:53 | I have an author authority file | |
| 11:53 | for one of my clients | |
| 11:53 | but there are no 'id's ... | |
| 11:53 | thd | kados: bulkauthimport.pl was a hardly started experiment |
| 11:53 | kados | so I don't see how it could be used to link between a record, etc. |
| 11:53 | ahh | |
| 11:54 | thd | kados: I do know how it could be done |
| 11:54 | kados | how? |
| 11:55 | thd | kados: It would require writing bulkauthimport.pl |
| 11:55 | kados | right |
| 11:55 | but still, without a way to link an author in the auth file to the biblio file | |
| 11:55 | it would be pretty pointless | |
| 11:56 | thd | kados: currently that script should not be presumed to be a guide to even how to start |
| 11:56 | kados | here are a couple samples from the auth file I have: |
| 11:56 | NUMBER 1 =>LDR 00066nz 2200037o 4500100 10 _aCarmichael, Ian, _d1920-NUMBER 2 =>LDR 00053nz 2200037o 4500100 10 _aHale, Gena | |
| 11:56 | all it has is a leader and 100$a | |
| 11:56 | thd | kados: the type of authority file can be determined based on a value in the record |
| 11:57 | kados | I have both author and subject authorities from this client |
| 11:57 | thd | kados: maybe it is in the leader |
| 11:57 | kados | but as I said before, all they contain is a list of values |
| 11:58 | many of them have an identical leader | |
| 12:00 | thd | kados: there is the element that identifies whether the authority is for a personal name, corporate name, uniform title, subject, etc. |
| 12:00 | kados | yep, that's probably in the leader |
| 12:00 | but as there is no ID in the record, it's useless to me | |
| 12:00 | ie, there is no id in the auth record and no id in the MARC record | |
| 12:01 | so no way to preserve any relationship between the two | |
| 12:01 | in fact, the MARC record has authors/subjects just like normal records do | |
| 12:01 | thd | kados: I remember now and I think it is nothing more than what field number holds the main value in MARC 21 |
| 12:02 | kados | ? |
| 12:03 | thd | kados: There are rather nice FRBR relations in many authority files. |
| 12:04 | kados: so the first problem is to identify what the type of authority file is and that is easily done. | |
| 12:05 | kados: That type is then stored in a special local use field for Koha to use. | |
| 12:06 | kados | well ... like I said before, without ids in the records the auth file is useless for importing |
| 12:06 | ie, there is no way to use the auth file for import | |
| 12:06 | so I"m not going to worry about it | |
| 12:06 | I may attempt to build authorities once the records have been loaded in | |
| 12:06 | thd: another question | |
| 12:07 | thd: remember we discussed you working on another framework for MARC21 | |
| 12:07 | thd | kados: Koha can assign IDs just as id does for bibliographic records. |
| 12:07 | kados | thd: I'm wondering if you could solve the problem we currently have when a set of MARC records has itemtypes at the item level rather at the record level |
| 12:07 | thd | s/id/it/ |
| 12:08 | kados: I solved that months ago | |
| 12:08 | kados | thd: (sure it can but there is no way to relate a given bib record to a given auth record on import) |
| 12:08 | thd: really? | |
| 12:08 | thd: how? | |
| 12:08 | thd | kados: virtual libraries can do everything |
| 12:09 | kados | what's a virtual library? |
| 12:10 | thd | kados: a library that contains a branch name as well as other information that you want to use more flexibly. |
| 12:11 | kados | I don't understand how that would solve the problem |
| 12:12 | here's what I mean ... say you have a record | |
| 12:12 | it does not contain itemtype information at the level of the record, but at the level of the item | |
| 12:12 | so a single record contains holdings for | |
| 12:12 | JNF, BK, LGPRNT, etc. | |
| 12:17 | this is problematic in Koha's current default framework because we don't track itemtype at the item level | |
| 12:18 | so you lose the ability to distinguish between itemtypes within a single record | |
| 12:18 | thd: does that make sense? | |
| 12:19 | thd | kados: That makes sense as a problem and I have a hack to Koha 2 to fix that problem. |
| 12:20 | kados: Let me send you the XML file. Warning it may not be valid XML yet. | |
| 12:20 | kados | XML file? |
| 12:21 | thd | kados: Stephen uses XML as the base record on kohadocs.org |
| 12:22 | kados: My starting point had been his XML markup for the previous document about holdings. | |
| 12:23 | kados: I also wrote my proposed solution to koha-devel months ago in a very long detailed message. | |
| 12:23 | kados | I see |
| 12:24 | thd | kados: Sourceforge never archived that message although I still have the original somewhere. |
| 12:25 | kados | thd: how difficult is it to implement the solution -- can it be done easily or would it require a lot of extra work? |
| 12:25 | thd | kados: This is merely a hack on Koha 2 not the elegant way that Koha 3 should address this problem. |
| 12:25 | kados | of course |
| 12:25 | thd | kados: I was thinking of creating a default value set that set up some small changes for new Koha installs. |
| 12:37 | kados: I have sent it | |
| 12:39 | kados | thanks, I"ll take a look |
| 12:39 | thd | kados: Authorities can be imported workably into Koha 2 with the right import script. |
| 12:42 | kados | thd: it looks like your proposed solution to the itemtype problem is just to create a new record |
| 12:43 | thd | kados: Authorities import for Koha 2 is not a very difficult problem. The authorities usage that paul created is very nice and can be used to best advantage only by importing rich authority files instead of the poor ones that can be built from existing record data. |
| 12:44 | kados: what do you mean, create a new record? | |
| 12:45 | kados: look for 'material types' | |
| 12:45 | kados | A difficulty of the Koha MARC implementation in relation to the |
| 12:45 | original Koha data design will not allow the assingnment of individual | |
| 12:45 | copies to different circulation fees without creating a separate | |
| 12:45 | bibliographic record for copies with different circulating fees. | |
| 12:46 | Creating a separate bibliographic record for copies with separate | |
| 12:46 | circulating fees is the workaround for not being able to treat | |
| 12:46 | different copies differently at all. | |
| 12:46 | Reassignment of an individual | |
| 12:46 | copy to different circulation fees requires creating a separate | |
| 12:46 | bibliographic record and removing the holdings information for that | |
| 12:46 | copy from the original bibliographic record. Obviously, that should | |
| 12:46 | be corrected at some point in the future. | |
| 12:46 | that's the solution right? | |
| 12:48 | thd | kados: I know that I was just looking at that section but that may only be for when a different fee is needed. |
| 12:49 | kados: the heading for that subsection is for fees. | |
| 12:49 | kados | yep ... and that's what itemtype is generally used for |
| 12:49 | for instance, one of the records I'm looking at | |
| 12:49 | has REF and JNF in the same record | |
| 12:49 | obviously REF doesn't circulate at all | |
| 12:49 | while JNF does | |
| 12:50 | but when I import this record into Koha, I have to decide which itemtype the whole record gets | |
| 12:50 | or else split it into two records | |
| 12:50 | thd | kados: split the record |
| 12:50 | kados | yea ... |
| 12:51 | I'll need to rewrite my import script to handle that | |
| 12:51 | but it's really a pain in the arse | |
| 12:51 | thd | kados: There is also pain from the inflexibility of manipulating items in Koha |
| 12:52 | kados: One pain is shorter than the other and affects fewer people | |
| 13:03 | kados: I think that splitting is only needed if the library charges money for circulation. | |
| 13:11 | kados: I have rarely seen libraries that charge money for mere circulation of some types of material. Do any of your libraries actually do that? | |
| 13:17 | kados | i don't think so |
| 13:18 | thd | kados: Then you would not need to split the records. |
| 13:18 | kados | i think I still need to |
| 13:18 | unless I'm not understanding | |
| 13:18 | because circulation rules are set via itemtype | |
| 13:18 | thd | kados: Obviously the text needs to make that clear. |
| 13:19 | kados: I propose to change that. | |
| 13:19 | kados: Setting circulation rules by item type is crazy. | |
| 13:19 | kados | thd: right, but that solution won't really help this migration :-) |
| 13:19 | thd | kados: why not |
| 13:19 | ? | |
| 13:20 | kados | thd: because Koha currently sets circulation rules by item type in issuingrules.pl |
| 13:20 | thd | kados: Each virtual library can have its own circulation rules. |
| 13:20 | kados | it's not practical to set it up that way |
| 13:21 | there would be too many matrices to fill out | |
| 13:21 | thd | kados: The document may be incomplete or unclear on that point. |
| 13:21 | kados: That is only at setup. | |
| 13:22 | kados: you have library MAINCIRCRULE1 | |
| 13:22 | etc. | |
| 13:22 | kados | thd: there is still no way to distinguish between a REF and a JNF within a single record |
| 13:22 | ie if a record has both itemtypes, and one is not supposed to circulate | |
| 13:23 | there is no way to tell Koha that without splitting the record | |
| 13:23 | thd | kados: you have library MAINCIRCRULE1 for JNF |
| 13:23 | kados: you have library MAINCIRCRULE2 for REF | |
| 13:24 | kados | yes but REF is lost on import :-) |
| 13:24 | unless I split the record :-) | |
| 13:24 | now do you understand? :-) | |
| 13:24 | thd | kados: why is REF lost? Where do you loose it? |
| 13:24 | kados | ok ... so in this client's case |
| 13:25 | they use local use fields for holdings | |
| 13:25 | thd | kados: REF and JNF are both in the record are they not? |
| 13:25 | kados | (because this is a Dynix system they are coming from) |
| 13:25 | so ... at the record level | |
| 13:25 | they have 949 and 994 tags | |
| 13:25 | then at the item level (multiple per record) | |
| 13:25 | they have: | |
| 13:25 | 959 | |
| 13:25 | $b barcode | |
| 13:26 | $t ITYPE | |
| 13:26 | $h call # | |
| 13:26 | $a branch code | |
| 13:26 | $b secondary agend | |
| 13:26 | thd | kados: Well you need to preserve REF and JNF for the library somehow. |
| 13:26 | kados | $k collection code |
| 13:26 | normally, I would just map the 959 directly to Koha's 952 | |
| 13:26 | and the 949 would normally contain the itemtype and I could map it to Koha's 942 | |
| 13:27 | thd | kados: You were not going to throw out that data. |
| 13:27 | kados | it doesn't matter if it's in there or not |
| 13:27 | the point is, Koha can't perform any business logic on it because it doesn't know where it is | |
| 13:28 | there is no way to map one koha field (itemtype) to many MARC fields (959$t in this case) | |
| 13:28 | thd | kados: If you append that information to the correct 952 subfield for branch then you have what you need. |
| 13:28 | kados | hmmm |
| 13:29 | I think I see what you mean | |
| 13:29 | so each 'itemtype' is also a branch? | |
| 13:29 | i have a feeling that would really mess up other areas of Koha | |
| 13:29 | like reserves | |
| 13:29 | when a patron goes to reserve a book | |
| 13:29 | thd | kados: so you create the 952 in advance where branches are MAINREF, MAINJNF, etc. |
| 13:30 | kados | it asks which branch they want to pick it up from |
| 13:31 | thd | kados: You can solve more complex branch problems that this might create with branch groups. |
| 13:32 | kados: Assign all MAINXXXX to the same branch group. | |
| 13:33 | kados: I did say that text needed a day's work and there are certainly missing sections and some rewriting needed. | |
| 13:35 | kados: It probably needed another day's reflection to put it back into my mind correctly. In December, I knew exactly what was missing. | |
| 13:38 | kados: Your branch group names can be normal names like MAIN , RIVERSIDE, etc. | |
| 13:38 | kados | I'm not sure what grouping the MAINXXXX branches into a category would do |
| 13:39 | in terms of keeping them out of reserves | |
| 13:39 | so say we had a COLLECTIONCODE | |
| 13:39 | branch category | |
| 13:39 | and each branch inside was of type: | |
| 13:39 | CCJNF, CCREF, CCXXX, etc. | |
| 13:40 | and in my MARC import, I added a CC in front of the 'itemtype' field in the records | |
| 13:40 | and mapped it to the branch | |
| 13:40 | thd | kados: nothing for keeping them out of reserves. The circulation rule associated with each individual virtual branch can keep them out of reserves. |
| 13:41 | kados | no, I don't mean that |
| 13:41 | I mean, when I go to put a reserve on an item | |
| 13:41 | Koha asks me: | |
| 13:41 | which branch do you want to pick the item up at? | |
| 13:41 | and it's a drop-down box with all the branches listed | |
| 13:42 | thd | kados: you can suppress all but the branch group name. |
| 13:43 | kados: Katipo does that sort of special modification all the time to gain some functionality. | |
| 13:43 | kados | right |
| 13:44 | well ... I'll try it out | |
| 13:44 | thd | kados: Katipo that can be done with the template. |
| 13:45 | kados | thd: one more problem I think :-) |
| 13:45 | take a look at this mapping: | |
| 13:45 | > 959 10 _b33529002736014 *b is the BARCODE | |
| 13:45 | > _tGE *t is the "ITYPE" GE=GENERAL | |
| 13:45 | > _h937 F151h *h is the CALL# | |
| 13:45 | > _aSPL *a is the AGENCY | |
| 13:45 | > _bST *b is the Secondary Agency | |
| 13:45 | > _kCX *k is the Collection Code | |
| 13:45 | > _tpart I *t is the Copy/Volume | |
| 13:45 | there are TWO $t s | |
| 13:45 | !! | |
| 13:45 | the first one is the ITYPE | |
| 13:45 | the second is Copy/Volume! | |
| 13:46 | i don't think there is a way in MARC::Record to handle this | |
| 13:46 | what I mean is, I don't think I've ever encountered repeatable subfields within a tag for holdings data | |
| 13:47 | where the second field was a different TYPE than the frist | |
| 13:47 | first even | |
| 13:48 | thd | kados: change the variant to another subfield based on content. |
| 13:48 | kados | yea, I might have to do that |
| 13:49 | thd | kados: The meaning of subfields often changes in MARC based on the value of indicators or even other subfields. |
| 13:49 | kados | yep |
| 13:50 | thd | kados: MARC 21 seems to be better about not doing that based on other subfields than is my memory of US MARC. |
| 13:51 | kados: UNIMARC is nicer still in trying to not create unnecessary confusion while still being MARC. | |
| 13:52 | kados: confusion and MARC are of course synonyms of some sort for one another :) | |
| 13:52 | kados | heh |
| 13:54 | thd | kados: Authorities import into Koha 2 could work well with a fair amount of code reusable for Koha 3. |
| 13:55 | kados: The only real difficulty is the need to search existing records for matching authorised values from authorities for linking. | |
| 13:57 | kados: That would be done on record import and then a $9 is added to the matching bibliographic record to support paul's authorities implementation. | |
| 13:58 | kados | thd: or you could just rebuild your authorities list based on data in the system |
| 13:58 | thd: which is what i suspect I'll be doing | |
| 13:58 | then import a separate list of authorities for new cataloging | |
| 13:58 | wait ... I guess that wouldn't work | |
| 13:58 | sigh | |
| 13:59 | thd | kados: The build authorities from existing records is currently coded for UNIMARC. |
| 14:02 | kados: Building from bibliographic records gives poor authorities because all the disused and unauthorised variants of the authority that a user may try are absent in the bibliographic records. | |
| 14:05 | kados: Paul's code allows searching for the variant forms and then finding the authorised record if the variant forms are included from a rich standard authority record. | |
| 14:06 | s/authorised record/authorised form to use in finding the bibliographic record/ | |
| 14:08 | kados: The major issue with paul's code is that it does not manage authorised values past $a. | |
| 14:10 | kados: Even with that present defect, using authorities is fun and a great advantage in Koha. | |
| 14:11 | kados | I'm not clear on how to set them up ... |
| 14:11 | so I import my MARC records | |
| 14:11 | then build authority records from the existing data? | |
| 14:12 | I cannot import an external authority file yet, right? | |
| 14:12 | perhaps I should talk to paul about this on Monday | |
| 14:12 | thd | kados: Actually, there is no user level search for authority data, but that could easily be modified from the intranet side. |
| 14:12 | kados | there is actually |
| 14:12 | it's the [...] | |
| 14:12 | on the OPAC and Intranet advanced search pages | |
| 14:15 | thd | kados: In the OPAC, that only searches the values present in the relevant fields. The OPAC does not search other forms that appear as unauthorised forms. |
| 14:16 | kados: In record editor plugins and in the authorities module a search for an unauthorised form will provide you with the authorised form. | |
| 14:18 | kados: Both of those are only on the intranet side. The MARC record editor plugin approach could easily be added to the OPAC. | |
| 14:20 | kados: The present answer to building and not importing is yes and building is written for UNIMARC. | |
| 14:24 | kados: Paul had no plans four or so months ago to fix the authority code for using values past $a in 3.0. Maybe that is a 3.X issue. However, his libraries seem to care very little about standards. | |
| 14:26 | kados: The library profession seems to be organised differently in France where standards maniacs do not inhabit every library there. | |
| 14:28 | kados: All his libraries encode authors differently in UNIMARC, and none of them in a UNIMARC compliant manner. | |
| 14:29 | kados: I can see what is present in their records I merely lack the history of how that evolved. | |
| 14:35 | kados: My problems yesterday were with your CVS symlink script and some recent changes in template behaviour. Your symlinks to the templates seem to derive from some version of Koha prior to at least 2.23 when the templates directory may have been organised differently. | |
| 14:36 | kados: That script would continue to work for older installs because of the koha.conf setting for the template directory. | |
| 14:39 | kados: are you still there? | |
| 20:13 | kados | sam:/home/koha/testing/cvsrepo/koha# misc/migration_tools/rebuild_zebra.pl -cGlobal symbol "$service_type" requires explicit package name at C4/Biblio.pm line 230.Compilation failed in require at misc/migration_tools/rebuild_zebra.pl line 10.BEGIN failed--compilation aborted at misc/migration_tools/rebuild_zebra.pl line 10. |
| 07:09 | paul | |hdl| tu es pas ici ? |
| 07:09 | s/pas/par/ | |
| 07:10 | |hdl| | oui |
| 07:10 | paul | un petit skype alors ? |
| 09:20 | osmoze | hello |
| 09:38 | paul | hello osmoze. |
| 09:38 | tiens, je vais jeter un coup d'oeil sur ton problème. | |
| 09:38 | osmoze | :) bonne nouvelle :) |
| 09:40 | par contre, je suis en repos aujourd hui et les JO risquent de ne pas me faire repondre si t as des questions ^^ ( mais je zieute de temps en temps) | |
| 10:26 | kados | hi all |
| 10:27 | paul | hello kados |
| 10:27 | good morning | |
| 10:27 | kados | z3950_extended_services |
| 10:28 | no code calls it yet ... | |
| 10:30 | paul: question for you about authorities | |
| 10:30 | a client has sent me an authority file | |
| 10:30 | it is MARC21 format | |
| 10:30 | so it has author in 100$a etc. | |
| 10:31 | but there are no record ids ... and in the client's MARC data the subjects and authors are there (instead of an id) | |
| 10:31 | am I correct that there is no way to import the authorities file | |
| 10:32 | since there is no way to match which authority record goes with which bibliographic record? | |
| 10:35 | paul | you're right. |
| 10:35 | the only solution is to write some code to find the matching depending on the content. | |
| 10:35 | if you are lucky (or the file is well formed), you should have no problems. | |
| 10:36 | kados | but really, I could just build a new authorities file from the data, no? |
| 10:36 | ie, import the MARC records | |
| 10:36 | paul | (mainly for subjects. for authors, you can't be sure there is only 1 joshua ferraro, and if there are 2, who is who. except if in your authority AND biblio you have dirthdate for example) |
| 10:36 | kados | then build an authority file? |
| 10:36 | paul | that could work, except if some authority isn't used, even if in the authority file. |
| 10:37 | (that will work...) | |
| 10:37 | kados | (the authority file does have birthdate) |
| 10:37 | paul | that's how I usually do it. |
| 10:37 | ok for birthdate and authority, but are birthdate in biblios too ! | |
| 10:37 | otherwise, it's quite useless. | |
| 10:37 | kados | no :( |
| 10:37 | paul | (to find the match I mean) |
| 10:37 | kados | exactly |
| 10:38 | paul | so, if you do what you plan, you will miss the birthdate. Maybe it's important for your library |
| 10:38 | kados | I will ask the client if they can attach a system identifier to the MARC record and the authority file |
| 10:38 | as this would solve the problem | |
| 10:39 | paul | for sure ! |
| 10:42 | kados | paul: second question has to do with updatedatabase in head |
| 10:43 | I would like to merge in changes from rel_2_2 | |
| 10:43 | but when I created a diff file it was over 2000 lines long ! | |
| 10:44 | paul | I cutted a LOT of things in head. |
| 10:44 | you should do a diff from 225 tag only | |
| 10:44 | -j option in cvs iirc | |
| 10:44 | kados | ok ... I'll try that |
| 10:51 | paul | kados : no news from DBD::mysql maintainer ? |
| 10:54 | kados | paul: no ... |
| 10:54 | paul: but I will contact mysqlab this morning | |
| 10:55 | paul: to see if they will pressure him to respond to me :-) | |
| 10:55 | paul | good idea i hope. |
| 10:55 | do you think that what I discover is right ? | |
| 10:55 | kados | if not, we can patch DBD::mysql and include it in C4 |
| 10:55 | paul | (right = correct) |
| 10:56 | you're probably right. | |
| 10:56 | kados | paul: about the utf-8 handling? |
| 10:56 | paul: (is that what you're asking?) | |
| 10:56 | paul | or provide a patched version of dbd::mysql and make it mandatory in Koha install |
| 10:56 | yep | |
| 10:56 | do you think my diagnostic is correct ? | |
| 10:56 | kados | I haven't tested it myself, but from the number of references you have provided I'd say it's right |
| 10:57 | paul | (I might have missed something or misunderstand something else) |
| 10:57 | kados | we could test by patching DBD::mysql and see if it fixes it |
| 10:57 | (shouldn't be too hard as it's Perl, right?) | |
| 10:57 | paul | call mysqlab, and if you have no solution, i'll try. |
| 10:57 | kados | ok |
| 10:57 | paul | but that's not an acceptable solution for public release ! |
| 10:57 | kados | in about an hour or so I will call |
| 10:58 | I agree it's not ideal | |
| 10:58 | paul | 3pm for me, so i'll be here for some time still |
| 10:58 | kados | paul: cvs update -d 225 updatedatabase |
| 10:58 | paul: is that correct? | |
| 10:59 | oops ... s/-d/-j/ | |
| 10:59 | paul | mmm... I had 2 j iirc |
| 10:59 | something like cvs update -j rel_2_2 -j 225 updatedatabase |
← Previous day | Today | Next day → | Search | Index