| Author |
Messages |
|
vincent kelly
Posts:2
 |
| 05-03-2008 3:33 AM |
 Alert
|
Dear All,
We are looking at moving to high end digital photography for multi purpose use, i.e., picture library, income generation,web use etc...Has anyone already done this and used the Multi Media Module to store high resolution images on. We are currently looking at importing 50mb RAW files to the system to be automatically resized for web use from the database.
I heard that Galleries in Australia and NZ may have already developed the module to do this kind of thing. Any advice would be much appreciated. Thanks. |
|
|
|
|
Adrian Kingston
Posts:12
|
| 05-03-2008 9:12 AM |
Alert
|
Hi Vincent I'll keep this as short as possible:
At Te Papa we use the Media Asset functionality, which is a combination Multimedia and Catalogue, and has an emphasis on long-term management (attempting to conform to standards such as NISO Z39.87, NLNZ preservation metadata standards etc) as well as enabling the creation, management, access and security of images of different levels/statuses.
We also use a tiered repository structure and Record Level Security to manage access to images through EMu and web.
Record level security manages the collections each user has access to, e.g. art curator has access to art records only (and therefore art images only), a Conservator would has access to most collections etc.
We have three repositories; read and write access to these repositories is managed by EMu:
1. The General Repository (on SAN), the general staff (Collection Managers, Curators, Conservation etc) write and read images to/from. These images range from temporary identification images (generally less than 3 MB, jpeg) to treatment images (18MB tiff) to higher quality/resolution images of collection images (eg of botanical specimen sheets) which could be up to 30MB tiff. And everything in between. The General Repository also holds ALL the derivatives of multimedia in EMu, which allows (depending on RLS) collections staff access to the derivatives of the Imaging Team’s images.
2. The Restricted Repository (on SAN) holds the full sized tiffs created by our Imaging Team. These range from 10MB (older images) to over 100MB. Most are between 40 and 60MB. Derivatives for these are written to the General Repository. Only the Imaging Team and Admin have access to this repository.
3. The Preservation Repository (on WORM UDO jukebox) holds images that meet our reasonably strict criteria for Digital Preservation Masters (which range from 150 to 800MB tiff depending on the source object). Derivatives for these are written to the General Repository.
This repository also stores our collected digital works (e.g. artworks with digital components) however these are managed as digital Collection Objects, not Media Assets. When we have a bit more budget (and expertise) for migrating our small but high risk film/video collections, these will most likely end up on this repository/UDO too. Only I have access to this repository.
Our Collections online (http://collections.tepapa.govt.nz/search.aspx) only has access to the General Repository, using only jpeg derivatives, but the derivatives are usually 2400 or 3000 on the longest dimension, which allows us to create Zoomify versions on the fly without having to use our large tiffs. See http://collections.tepapa.govt.nz/objectdetails.aspx?oid=127535&coltype=photography®no=d.000035 and click on the image for a Zoomify example.
As an aside, we don't maintain any type of RAW images formats in EMu; just don't have any faith in their long-term sustainability, not even in DNG yet, much as our Imaging Team would like to.
Adrian Kingston
Collections Information Manager - Digital Assets and Development
Museum of New Zealand Te Papa Tongarewa
|
|
Adrian Kingston
Acting Collections Information Manager
Museum of New Zealand Te Papa Tongarewa
|
|
|
EMuUsers Administrator
Posts:41
|
| 06-03-2008 11:24 AM |
Alert
|
Hi Vincent
Short answer is yes; at Museum Victoria (MV) we are doing this, with Tiffs and DNG, which average around 120MB in size. (We're very comfortable with DNG; recommend you research the pro's/con's and form your own opinion re this).
Okay, now here's the fine print...
I am not sure if you need to be able to view these images in your EMu client. If so, then there are a few issues that you need to be aware of, for which (fortunatly) there are workarounds.
a) Thumbnail generation is performed by the EMu client. At this stage, generation of thumbnails derived from certain formats, including RAW, is unsupported
b) The image viewer built into the EMu client is unable to display some formats, inc. RAW
c) Not sure if you want to load hi-res tiffs. Tiffs are supported, but there can be significant performance issues when these images (or indeed any large files) are viewed using the EMu client, for two reasons:
1) The image has to be copied down to the client side from the server - larger the file, the longer this takes
2) The image has to be opened by the image viewer built into the EMu client - again, the larger the file, the longer this takes
Local caching of images can mitigate the 1st problem to some degree, but how effective this is depends on how many of these images you are looking at, how large they are, how large your cache is set to (which you can configure in your EMu client), and how fast your local disk is etc etc.
Even if local caching helps somewhat, the 2nd problem remains: a 50mb tiff takes more time to open in the image viewer built into the EMu client than does a 1000 kb jpeg.
These performance issues are very noticeable and (in our experience) frustrating for users if they are paging through sets of MMR records. When one of these records is viewed, the EMu client tends to hangs while it loads the image. This takes several seconds - or much longer for seriously large files.
Fortunately, it is possible to work around this.
-------------------
The TIFF Workaround
-------------------
At MV, we deal with large tiffs in the MMR as follows.
When we insert a tiff into the MMR, a jpeg derivative created automatically. It becomes one of the "resolutions" available to the record. So a jpeg "resolution" is available for every tiff.
We have written a script which runs on our server every night. Any tiff records in the MMR are modified, so that the jpeg "resolution" becomes the "primary" image, and the tiff "primary" image becomes a "resolution".
When a user next views the record the EMu client only has to deal with the jpeg version, which is much faster. The tiff remains available if you want to view it.
----------------
Dealing with RAW
----------------
We are only just starting to store DNG files in the MMR. So far, we haven't needed to worry about viewing these in the EMu client. If we did, then this is how we would deal with it:
You need a script which finds records for RAW files. For each record, it:
- creates a jpeg derivative (using something like Imagemagick)
- creates a thumbnail (Imagemagik again)
- makes the jpeg the "primary" image, and the RAW file a "resolution"
So now you can see the jpeg version of your RAW image from the EMu client (and from the web if you have this hooked up). The RAW image is still available as a "resolution".
This process works for any image format not natively supported by EMu (Jpeg 2000 for example) - as long as the image conversion tool you are using supports that format (Imagemagik is capable of supporting lots of formats)
-------------------------
Storage management issues
-------------------------
One other thing to consider; if you're intending to deal with large volumes of large files, then the default way of storing them on the EMu server may not be appropriate. EMu is able to resolve file paths from a number of different potential storage locations. Storing the larger files elsewhere can make it much easier to manage the files (backup etc) and may bring performance benefits.
At MV, we have a script running overnight which moves all files larger than 1Mb to our SAN. (It is possible to get this to happen on the fly but is easier to implement as a batch process).
------------------------
DIGITAL ASSET MANAGEMENT
------------------------
The following is not directly relevant to your question, but may be of interest.
At Museum Victoria, we are in the process of implementing a third party digital asset management system (DAMS). The DAMS will be integrated with EMu through the use of a web service; we developed the functional brief and asked KE to build it for us.
Integration of the DAMS and EMu is important; many of the images that (for various reasons) need to available in the DAMS also have a function in EMu, as a reference image attached to other EMu module records and/or images to be served to the web via pages built using KE's php classes.
The MMR will serve master/co-master images to the DAMS. DAMS users will be able to ingest these files directly into the MMR via the DAMS interface. Most of the time (but not always), these will be hi-res DNG/Tiff files
Many of the records in the MMR do not need to be available to the DAMS, and many images in the DAMS have no relevance to EMu. Therefore, we have included into the system the ability for records in the MMR to be flagged as either "external", "internal" or "Shared".
* Internal records are available as normal to EMu users; they are not exposed to the DAMS.
* External records are stored in the MMR, but are invisible to the EMu client; they are only available to the DAMS.
* Shared records are available to the DAMS, and visible in EMu; updates to "shared" records/resources may only be made in the DAMS.
Let me know if any of the above requires further clarification.
Regards
Forbes Hawkins
Collection Systems Senior Developer
Museum Victoria
|
|
|
|
|
vincent kelly
Posts:2
|
| 07-03-2008 10:33 PM |
Alert
|
Forbes,
Many thanks for your speedy repsonse to my query.
I have had a quick read through the information that you have very kindly passed on and can say there is quite a lot to take in that I will have to think about and discuss with colleagues.
The information from Te Papa and yourself is incredibly useful and will help shape out thinking in this area.
I hope you don't mind if I come back to you later to clarify anything that I may discuss with colleagues that raises further queries.
It certainly looks like RAW files are not the way to go, and that perhaps we should stick with TIFF files for high resolution images.
Hope you have a good weekend in OZ
Vincent |
|
|
|
|
Karen Lovaas
Posts:12
|
| 08-03-2008 4:18 AM |
Alert
|
Adrian and Forbes: Thanks for the information about how your organizations are managing digital assets. A couple of follow-up questions:
1) Adrian, you mention EMu's "Media Assets functionality". Are there any special fields or tabs that KE has developed to provide this functionality? Or are you just using standard EMu Catalog and Multimedia module fields?
2) Have either of your organizations written any best practices guidelines for images to be managed in your systems, or workflow or data documentation that you'd be willing to share?
Thanks again for the info. It's very helpful for those of us just beginning to dip our toes in the water.
Karen Lovaas
CMS Manager
Minnesota Historical Society
St. Paul, MN |
|
|
|
|
Adrian Kingston
Posts:12
|
| 10-03-2008 12:27 PM |
Alert
|
Hi Karen
Media Assets includes a hierarchy tab (works in a similar way to parts) for managing versions, and relating the Media Asset to the Collection Object. There is functionality involved in reading information to and from the Media Asset and Multimedia records etc, automatically attaching the MM record to the appropriate Collection Object, copying common information between versions etc
We designed our catalogue tabs, and these include Digital Production, Digital Image and Digital Audio/Video/Text tabs (these aren't used as well as we would like, though we are automatically recording basic technical info (filesize,height/width/bit depth etc). We designed tabs to be common to Collection and Media Asset records, particularly as Media Assets can be analogue or digital, and we have digital Collection objects also. Our Multimedia module is pretty standard. Multimedia records the EXIF/IPTC/XMP as per standard EMU functionality.
We also have an Image Import admin tool, though this is used exclusively to load large Media Asset backlogs and the Imaging Team's images. The general collection staff use the client to load images one at a time (forcing them to be more considered and deliberate with what they add and info that goes with the images).
I have attached our Media Asset document from when we first went live with Media Assets to the general Collection staff. Unfortunately it is slightly out of date already, as we have made some minor changes since then, and are also waiting on a new version of our client which has some addition modifications, but it should at least give you an idea. It also only covers General Staff Media Assets, so does not address any criteria for our Digital Preservation Masters etc
You may (or may not) also want to check out http://www.emuusers.org/Forums/tabid/57/view/topic/forumid/14/postid/524/Default.aspx for the initial online discussion started by KE a while ago, though the discussion didn't go very far. Again this might be a bit out of date now.
Adrian Kingston
Collections Information Manager - Digital Assets and Development
Collections Information Department
Museum of New Zealand Te Papa Tongarewa |
Attachment: KE EMU Media Asset Manual - general staff version.
|
Adrian Kingston
Acting Collections Information Manager
Museum of New Zealand Te Papa Tongarewa
|
|
|
|
| You are not authorized to post a reply. |
|
|
|
ActiveForums 3.6
|