Video – DNG and Auto-Saving Changes
Well everyone, after this week I think we will have successfully beat this DNG thing into the ground. It’s all good though. I think the topic is worth a little extra time. Anyway, I realized there was one question that came up a lot. It deals with if/how/when Lightroom will save all of your edits to the DNG files and where those edits get saved to. After trying to write out a long explanation I figured we could cover a lot more ground in a video.
Enjoy and have a great weekend!
Click here to download the video to your computer. [Right-click and choose the “Save As” option]
Hey Matt – been using DNG as a result of your recommendation for over a year and very happy with it. I’m using Lightroom 2 and have selected the preference box to force Lightroom changes to save into the actual DNG file.
Ever run into the problem that even though the preference option is selected, it doesn’t automatically save changes to the DNG file? If I manually force the update (via menu command) it works, but not otherwise.
Appreciate anyone’s thoughts on this.
“Good video.Thanks a lot.Didnt find it in preferences because I was looking for dng but found only xmp which was very confusing.”
Matt probably wanted to gloss over this because he didn’t want to get bogged down in terminology, but what the preference is saying is “update changes to the XMP block automatically.” Whether the XMP block is stored in a sidecar file, or in the XMP block part of the DNG file, the XMP block is updated with the develop settings.
Because and XMP file is an abstraction of what would normally be part of the file. Adobe doesn’t want to attempt to write into camera specific raw files, though, so it came up with the sidecar concept.
But the key take-way is that the preference is updating the XMP block, wherever that happens to be.
“Still I dont get the problem about the whole point. If I want to give somebody my dng pic, I dont go to Finder and copy the pic from there.
I export the pic within LR as a dng. The advantage is that whilst exporting it, all changes are written from the LR catalogue to the dng file during the export process. Am I correct?”
Perhaps an actual export would do this (perhaps someone else can verify this) but if you have more than one application sharing the same images that both understand DNG (i.e., Bridge and ACR along with Lr) then the only way they can communicate the processing develop steps is via XMP.
I don’t think turning on the option in Lightroom preferences is a good idea. Every time you tweak something LR will be writing to the files bogging down your system. Think about syncing 200 files, its going to be busy writing for a while.
It is much better to wait until you are finished editing. The select all the files and write to them.
My two cents.
my photos are converted to dng on import. last night i checked the “autosave changes” box. Today everything runs incredibly slow…so i unchecked the box “autosave changes”. Is there a way to reverse so the files don’t have all that data embedded into the dng? help Please.
Matt, thanks for bringing this all up and clear a lot of questions!
Updating the DNGs is the single most important thing you can do to work future proof when you don t want to be depended on a programm or let s even say one lr catalog.
one question – why does it make the files bigger?
Ben, I just rechecked and the Date Modified attribute of the DNG file gets updated in about 8-10 seconds. What program are you using to view the pictures? If I use Picasa’s viewer (just dbl-clicking in Windows explorer) the original image appears. Picasa and maybe other graphic viewers may not use the metadata to display the final version. If I open the DNG in PS4 Camera Raw the final version does appear (and I can reset it back to original in CR)
Here’s a liitle tip, if you convert to B&W in LR the file will open in B&W and will remain in B&W in CR even after a reset to default!!! What you have to do in CR is click on the HSL/Grayscale button and uncheck Grayscale to see the original color image!
John said: “Adobe maintains patent rights to key portions of the DNG standard; they reserve the right to revoke the use of that license at their whim (which is why DNG was rejected as an official ISO standard)”.
Adobe said in September 2009 “there are no known intellectual property encumbrances or license requirements for DNG”. (The license is reassurance for lawyers that Adobe won’t object if you use DNG. It certainly doesn’t claim that there ARE patents on DNG!)
ISO have not rejected DNG. In fact, they are using it in the revision of ISO 12234-2 (TIFF/EP).
DNG the same status as TIFF (until ISO take over): a free-to-use file format owned by Adobe. Who is worried about TIFF? So why be worried about DNG?
Hi again
DNG still makes me nervous. I use RAW as the digital original. I do backups of these RAW files. This leaves me with two copies, one on the backup media and one in the LR structure. Works like a charm and ensures that I sleep well 🙂
If I would convert my RAW’s into DNG, the scenario shifts, As soon as I save edits to the DNG file I no longer have 2 copies of the original since the LR one is modified. This makes me nervous and ultimately forces me to make a backup of my backup. Net result, double the storage for my backups. This outweights the 20% size reduction of DNG files by a large amount.
Regards
/Thomas
It’s been discussed that having “Automatically write changes into XMP” turned on will slow LR’s performance. The way I save the metadata to the file is just hit Ctrl/S once I’m done making my edits (I do this after moving the image back into Library but I would assume you could do it while still in Develop…). Another way is to click on the downward facing arrow that shows up in the upper right corner of the “mount” after moving back into Library.
I hope I haven’t missed this being discussed elsewhere in these comments, but what I didn’t get was this:
IF you select saving the data as a Catalog Pref, when do the saves occur. Matt: you immediately move to the Finder without doing anything like a save, and the changes are visible in the file. How is this different from manually writing the changes? Why bother with the latter?
Thanks for all this info. Is there a way to automatically update the preview rather than manually updating?
Great video. Between you and Julieanne I have been convinced about DNG files. Believe it or not I still have one question!!! Over the past year I have been making less and less post processing with Photoshop. When I want to do something crazy I do it there. Therefore, most changes have been non-distructive to my raw files. Now exactly what are the steps & options (if any) for doing one complete conversion to DNG with those changes being included. Currently, I export JPEGs to a subfolder on the desktop . From there I access these pics with what ever program I happen to be using.
As an aside I think the Adobe training videos that you and Scott and Julianne have done are amazing. It was that support that lead me to drop my Aperture program in favor of LR. Keep up the great work.
Hmm… I have used the ” Update DNG preview & Metadata” on my library, as it seemed to be a no-brainer after watching this video. Two side-effects: Firstly, I seem to save a lot of disk space, as all the .bak files get deleted and they don’t seem to be recreated after switching the “Automatically Save to XMP” option on. Not bad… However, I can no longer toggle between “before” and “after” on all of my legacy images! The only way to see the “before” image is to use the LR history function. Man, you should have mentioned that. I am glad I am running Time Machine…
Dear Tony,
Your files are growing bigger because you have the “Embed Original Raw file” preference switch turned on. With this switch active your dng file has three sections– the original raw file, a dng version, and an xmp metadata section. Putting all three in the same container makes for a giant file. Turn off this preference and then your files will get smaller because the dng container will only have the dng version and the xmp metadata section.
If you are interested I have a video tutorial that covers the three ways to convert to dng within Lightroom.
http://thelightroomlab.com/2009/06/converting-digital-camera-raw-files-to-the-dng-format-using-adobe-photoshop-lightroom/
hope this helps,
David Marx
http://www.thelightroomlab.com
http://www.davidmarx.com
Fred, can you expand on that, please.
I’ve converted a DNG in LR 2.6, to B&W and again, modifications never get written to disc (the photo remains in color when looking at ) unless I save them manually. And yes, I have checked “Automatically write XMP..”
Config.: Win XP.
Great stuff, as usual.
hey Nathaniel T. ,
thats cool i didnt know that. it could save some time doing that while editing and then only doing the ones i like.
Thanks!
Ron
Thanks for the excellent video. Does the jpeg preview in dng get updated when you select “Automatically write changes into xmp”?
“HEY Davem Photography ,
you can go to adobe.com and get the dng converter. Its free and will do folers of raws as batches.
Ron”
That’s true Ron.
I just found the way to convert in Lr.
In the Library Module select the images you want to convert> Library menu> Convert to DNG. I tried it out and it seems to work quite well.
By doing it inside Lr you can save a couple steps. I dont know that there is a way to select a bunch of folders in Lr all at one time, but you can run the multiple conversions at once.
That is true Ron. I just found the way to convert in Lr.
In the Library Module select the images you want to convert> Library menu> Convert to DNG. I tried it out and it seems to work quite well.
By doing it inside Lr you can save a couple steps. I don’t know that there is a way to select a bunch of folders in Lr all at one time, but you can run the multiple conversions at once.
I have a question about the gradient filter.
I use LR 2.6 as well on a laptop (Vista) as on a desktop (Windows7).
Applying a first gradient filter is’nt a problem and the applied effect remains inside the filter limits, but applying a second gradient filter to the same image changes also the applied effects in the first filter.
For example: the first gradient filter is only an exposure filter and the second is only a contrast filter. Well, changing the contrast value in the second filter changes also the contrast of the whole image ouside the filter bounderies!
I’m not sure, but this bug wasn’t in LR2.4 where I used the last time a graduated filter.
Are you aware of this problem?
thanks
I haven’t checked using LR3 but make sure you’re looking at “Date Modified”
I made that mistake initially myself.
LR 3 doesn’t seem to automatically save the modifications on the DNG file. At least it didn’t do it in the past 10 minutes while I let LR idle or when I switch do another file in the Develop panel.
A bug?
No, DNG does not require you to go back into Lightroom to reset the image.
After making changes to a DNG file I exited LR, renamed and move the DNG file and then imported it into LR. I was able to reset the image back to the original. The Lightroom catalog would not have had any information to restore the original.
I then took another DNG file, edited it LR and exited. Opened the DNG into PS4 (Camera Raw) and clicked on DEFAULT which restored the original.
Finally, I took another DNG, made changes in PS4 (Camera Raw) saved the DNG file. Then imported it into Lightroom and clicked on Reset which again restored the original image!
So, yes, the DNG does get touched but the information to restore back to the original is still there. It’s like having the XMP file info embedded in the DNG file.
Fred,
True, thats the point of “RAW” in the first place.
But DNG forces you to go back into LR to undo everything before having a clean file.
RAW + XMP does not touch the RAW file at all so you can still have a clean RAW file if you want to do something else with it. I do hundreds of sports pics at a time in LR but a few are edited further. Which I sometimes do with other photo editing programs. And other programs may not see any DNG changes or may not even read DNG files.
Even the few files I edit in Capture NX2 (shudder), I save with a new file name, another way to keep the original DNG if you only edit a few files.
Bryan
After some testing it appears the DNG file will allow you to Reset ALL changes back to the original image. The DNG does not retain all the Edit History but neither does the RAW file with the XMP sidecar.
If you make any changes and quickly quit Lightroom the DNG will not get updated. When you return to Lightroom the DNG file will then update.
(You must set the Update XMP Automatically in the Catalog settings)
So with that option on your DNG files will have the latest update along with the ability to reset back to the original image.
I agree Leszel!
No point to dng at all really from my perspective still, after all this.
XMP also saves the trouble of backing up or transferrring LR catalogs too, just re-import the pictures.
Bryan
Hi Matt, if I understood correctly you’ve just convinced me to switch back to RAW format. The DNG saves just my original file, or an edited version, while RAW stores both. That’s a strong argument in the DNG vs RAW discussion.
I’m sorry for any mistakes, English is not my native language
Cheers
Leszek
Seems that either way there are +’s and -‘s. So, why not use Lightroom to backup your original images on import (I use a separate directory structure to backup my original .nef files ), and then go ahead and use DNG on the import. You want a backup anyway – so why not the original file. As for your working copy, there are advantages to DNG (as has been posted here and elsewhere). The major negative is the space – but you have to get past that. Space is CHEAP. I use a laptop with 2 1TB disk drives (a total $200 investment). I put both sets of files onto one drive, then have a nightly task (via a free piece of sw) that copies any changes on the first drive to the second one.
I think you all are paradoid,
its just the legal mumbo jumbo of the patent.
John, that’s interesting, so in that case it’s back to square one and this whole discussion about converting to DNG and it’s benefits is essentially worthless because at the end of the day Adobe owns the patent and if they wanted to they could revoke it tomorrow. I’m going back to NEF.
>>”If I were a Nikon or Canon shooter Id be beating on them to add save RAW as DNG right in the camera the way Pentax, Hasselblad, and Leica do, to name a few.”
Adobe maintains patent rights to key portions of the DNG “standard”; they reserve the right to revoke the use of that license at their whim (which is why DNG was rejected as an official ISO standard):
http://www.adobe.com/products/dng/license.html
Pentax, Hasselblad, and Leica are taking a huge risk that their relationship with Adobe will remain positive and productive; they take that risk because they can’t be sure other software vendors who write conversion software will support yet another proprietary raw format beyond the Big Two.
The odds of Nikon and Canon chaining themselves to Adobe’s good graces unnecessarily are slim and none.
HEY Davem Photography ,
you can go to adobe.com and get the dng converter. Its free and will do folers of raws as batches.
Ron
I get it….what is not clear is once the DNG file is written with your changes can you ever go back to the original file? For example….you have file.DNG make changes in LR and then save those change to file.DNG….if you close LR and open LR and open the file.DNG and hit RESET doe the file.DNG go back to the original file with non of the changes or not? I thought DNG was non destructive?
I am work so can’t try this out just yet.
Great explanation, Matt. My question is why would you want to make that permanent change to the DNG file? I work in Lightroom and the changes I make are saved in that catalog, so what is the reason I would want to change the original DNG file? To my way of thinking, if I have the original, I can always go back to it to process it differently. For example, I might process something in color and then decide a while later to process it again in B&W. Don’t all of those DNG changes start getting in the way?
Thanks Matt for clearing up the save issue. I hope Adobe adds “and or DNG” to that option.
If I were a Nikon or Canon shooter I’d be beating on them to add save RAW as DNG right in the camera the way Pentax, Hasselblad, and Leica do, to name a few.
Wait! We haven’t yet beat it to the ground! I just found out that the Wildlife Photographer of the Year competition requires the original raw file if you have been chosen as a winner. Here’s the paragraph:
RAW file (eg *.NEF, *.CR2, *.CRW), DNG files are only permitted if DNG is the native RAW format of the camera.
I’ve converted all of my images since I started using digital. I didn’t save any raw files. And I know I could win!
Everything in the article and the video makes great sense to me, thanks Matt. I have now started experimenting with DNGs in Lightroom, one problem though the files end up twice the size of the CR2’s straight out of a Canon 5D MKII. I was expecting them to be 20% smaller. Any comments or is there a setting in Lightroom I am missing?
Matt, Thanks for the video. Didn’t realize that my changes weren’t being saved in the XMP file, just in the LR catalog. Your video made it easier to understand. I’ll be changing my preferences today. Not sure why it’s not turned on by default. Much like you I haven’t made the switch from RAW, but considering it. I would be interested when you do make the switch and what to do about the RAW images I already have. How I do I go about converting them to DNG? Thanks so much.
The new season of BURNT NOTICE will start pretty soon … how will you manage to post here with so many things to do?
;-)))))
Man … there is NO WORDS to say how appreciative we are to get this help from you Matt.
God bless all your efforts and the family to be so supportive either!
C’ya.
I’m at work so I cannot test this but I think the Edit History is not saved in the DNG (or the XMP) files. If I convert all my images to DNG and saved the metadata I would be changing the DNG and the original image would be lost (except for the undo history function of Lightroom)
But with RAW files and seperate XMP files I assume the original RAW file remains untouched?
Helpful…I think you are right Erol…this is no different…if you live in LR…like me…no change
For got to mention…
Normally I would export the vcopy as a .dng and re-import to LR.
Thanks for the video explanation Matt.
I have a question that stems from that.
I have a .dng image, my preferences are set to auto save metadata, so I do all my edits etc. Now my client wants a b&w version of the file, but I also want to keep the original colour version. To do this I make a virtual copy and I make my b&w changes to the vcopy. My question is, where does the metadata changes get stored?
Cheers
Owin
Hello,
It’s this write to DNG file that bothers me. As a computer consultant I’ve seen a number of failures when writing to files. If I have an original that I want to keep intact, I surtenly will not allow anything to write into it.
I know that I can’t stop all writes beccause I copy the file and so on but I want to limit this to a minimum. Yea you say, just don’t tick the “save… XMP” option then. The problem as I see it is that vendors are encoruaged to use this feature of the DNG format…
I would love a universal and standardized RAW picture format, a format that is treated as a “no-touch” format with embedded checksums and stuff to ensure the integrity.
I might have misunderstood things, plase correct me if I have 😉
Regards
Thomas
Hello,
very intersting video – many thanks for it.
For DNG, I think the biggest issue is that they don’t offer enough space for very heavy developments. I heared from a big vendor of plugins that they cannot write their changes into the database of Lr because all changes for one photo needs to be put into DNG ( when using this format ) too. And exactly DNGs don’t offer enough space for such changes, so they have to create a TIF-file when using their sharpening or denoising plugins.
Best wishes
Robert
@Simone: I tried to “Save metadata to file” and it worked, too. If there is a difference I also appreciate an answer to your question.
Regards,
Knut
Thanks, Matt, I think that makes it clear.
For me it’s a non issue if I’m using Photoshop Elements. I just choose the option in Lightroom to Export to DNG and then when I open it in Elements, it preserves the edits.
Erol, it is a problem if you’re exporting the file and want to view it with ‘Preview’ since as Matt pointed out, the edits are not saved to DNG. The way I see it is, 99% of the time I edit the image and whoever I send it to gets the Jpeg version. For commercial photographers this might be relevant, but for most of us I’m not sure:).
matt,
how about virtual copies? and history? can you have several versions of one dng and all those versions stored in dng file?
Hi and thanks for some nice DNG posts!
Have one question though;
I just yesterday activated the “Automatically write changes into XMP”, which is nice. But how do I update all my old files that where edited in LR before this setting was turned on?
Do I have to select ALL photos (that’s a lot of photos) and then select “Update DNG preview & Metadata”? This could take forever…
Thanks for a great site!
That was very helpful, Matt, thanks.
Just one more question: what’s the difference between “update DNG Preview and Metadata” and “save metadata to file”?
Cheers
Simone
Good video.Thanks a lot.Didn’t find it in preferences because I was looking for dng but found only xmp which was very confusing.
Still I don’t get the problem about the whole point. If I want to give somebody my dng pic, I don’t go to Finder and copy the pic from there.
I export the pic within LR as a dng. The advantage is that whilst exporting it, all changes are written from the LR catalogue to the dng file during the export process. Am I correct??
Excellent explanation! Many thx!