I'm trying to figure out the right way to access the CinemaDNG frame rate using the DNG SDK --- preferably without having to hack up the DNG SDK, since that would have to be redone each time the DNG SDK changes. It looks like all the code has little loops for scanning tag directories, and when something unknown is found, it is dropped silently with no callback, so that there is no way to process out-of-scope tags such as those for CinemaDNG without replicating the original code.

On initial study it looks like the Cinema DNG tags, as produced by Blackmagic cameras, are DNG TIFF-like data, not as XMP-only data as the CinemaDNG 1.1 standard seems to suggest (Annex A). I suspect this is because the odds of a separate XMP file being carried around that matches the DNG images is low. Not apparent that they can be retrieved from either place.

So I'm wondering what the nominal Cinema DNG overlay onto the DNG SDK is. This is low-hanging fruit.

If the entire XMP library was to fall out of DNG and Cinema DNG, the world would be a much simpler place. Too many answers for too little problem. It's taken about a week to get the DNG SDK library to compile on Linux, with Windows still in progress and Mac to go. (cmake another non-helpful thing.... too much that does too little --- that doesn't match my build environment). KISS!

Hello all, I'm trying to build a proper DNG file from the raw CFA image acquired directly form an industrial camera.

As usually there are a number of bad pixels around and I would like to mark them in the DNG so they can be correctly interpolated when the image is processed.

In the latest DNG SDK (1.4) I've only found this rather thin references:

"Defective Pixels

There are two ways to deal with defective pixels in DNG.

The first is to map out (interpolate over) the defective pixels before storing the raw data in DNG.

The second is to include a bad pixel fixing opcode in the OpcodeList1 tag."

OK, as each program/customer/etc. seem to have its own ( best ) algorithm to interpolate the bad pixels then OpcodeList1 it is.

But then I've got this: FixBadPixelsConstant and I'm not clear about its usage, especially BayerPhase, is the same BayerPhase that is specified as a global property of the CFA or it is something separate and what ?

Also is there some "industry standard" value for the Constant that will make more likely the established programs to process correctly the image ?

And this Constant value is unique per DNG or can it be more ot them added, i.e. to somehow characterize the "badness" of the pixel, like reduced sensitivity, stuck at 0/max, etc ?

In the end if somebody can share an one-liner on how to add programatically this tag using the SDK I'll be very grateful.

Sorry for my newebie questions and many thanks for your help.

Really painful to build but in the end possible (had to build XMP* libraries separately, I didn't merge the projects) , is there a way to submit patches or is Adobe looking into this issue to re-release the XMP* kit and the DNG_sdk kit updated for the latest versions of Visual Studio ?

It has been 2 years since the last release.

I am currently looking at the DNG spec and the SDK, specifically using the LookTable to encode an HSV conversion that does not necessarily preserve grays.

But as these are HSV tables and the way these tables are setup, there does not seem to be a way to increase saturation of a pixel whose initial saturation is 0 (zero), since the values stored are saturation scales.

For example, below are the input HSV values (using DNG SDK's encoding range) and the value it needs to be transformed to.

Input= Hue: 2.5 Sat: 0 Val: 0.0666667

Output= Hue: 3.96332 Sat: 0.285995 Val: 0.0977313

So, the question I am hoping someone has an answer to is, is this even possible using a DCP file? If yes, any pointers would be appreciated!

Thank you!

Hello everyone,

I've been trying investigate DNG tags in file using dng_validate in DNG sdk v1.4. In 'dng_validate' -d <num> option should show all information i.e. tags in .dng. However some of them are not visible. For example, ProfileToneCurve.

Why I cannot access them? Is there any other ways to access tags?

Thanks.

I am trying to figure out GainMap tags change by ISO in ".dng" file. I came across with this question: https://forums.adobe.com/thread/1410199 and this answer: https://forums.adobe.com/message/6145362#6145362 helped me to understand.

To understand ISO’s effect better, I took 10 pictures and averaged them when ISO was 100. Then I took 10 more of the same scene and averaged them again, but this time ISO was 200. Among the two sets of images, the only change is ISO, i.e. all other parameters are fixed.

Pixel values were doubled in RAW of the average of second set w.r.t the first set, globally, since only the ISO was doubled (as I expected). I also averaged 4 GainMap opcodes separately for the first set of images (ISO100), and for the second set of images (ISO200). Again, when I compared the two sets (ISO100 vs. ISO200), second and third opcodes are exactly the same for the two sets. However, first and fourth GainMap opcodes are different. Can you help me to figure out what is the reason of this difference?

Here I'm sharing the change of GainMap opcodes, which are obtained by point-wise dividing the two matrices which are obtained by the two sets (ISO100 vs. ISO200). Colorbar shows the scale of change. As I mentioned, only first and fourth GainMap opcodes are different, so that I’m putting the two here. The other two are the same with each other.

Image may be NSFW.
Clik here to view.

Thanks in advance for your time.

I'm trying to find Photoshop's image processing operations applied to an image by comparing DNG SDK.

For example, this is the difference between DNG SDK's sRGB result and Photoshop's sRGB result . (SDK's full pipeline is applied to RAW image, and for the Photoshop case RAW image is opened and saved as sRGB .tiff file.)

Image may be NSFW.
Clik here to view.

Then I realized that Photoshop applies noise reduction and sharpening to the images. Then I decreased all these parameters to 0, as much as I can do. Here is what I did.

Image may be NSFW.
Clik here to view.

Then the difference is less. However it is not zero. Here, you can see the difference between the two images.

Image may be NSFW.
Clik here to view.

What could be the other operations that Photoshop applies to the images rather than noise reduction and sharpening?

Thanks in advance for your time.

Hi All, I am new on this forum. I just started to assess the possibilities of DNG (and its SDK) for my photo processing pipeline. Being able to add (semi-) transparent pixels to sets of photos is one of my requirements, and since DNG SDK v1.4 this seems supported. Also, I was wondering how these transparent are pixels would be treated when popular end-user tools such as LightRoom and Adobe Camera Raw are used to further post-process the resulting DNGs.

I searched this forum and skimmed the DNG SDK source code, but effectively I haven't been able to generate results.

Does anyone have pointers here to:

• Sample DNG file(s) with transparent layer ?
• Code snippets for adding/accessing (an existing image as) a layer in an existing DNG file ?

Hello

I want export Bayer data from my DNG file. I am reading DNG SDK, the dng_validate can parse my DNG file ,there are four lossless jpg in the file. But I can't find the rule to make a RAW data file.

ConstPixel can get a buffer, but can't preview the buffer in Photoshop.

Is there any explain about  the order of RGGB in the ConstPixel, or and document specifies how to convert the four planes jpg to a bayer raw data?

Thanks

Hello everyone,

I am trying to build the DNG_SDK Version 1.4 with XCode 6.2. I have downloaded all the third party libraries for xmp and dng sdk. After linking errors with the current xmp sdk 201412, I tried to compile DNG_SDK with XMP SDK Version CS6. The XMP Subproject is found in my DNG_SDK and is build successfully.

For the DNG SDK I still get linker errors and I have no clue why (regardless which architecture I select i386 or x64):

Ld /Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/targets/mac/release/dng_validate normal i386

    cd /Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/projects/mac

    export MACOSX_DEPLOYMENT_TARGET=10.9

    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cl ang++ -arch i386 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOS X10.9.sdk -L/Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/targets/mac/release -F/Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/targets/mac/release -filelist /Users/sboegel/Library/Developer/Xcode/DerivedData/dng_validate-fjjibtptniqsozaqnhhmkybxy udz/Build/Intermediates/dng_validate.build/Default/dng_validate\ release.build/Objects-normal/i386/dng_validate.LinkFileList -mmacosx-version-min=10.9 /Users/sboegel/Library/Developer/Xcode/DerivedData/dng_validate-fjjibtptniqsozaqnhhmkybxy udz/Build/Products/libXMPFilesStatic.a /Users/sboegel/Library/Developer/Xcode/DerivedData/dng_validate-fjjibtptniqsozaqnhhmkybxy udz/Build/Products/libXMPCoreStatic.a -framework Carbon -Xlinker -dependency_info -Xlinker /Users/sboegel/Library/Developer/Xcode/DerivedData/dng_validate-fjjibtptniqsozaqnhhmkybxy udz/Build/Intermediates/dng_validate.build/Default/dng_validate\ release.build/Objects-normal/i386/dng_validate_dependency_info.dat -o /Users/sboegel/Downloads/dng_sdk_1_4/dng_sdk/targets/mac/release/dng_validate

Undefined symbols for architecture i386:

  "TXMPMeta<std::string>::Initialize()", referenced from:

      XMPFiles::Initialize(unsigned int, char const*, char const*) in libXMPFilesStatic.a(XMPFiles.o)

  "TXMPMeta<std::string>::SetProperty(char const*, char const*, std::string const&, unsigned int)", referenced from:

      P2_MetaHandler::SetAudioInfoFromLegacyXML(bool) in libXMPFilesStatic.a(P2_Handler.o)

      P2_MetaHandler::SetVideoFrameInfoFromLegacyXML(XML_Node*, bool) in libXMPFilesStatic.a(P2_Handler.o)

      MP3_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(MP3_Handler.o)

      ReadIDXFile(std::string const&, std::string const&, TXMPMeta<std::string>*, bool&, MD5_CTX*, bool) in libXMPFilesStatic.a(SonyHDV_Handler.o)

      XDCAMEX_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(XDCAMEX_Handler.o)

      XDCAM_Support::GetLegacyMetadata(TXMPMeta<std::string>*, XML_Node*, char const*, bool, std::string&) in libXMPFilesStatic.a(XDCAM_Support.o)

      RIFF::importProperties(RIFF_MetaHandler*) in libXMPFilesStatic.a(RIFF_Support.o)

      ...

  "TXMPMeta<std::string>::SetProperty(char const*, char const*, char const*, unsigned int)", referenced from:

      ID3_Support::ID3v1Tag::read(XMP_IO*, TXMPMeta<std::string>*) in libXMPFilesStatic.a(ID3_Support.o)

      P2_MetaHandler::SetXMPPropertyFromLegacyXML(bool, XML_Node*, char const*, char const*, char const*, bool) in libXMPFilesStatic.a(P2_Handler.o)

      P2_MetaHandler::SetGPSPropertyFromLegacyXML(XML_Node*, bool, char const*, char const*) in libXMPFilesStatic.a(P2_Handler.o)

      P2_MetaHandler::SetAltitudeFromLegacyXML(XML_Node*, bool) in libXMPFilesStatic.a(P2_Handler.o)

      MP3_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(MP3_Handler.o)

      MPEG4_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(MPEG4_Handler.o)

      ReadIDXFile(std::string const&, std::string const&, TXMPMeta<std::string>*, bool&, MD5_CTX*, bool) in libXMPFilesStatic.a(SonyHDV_Handler.o)

      ...

  "TXMPMeta<std::string>::DeleteProperty(char const*, char const*)", referenced from:

      P2_MetaHandler::SetRelationsFromLegacyXML(bool) in libXMPFilesStatic.a(P2_Handler.o)

      P2_MetaHandler::SetDurationFromLegacyXML(bool) in libXMPFilesStatic.a(P2_Handler.o)

      P2_MetaHandler::SetGPSPropertyFromLegacyXML(XML_Node*, bool, char const*, char const*) in libXMPFilesStatic.a(P2_Handler.o)

      P2_MetaHandler::SetAltitudeFromLegacyXML(XML_Node*, bool) in libXMPFilesStatic.a(P2_Handler.o)

      P2_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(P2_Handler.o)

      JPEG_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(JPEG_Handler.o)

      MPEG4_MetaHandler::ProcessXMP() in libXMPFilesStatic.a(MPEG4_Handler.o)

      ...

Any hint is appreciated. Thanks

I found a hint here (Xcode Compile Error Undefined symbols for architecture x86_64:   "TXMPMeta<std::string>::Initialize() that a include statement would be missing, but that is defined in the dng sdk (dng_xmp_sdk.cpp):

#if qMacOS

#ifndef MAC_ENV

#define MAC_ENV 1

#endif

#endif

#if qWinOS

#ifndef WIN_ENV

#define WIN_ENV 1

#endif

#endif

#include <new>

#include <string>

#define TXMP_STRING_TYPE std::string

#define XMP_INCLUDE_XMPFILES qDNGXMPFiles

#define XMP_StaticBuild 1

#include "XMP.incl_cpp"

This cannot be the reason..

Hello everybody,

Is it possible to create DNG from RGB data? I need to implement export of images into DNG format in my application. All I have is image width, height and array of floating-point RGB data.

I'm experimenting with DNG SDK for two days, playing with dng_negative parmeters. I've figured out that I can't use ttFloat type of image — it's not supported, BuildStage2Image thorws an exception. Okay, I've converted source data into uint16 and stored it as ttShort dng_image. But my best result was DNG file with very weird colors.

So what do I need to store my data into DNG? What color matrix and calibration illuminant should I use? What dng_negative parameters could be leaved with default values? Doxygen documentation is very unclear. All examples I've found over network (on this forum, in sources of digiKam project, etc) is about converting some raw image into DNG. I don't need to convert image file, I need to build DNG from scratch.

This is my source, if it can help to undestand what am I doing:

dng_host dngHost;
dngHost.SetSaveLinearDNG(false);

dng_rect imageRect(img.height(), img.width());

AutoPtr<dng_image> dngImage(new dng_simple_image(imageRect, 3, ttShort, dngHost.Allocator()));

// Convert source floating-point image into uint16
QVector<uint16> shortData(img.width() * img.height() * 3);
float *srcData = img.data();
uint16 *dstData = shortData.data();

for (int i = 0; i < img.width() * img.height() * 3; ++i) {    *(dstData++) = static_cast<uint16>(qMin(*(srcData++), 1.0f) * 65535.0f);
}

dng_pixel_buffer buffer;
buffer.fArea = imageRect;
buffer.fPlane = 0;
buffer.fPlanes = 3;
buffer.fRowStep = 3 * img.width();
buffer.fColStep = 3;
buffer.fPlaneStep = 1;
buffer.fPixelType = ttShort;
buffer.fPixelSize = TagTypeSize(ttShort);
buffer.fData = shortData.data();
dngImage->Put(buffer);

AutoPtr<dng_negative> dngNegative(dngHost.Make_dng_negative());

dngNegative->SetDefaultScale(dng_urational(1, 1), dng_urational(1, 1));
dngNegative->SetDefaultCropOrigin(0, 0);
dngNegative->SetDefaultCropSize(img.width(), img.height());
dngNegative->SetActiveArea(imageRect);
dngNegative->SetModelName("Dummy");
dngNegative->SetLocalName("Dummy");
dngNegative->SetOriginalRawFileName(fi.fileName().toUtf8().constData());
dngNegative->SetColorChannels(3);
dngNegative->SetColorKeys(colorKeyRed, colorKeyGreen, colorKeyBlue);
dngNegative->SetWhiteLevel((0x01 << 16) - 1);
dngNegative->SetBlackLevel(0);
dngNegative->SetBaselineExposure(0.0);
dngNegative->SetNoiseReductionApplied(dng_urational(0, 1));
dngNegative->SetBaselineNoise(1.0);
dngNegative->SetBaselineSharpness(1.0);
dngNegative->SetBaseOrientation(dng_orientation::Normal());
dngNegative->SetLinearResponseLimit(1.0);

AutoPtr<dng_camera_profile> prof(new dng_camera_profile);
prof->SetName("dummy");
dng_matrix_3by3 camXYZ(1.0f, 1.0f, 1.0f);
prof->SetColorMatrix1(static_cast<dng_matrix>(camXYZ));
prof->SetCalibrationIlluminant1(lsD65);
prof->SetWasReadFromDNG(true);
dngNegative->AddProfile(prof);

dngNegative->SetStage1Image(dngImage);
dngNegative->BuildStage2Image(dngHost);
dngNegative->BuildStage3Image(dngHost);
dngNegative->SynchronizeMetadata();
dngNegative->RebuildIPTC(true, false);

dng_image_preview thumbnail;
dng_render render(dngHost, *dngNegative);
render.SetFinalSpace(dng_space_sRGB::Get());
render.SetFinalPixelType(ttByte);
render.SetMaximumSize(256);
thumbnail.fImage.Reset(render.Render());

dng_image_writer writer;
dng_file_stream stream(QFile::encodeName(filename).constData(), true);
writer.WriteDNG(dngHost, stream, *dngNegative.Get(), thumbnail, ccUncompressed);

Hey,

I'm currently developing code to write raw camera image data to dng files (I'm doing some processing in between so I'm not using the DNG Converter exe). I have not been able to find any tutorial on the dng sdk, so my code is based partly on the dng_validate code and partly on incomplete code provided by others.

Anyway, my problem is that the dngs are not being written correctly. When I run dng_validate on the created dng, I get the following: "Error: Missing or invalid SamplesPerPixel (SubIFD 1)". However, when I set the SamplesPerPixel tag in SubIFD 1, the change is not reflected in the output of dng_validate when run on the created dng. Similarly, when I set the PhotometricInterpretation tag to CFA (32803), the output of dng_validate still shows it as being set to LinearRaw. My question is are there other SubIFD 1 tags that can have the effect of overwriting the above tags?

Thanks,

aubrey

I scanned a black & white negative. Can I turn it into a positive in PhotoShop?

Hi,

Hopefully someone can help me here.  I'm looking to place either simulated or camera raw output (and here I mean a matrix of 8 or 10 bit values without debayering) into a RAW format so that I can experiment with the debayering, white balance, colour profiles, linearization or LUTs that I need to shift my camera output into a useable image.  I was thinking about using the Adobe DNG format - but I'm struggling to understand where to start with this.

What I'd like to do to start with is simply create a DNG file with, say, a 10x10 image size, 8 bit data with a ramp profile: e.g.

0 20 40 60 80 100 120 140 160 180

0 20 40 60 80 100 120 140 160 180

.

.

.

0 20 40 60 80 100 120 140 160 180

I've been looking at using the dng_validate example and creating a new function:

dng_error_code dng_createsim (constchar *filenameOut)

{

}

After several attempts at this, I've sort of guessed that I need to access the data buffer in negative->Stage1Image(), but I've not succeeded in this.

Thanks for any help.

Simon

Hello all, I loaded the project solution coming with the current source into my Microsoft Visual C++ 2010 IDE and hit BUILD having prepared everything as in the README - I even replaced XMP 5.1.2 with 4.4.2 version in one trial as found here in the forum, but then the error messages got even worst - the build always stucks.

I even put everything dng_sdk and XMP-SDK as siblings (as recommended) in the root folder to avoid too long path errors, but all didn't help...

Side note: Would it be possible to generate a Makefile to be used with another IDE too (something like CodeLite, which imports MS project solution files at least, tried it, set all ENV variables as in vcvars32.bat but then it stucked on the build profiles - VALIDATE_DEBUG etc. - and could not find the .obj files it had generated though, so now I tried with one MSVC++ 2010 IDE instance available for me).

Cheers, hoping for help

Daniel

I have a matrix that will convert my raw camera RGB data to sRGB space, for example when I apply the matrix to the white balance corrected raw data in MATLAB, I get a good looking output image.

I want to create a DNG file from this raw data, but I can't figure out how to generate the ForwardMatrix. For that matter, I'm not even sure whether it is the ForwardMatrix I should be generating or the ColorMatrix (or both?).

To simplify things let's assume the cam_to_sRGB matrix I have is for the A illuminant and the image was captured under that same illuminant. (I can extrapolate later for high color temperature matrix.)

What exact steps must I take to generate the ForwardMatrix1 given the cam_to_sRGB matrix that I have? What should the ColorMatrix1 be in this case? (identity?)

Specific example:

cam_to_sRGB =

What would the ForwardMatrix1 be? What would the ColorMatrix1 be?

I found the DNG spec not helpful. It tells me how to generate an image given already existing DNG matrixes. I need to know how to generate those matrixes in the first place.

Thanks,

Ken

Hello,

Is there a dng specification raw image size limit? I would like to save huge raw images. We changed the kMaxImageSide from 65000 to 300000. I know the dng's which are bigger then 65000 per size can't be opened in other dng readers. But also if you just have a raw image 30000x30000 it can be bigger then 2.147Gb. It's depends on bits per pixels. For now we have images like 7500 x 60000, but soon we want to save raw images like 7500 x 314169 or better 7500 x 628328. That means for the first image we have raw image data like 4.4Gb and for the other 8.8Gb. Of course this is just possible on a 64bit system.

Is there any plan to support huge raw images (64bit dng sdk)? If so when will you release it?

Best regards,

Marius

Hi all,

I would like to do some simple dng raw image manipulation like black image subtraction.

Therfore I though I would get the stage2 image (because working on stage1 is not  correct right?) of both the main-image and the black-image and create an image "main minus black".

Now I want to have this image saved back as dng as if it was the main dng.

just setting the new image as stage2 does not work because writeDNG uses the stage1 image which is set as fRawImage in buildStage2Image.

Setting as stage1 does not work too because it is already a linearized image.

How can I reverse the linearization?

Or are there other way to do this correctly?

Thank you all and best regards

Andreas

Dear all,

I’m trying to install DNG-SDK (1.3) but with no success, I have tried to compile with both XMP Toolkit SDK CC-2013.06 and XMP Toolkit SDK 5.1.2 but with the same error message:

error C2039: 'AppendProperties' : is not a member of 'TXMPUtils<tStringObj>'

Unfortunately I was not able to find XMP Toolkit SDK 4.4.2 that should solve the problem. The compiler is VS 2010 64bit.

Please can you give me some tips on how solve this issue?

Regards,

Hal

Hi,

I would like to embed only the mosaiced image into the DNG. Is this possible? I have tried to comment BuildStage2Image and BuildStage3Image, but with no success. So any clue will be appreciated.

Thank you.