mobian, pinephone camera, and megapixels

[NormandC]

Hello Cal,

I guess it still hasn't rolled out in a Mobian update / upgrade yet?

Unfortunately not, the megapixel package in Mobian is still 0.15, and the fix is in the 0.16 release. Because of the bullseye upcoming release, matters are in some limbo.

I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

You did, and I applied your fix a couple weeks ago. Thanks for that!

- Norm

[Zebulon Walton]

Zebulon,

I found this bug (non-POSIX syntax in the /usr/share/megapixels/postprocess.sh). Then I patched my script and reported it to Martijnj. He promptly squashed the tiny (but deadly bug) in his source. I guess it still hasn't rolled out in a Mobian update / upgrade yet?

All you need to do is change == to = to make postprocess.sh work, like this:

Code:

- if [ "$CONVERT" == "convert" ]; then

+ if [ "$CONVERT" = "convert" ]; then

As documented here:
https://git.sr.ht/~martijnbraam/megapixe...process.sh

I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

I get .dng and .jpg now so I need to double check that you have the same helper aps that I'm using, which I'll do later today.

-Cal

Must have missed that, I'll make the change and see how it works, thanks!

[calinb]

Hello Cal,

I guess it still hasn't rolled out in a Mobian update / upgrade yet?

Unfortunately not, the megapixel package in Mobian is still 0.15, and the fix is in the 0.16 release. Because of the bullseye upcoming release, matters are in some limbo.

Thanks for the update, Norm!


Hopefully, Zebulon will be able to similarly patch his postprocess.sh file and get his .jpg files back too.

-Cal

---

Must have missed that, I'll make the change and see how it works, thanks!

I missed your post while I was replying to Norm.

Please let us know how it goes.

-Cal

[Zebulon Walton]

Thanks for the update, Norm!


Hopefully, Zebulon will be able to similarly patch his postprocess.sh file and get his .jpg files back too.

-Cal

---

I missed your post while I was replying to Norm.

Please let us know how it goes.

-Cal

Success, the jpg is now being produced. (Takes a little while for processing, nature of the beast.) The dng version of the photo is also being saved. Is there a shell script mod to have those automagically deleted?

[calinb]

Success, the jpg is now being produced. (Takes a little while for processing, nature of the beast.) The dng version of the photo is also being saved. Is there a shell script mod to have those automagically deleted?

I can't look at the script right now, but IIRC, I there might be a problem with the simple postprocess.sh script in targeting just the latest .dng files for deletion. However, I think I quickly came to the conclusion that it would be possible and simple to just "rm ~/Pictures/*.dng" to nuke all of them after each invocation of the script--or maybe mv them to Trash.

[Zebulon Walton]

I can't look at the script right now, but IIRC, I there might be a problem with the simple postprocess.sh script in targeting just the latest .dng files for deletion. However, I think I quickly came to the conclusion that it would be possible and simple to just "rm ~/Pictures/*.dng" to nuke all of them after each invocation of the script--or maybe mv them to Trash.

Of course, simple solution!

[MtnSk8]

I added:

Code:

rm "$TARGET_NAME.dng"

as the last line of postprocess.sh.

----------------------------------------

Thanks for all the debugging this was a super helpful thread!

Cheers!

[dukla2000]

...

I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

You did, and I applied your fix a couple weeks ago. Thanks for that!

- Norm

+1 thanks to @calinb 

As identified the "silver bullet" in megapixels is the post-processing script.

• The basic camera output is a raw .dng format
  
• It is converted to .TIFF if dcraw is installed
  
• The .TIFF is converted to .jpg if imagemagick is installed
  
• The .jpg file gets some exif data if libimage-exiftool-perl is installed (in Mobian - Arch has a different name for it)
  

I don't have a note on what any default Mobian config has but for sure I always ensure I have dcraw, imagemagick and libimage-exiftool-perl packages installed each time I rebuild my Mobian system.

The other nugget for postprocess.sh is if you put a version in ~/.config/megapixels/ then that is used in preference to the default in /usr/share/megapixels/ . Which will survive any upgrades and can do stuff the way you prefer. So my version has 3 mods:

1. the @calinb / v0.16 patch
   
2. I have commented out the initial copy of the .dng file to /Pictures as I dont want to preserve any raw images
   
3. I have added an exiftool line (and suggested it to Martijn) to set the DateTimeOriginal tag. (Because the package I use for my media library, fotoxx, can sort on that tag!)
   

My version

Code:

$ cat .config/megapixels/postprocess.sh
#!/bin/sh

# The post-processing script gets called after taking a burst of
# pictures into a temporary directory. The first argument is the
# directory containing the raw files in the burst. The contents
# are 1.dng, 2.dng.... up to the number of photos in the burst.
#
# The second argument is the filename for the final photo without
# the extension, like "/home/user/Pictures/IMG202104031234"
#
# The post-processing script is responsible for cleaning up
# temporary directory for the burst.

if [ "$#" -ne 2 ]; then
    echo "Usage: $0 [burst-dir] [target-name]"
    exit 2
fi

BURST_DIR="$1"
TARGET_NAME="$2"

MAIN_PICTURE="$BURST_DIR"/1

# Copy the first frame of the burst as the raw photo
# cp "$BURST_DIR"/1.dng "$TARGET_NAME.dng"

# Create a .jpg if raw processing tools are installed
DCRAW=""
TIFF_EXT="dng.tiff"
if command -v "dcraw_emu" > /dev/null
then
    DCRAW=dcraw_emu
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif [ -x "/usr/lib/libraw/dcraw_emu" ]; then
    DCRAW=/usr/lib/libraw/dcraw_emu
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif command -v "dcraw" > /dev/null
then
    DCRAW=dcraw
    TIFF_EXT="tiff"
    set --
fi

CONVERT=""
if command -v "convert" > /dev/null
then
    CONVERT="convert"
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif command -v "gm" > /dev/null
then
    CONVERT="gm"
fi


if [ -n "$DCRAW" ]; then
    # +M        use embedded color matrix
    # -H 4        Recover highlights by rebuilding them
    # -o 1        Output in sRGB colorspace
    # -q 3        Debayer with AHD algorithm
    # -T        Output TIFF
    $DCRAW +M -H 4 -o 1 -q 3 -T "$@" "$MAIN_PICTURE.dng"

    # If imagemagick is available, convert the tiff to jpeg and apply slight sharpening
    if [ -n "$CONVERT" ];
    then
        if [ "$CONVERT" = "convert" ]; then
            convert "$MAIN_PICTURE.$TIFF_EXT" -sharpen 0x1.0 "$TARGET_NAME.jpg"
        else
            gm convert "$MAIN_PICTURE.$TIFF_EXT" -sharpen 0x1.0 "$TARGET_NAME.jpg"
        fi

        # If exiftool is installed copy the exif data over from the tiff to the jpeg
        # since imagemagick is stupid
        if command -v exiftool > /dev/null
        then
            exiftool -tagsFromfile "$MAIN_PICTURE.$TIFF_EXT" \
                 -software="Megapixels" \
                 -overwrite_original "$TARGET_NAME.jpg"
            exiftool -tagsFromFile @ "-ModifyDate>DateTimeOriginal" -overwrite_original "$TARGET_NAME.jpg"
        fi

        echo "$TARGET_NAME.jpg"
    else
        cp "$MAIN_PICTURE.$TIFF_EXT" "$TARGET_NAME.tiff"

        echo "$TARGET_NAME.tiff"
    fi
fi

# Clean up the temp dir containing the burst
rm -rf "$BURST_DIR"

[Zebulon Walton]

@dukla2000:
The other nugget for postprocess.sh is if you put a version in ~/.config/megapixels/ then that is used in preference to the default in /usr/share/megapixels/ . Which will survive any upgrades and can do stuff the way you prefer. So my version has 3 mods:

    the @calinb / v0.16 patch
    I have commented out the initial copy of the .dng file to /Pictures as I dont want to preserve any raw images
    I have added an exiftool line (and suggested it to Martijn) to set the DateTimeOriginal tag. (Because the package I use for my media library, fotoxx, can sort on that tag!)

Neat, I copied the modified script to ~/.config/megapixels to play around with. Full processing of a photo with everything enabled takes about 45-50 seconds. By eliminating the exiftool step (I don't care about metadata) that gets cut down to about 30 seconds. Now I'm admittedly not familiar with the image processing tools, but if I understand correctly the script uses tiff format as an intermediate step to get to jpg (dng-->tiff-->jpg). Tiff format as far as I know is not widely used for photos and seems kind of a waste, would it be feasible to convert directly from dng to jpg to save processing time?

Another thing I'm noticing is the jpegs being produced now are much larger than when I last used the camera months ago, my photos from back then are under a megabyte, new ones are 3-4 megabytes. Unless I reduced the size of the old photos and forgot (which is possible) something changed.

[dukla2000]

... Now I'm admittedly not familiar with the image processing tools, but if I understand correctly the script uses tiff format as an intermediate step to get to jpg (dng-->tiff-->jpg). Tiff format as far as I know is not widely used for photos and seems kind of a waste, would it be feasible to convert directly from dng to jpg to save processing time?
...

Yes but ...!
Technically imagemagick could do a direct conversion .dng -> .jpg. The "but" is that dcraw is doing more than just a format conversion, it is using some magic options (chosen by Martijn) to try make the picture look better as per the comments in the postprocess script. dcraw cannot output .jpg though. But again the megapixels choice to use imagemagick to do the conversion to .jpg also allows the chance to get imagemagick to try sharpen things.

Or you could cut out most of the (megapixels) postprocessing and use a package like RawTherapee to process your images.

The fundamental limitation though is the inexpensive sensors chosen by Pine64 are pretty much point and click cheapos and so no amount of postprocessing can turn me into David Bailey! Then again, even with slightly expensive kit I struggle to get stuff in focus so having the PinePhone for quick snaps is fine by me.