mobian, pinephone camera, and megapixels
#21
Hello Cal,

(04-28-2021, 04:16 PM)calinb Wrote: I guess it still hasn't rolled out in a Mobian update / upgrade yet?

Unfortunately not, the megapixel package in Mobian is still 0.15, and the fix is in the 0.16 release. Because of the bullseye upcoming release, matters are in some limbo.

(04-28-2021, 04:16 PM)calinb Wrote: I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

You did, and I applied your fix a couple weeks ago. Thanks for that!

- Norm
  Reply
#22
(04-28-2021, 04:16 PM)calinb Wrote: Zebulon,

I found this bug (non-POSIX syntax in the /usr/share/megapixels/postprocess.sh). Then I patched my script and reported it to Martijnj. He promptly squashed the tiny (but deadly bug) in his source. I guess it still hasn't rolled out in a Mobian update / upgrade yet?

All you need to do is change == to = to make postprocess.sh work, like this:

Code:
- if [ "$CONVERT" == "convert" ]; then

+ if [ "$CONVERT" = "convert" ]; then

As documented here:
https://git.sr.ht/~martijnbraam/megapixe...process.sh

I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

I get .dng and .jpg now so I need to double check that you have the same helper aps that I'm using, which I'll do later today.

-Cal

Must have missed that, I'll make the change and see how it works, thanks!
  Reply
#23
(04-28-2021, 04:29 PM)NormandC Wrote: Hello Cal,

(04-28-2021, 04:16 PM)calinb Wrote: I guess it still hasn't rolled out in a Mobian update / upgrade yet?

Unfortunately not, the megapixel package in Mobian is still 0.15, and the fix is in the 0.16 release. Because of the bullseye upcoming release, matters are in some limbo.

Thanks for the update, Norm!


Hopefully, Zebulon will be able to similarly patch his postprocess.sh file and get his .jpg files back too.

-Cal

(04-28-2021, 04:40 PM)Zebulon Walton Wrote: Must have missed that, I'll make the change and see how it works, thanks!

I missed your post while I was replying to Norm.

Please let us know how it goes.

-Cal
  Reply
#24
(04-28-2021, 04:44 PM)calinb Wrote: Thanks for the update, Norm!


Hopefully, Zebulon will be able to similarly patch his postprocess.sh file and get his .jpg files back too.

-Cal



I missed your post while I was replying to Norm.

Please let us know how it goes.

-Cal


Success, the jpg is now being produced. (Takes a little while for processing, nature of the beast.) The dng version of the photo is also being saved. Is there a shell script mod to have those automagically deleted?
  Reply
#25
(04-28-2021, 05:21 PM)Zebulon Walto Wrote: Success, the jpg is now being produced. (Takes a little while for processing, nature of the beast.) The dng version of the photo is also being saved. Is there a shell script mod to have those automagically deleted?

I can't look at the script right now, but IIRC, I there might be a problem with the simple postprocess.sh script in targeting just the latest .dng files for deletion. However, I think I quickly came to the conclusion that it would be possible and simple to just "rm ~/Pictures/*.dng" to nuke all of them after each invocation of the script--or maybe mv them to Trash.
  Reply
#26
(04-28-2021, 05:50 PM)calinb Wrote: I can't look at the script right now, but IIRC, I there might be a problem with the simple postprocess.sh script in targeting just the latest .dng files for deletion. However, I think I quickly came to the conclusion that it would be possible and simple to just "rm ~/Pictures/*.dng" to nuke all of them after each invocation of the script--or maybe mv them to Trash.

Of course, simple solution!
  Reply
#27
I added:
Code:
rm "$TARGET_NAME.dng"
as the last line of postprocess.sh.

----------------------------------------

Thanks for all the debugging this was a super helpful thread!

Cheers!
  Reply
#28
(04-28-2021, 04:29 PM)NormandC Wrote: ...

(04-28-2021, 04:16 PM)calinb Wrote: I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

You did, and I applied your fix a couple weeks ago. Thanks for that!

- Norm

+1 thanks to @calinb 

As identified the "silver bullet" in megapixels is the post-processing script.
  • The basic camera output is a raw .dng format
  • It is converted to .TIFF if dcraw is installed
  • The .TIFF is converted to .jpg if imagemagick is installed
  • The .jpg file gets some exif data if libimage-exiftool-perl is installed (in Mobian - Arch has a different name for it)
I don't have a note on what any default Mobian config has but for sure I always ensure I have dcraw, imagemagick and libimage-exiftool-perl packages installed each time I rebuild my Mobian system.

The other nugget for postprocess.sh is if you put a version in ~/.config/megapixels/ then that is used in preference to the default in /usr/share/megapixels/ . Which will survive any upgrades and can do stuff the way you prefer. So my version has 3 mods:
  1. the @calinb / v0.16 patch
  2. I have commented out the initial copy of the .dng file to /Pictures as I dont want to preserve any raw images
  3. I have added an exiftool line (and suggested it to Martijn) to set the DateTimeOriginal tag. (Because the package I use for my media library, fotoxx, can sort on that tag!)
My version
Code:
$ cat .config/megapixels/postprocess.sh
#!/bin/sh

# The post-processing script gets called after taking a burst of
# pictures into a temporary directory. The first argument is the
# directory containing the raw files in the burst. The contents
# are 1.dng, 2.dng.... up to the number of photos in the burst.
#
# The second argument is the filename for the final photo without
# the extension, like "/home/user/Pictures/IMG202104031234"
#
# The post-processing script is responsible for cleaning up
# temporary directory for the burst.

if [ "$#" -ne 2 ]; then
    echo "Usage: $0 [burst-dir] [target-name]"
    exit 2
fi

BURST_DIR="$1"
TARGET_NAME="$2"

MAIN_PICTURE="$BURST_DIR"/1

# Copy the first frame of the burst as the raw photo
# cp "$BURST_DIR"/1.dng "$TARGET_NAME.dng"

# Create a .jpg if raw processing tools are installed
DCRAW=""
TIFF_EXT="dng.tiff"
if command -v "dcraw_emu" > /dev/null
then
    DCRAW=dcraw_emu
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif [ -x "/usr/lib/libraw/dcraw_emu" ]; then
    DCRAW=/usr/lib/libraw/dcraw_emu
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif command -v "dcraw" > /dev/null
then
    DCRAW=dcraw
    TIFF_EXT="tiff"
    set --
fi

CONVERT=""
if command -v "convert" > /dev/null
then
    CONVERT="convert"
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif command -v "gm" > /dev/null
then
    CONVERT="gm"
fi


if [ -n "$DCRAW" ]; then
    # +M        use embedded color matrix
    # -H 4        Recover highlights by rebuilding them
    # -o 1        Output in sRGB colorspace
    # -q 3        Debayer with AHD algorithm
    # -T        Output TIFF
    $DCRAW +M -H 4 -o 1 -q 3 -T "$@" "$MAIN_PICTURE.dng"

    # If imagemagick is available, convert the tiff to jpeg and apply slight sharpening
    if [ -n "$CONVERT" ];
    then
        if [ "$CONVERT" = "convert" ]; then
            convert "$MAIN_PICTURE.$TIFF_EXT" -sharpen 0x1.0 "$TARGET_NAME.jpg"
        else
            gm convert "$MAIN_PICTURE.$TIFF_EXT" -sharpen 0x1.0 "$TARGET_NAME.jpg"
        fi

        # If exiftool is installed copy the exif data over from the tiff to the jpeg
        # since imagemagick is stupid
        if command -v exiftool > /dev/null
        then
            exiftool -tagsFromfile "$MAIN_PICTURE.$TIFF_EXT" \
                 -software="Megapixels" \
                 -overwrite_original "$TARGET_NAME.jpg"
            exiftool -tagsFromFile @ "-ModifyDate>DateTimeOriginal" -overwrite_original "$TARGET_NAME.jpg"
        fi

        echo "$TARGET_NAME.jpg"
    else
        cp "$MAIN_PICTURE.$TIFF_EXT" "$TARGET_NAME.tiff"

        echo "$TARGET_NAME.tiff"
    fi
fi

# Clean up the temp dir containing the burst
rm -rf "$BURST_DIR"
  • ROCKPro64 v2.1 2GB, 16Gb eMMC for rootfs, SX8200Pro 512GB NVMe for /home, HDMI video & sound, Bluetooth keyboard & mouse. Arch (6.2 kernel, Openbox desktop) for general purpose daily PC.
  • PinePhone Pro Explorer Edition, daily driver, rk2aw & U-boot on SPI, Arch/SXMO & Arch/phosh on eMMC
  • PinePhone BraveHeart now v1.2b 3/32Gb, Tow-boot with Arch/SXMO on eMMC
  Reply
#29
Quote:@dukla2000:
The other nugget for postprocess.sh is if you put a version in ~/.config/megapixels/ then that is used in preference to the default in /usr/share/megapixels/ . Which will survive any upgrades and can do stuff the way you prefer. So my version has 3 mods:

    the @calinb / v0.16 patch
    I have commented out the initial copy of the .dng file to /Pictures as I dont want to preserve any raw images
    I have added an exiftool line (and suggested it to Martijn) to set the DateTimeOriginal tag. (Because the package I use for my media library, fotoxx, can sort on that tag!)

Neat, I copied the modified script to ~/.config/megapixels to play around with. Full processing of a photo with everything enabled takes about 45-50 seconds. By eliminating the exiftool step (I don't care about metadata) that gets cut down to about 30 seconds. Now I'm admittedly not familiar with the image processing tools, but if I understand correctly the script uses tiff format as an intermediate step to get to jpg (dng-->tiff-->jpg). Tiff format as far as I know is not widely used for photos and seems kind of a waste, would it be feasible to convert directly from dng to jpg to save processing time?

Another thing I'm noticing is the jpegs being produced now are much larger than when I last used the camera months ago, my photos from back then are under a megabyte, new ones are 3-4 megabytes. Unless I reduced the size of the old photos and forgot (which is possible) something changed.
  Reply
#30
(04-29-2021, 07:09 AM)Zebulon Walton Wrote: ... Now I'm admittedly not familiar with the image processing tools, but if I understand correctly the script uses tiff format as an intermediate step to get to jpg (dng-->tiff-->jpg). Tiff format as far as I know is not widely used for photos and seems kind of a waste, would it be feasible to convert directly from dng to jpg to save processing time?
...

Yes but ...!
Technically imagemagick could do a direct conversion .dng -> .jpg. The "but" is that dcraw is doing more than just a format conversion, it is using some magic options (chosen by Martijn) to try make the picture look better as per the comments in the postprocess script. dcraw cannot output .jpg though. But again the megapixels choice to use imagemagick to do the conversion to .jpg also allows the chance to get imagemagick to try sharpen things.

Or you could cut out most of the (megapixels) postprocessing and use a package like RawTherapee to process your images.

The fundamental limitation though is the inexpensive sensors chosen by Pine64 are pretty much point and click cheapos and so no amount of postprocessing can turn me into David Bailey! Then again, even with slightly expensive kit I struggle to get stuff in focus so having the PinePhone for quick snaps is fine by me.
  • ROCKPro64 v2.1 2GB, 16Gb eMMC for rootfs, SX8200Pro 512GB NVMe for /home, HDMI video & sound, Bluetooth keyboard & mouse. Arch (6.2 kernel, Openbox desktop) for general purpose daily PC.
  • PinePhone Pro Explorer Edition, daily driver, rk2aw & U-boot on SPI, Arch/SXMO & Arch/phosh on eMMC
  • PinePhone BraveHeart now v1.2b 3/32Gb, Tow-boot with Arch/SXMO on eMMC
  Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  bookworm vs trixie discussion for mobian in pinephone regular. zetabeta 39 7,617 10-15-2024, 01:12 PM
Last Post: anonymous
  The program doesn`t detect the camera blag0rag 6 1,387 09-17-2024, 03:45 PM
Last Post: Kevin Kofler
  atinout binaries for mobian/debian? NeutralGrey 3 1,160 08-07-2024, 04:27 AM
Last Post: baptx
  Pinephone seems tobe using 4G data instead of Wifi data grump_fiddle_reinstall 1 711 06-25-2024, 10:38 AM
Last Post: mikehenson
  Mobian-Kicksecure? 3460p 0 738 05-26-2024, 02:09 PM
Last Post: 3460p
  Mobian repository status henrythemouse 16 11,726 04-10-2024, 10:02 AM
Last Post: diederik
  cant verify mobian image at website gnugpg penguins_rule 0 800 03-18-2024, 08:54 PM
Last Post: penguins_rule
  unofficial kali linux for pinephone help needed pinesmell 5 4,895 01-08-2024, 02:50 PM
Last Post: ERPlegend
  mobian installed to eMMC - how to install tow-boot grump_fiddle_reinstall 6 3,713 11-22-2023, 11:46 AM
Last Post: aLoop100o
  What actions needed to keep on mobian testing user641 3 2,471 09-05-2023, 06:44 AM
Last Post: Zebulon Walton

Forum Jump:


Users browsing this thread: 1 Guest(s)