mobian, pinephone camera, and megapixels
#21
(04-28-2021, 02:39 PM)Zebulon Walton Wrote: I just used Megapixels for the first time months while and I'm finding it's producing dng files instead of the jpg format it was previously saving. (A couple of pictures I took and saved back in January with Megapixels are in jpg format.)
<snip>

Zebulon,

I found this bug (non-POSIX syntax in the /usr/share/megapixels/postprocess.sh). Then I patched my script and reported it to Martijnj. He promptly squashed the tiny (but deadly bug) in his source. I guess it still hasn't rolled out in a Mobian update / upgrade yet?

All you need to do is change == to = to make postprocess.sh work, like this:

Code:
- if [ "$CONVERT" == "convert" ]; then

+ if [ "$CONVERT" = "convert" ]; then

As documented here:
https://git.sr.ht/~martijnbraam/megapixe...process.sh

I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

I get .dng and .jpg now so I need to double check that you have the same helper aps that I'm using, which I'll do later today.

-Cal
  Reply
#22
Hello Cal,

(04-28-2021, 04:16 PM)calinb Wrote: I guess it still hasn't rolled out in a Mobian update / upgrade yet?

Unfortunately not, the megapixel package in Mobian is still 0.15, and the fix is in the 0.16 release. Because of the bullseye upcoming release, matters are in some limbo.

(04-28-2021, 04:16 PM)calinb Wrote: I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

You did, and I applied your fix a couple weeks ago. Thanks for that!

- Norm
  Reply
#23
(04-28-2021, 04:16 PM)calinb Wrote: Zebulon,

I found this bug (non-POSIX syntax in the /usr/share/megapixels/postprocess.sh). Then I patched my script and reported it to Martijnj. He promptly squashed the tiny (but deadly bug) in his source. I guess it still hasn't rolled out in a Mobian update / upgrade yet?

All you need to do is change == to = to make postprocess.sh work, like this:

Code:
- if [ "$CONVERT" == "convert" ]; then

+ if [ "$CONVERT" = "convert" ]; then

As documented here:
https://git.sr.ht/~martijnbraam/megapixe...process.sh

I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

I get .dng and .jpg now so I need to double check that you have the same helper aps that I'm using, which I'll do later today.

-Cal

Must have missed that, I'll make the change and see how it works, thanks!
  Reply
#24
(04-28-2021, 04:29 PM)NormandC Wrote: Hello Cal,

(04-28-2021, 04:16 PM)calinb Wrote: I guess it still hasn't rolled out in a Mobian update / upgrade yet?

Unfortunately not, the megapixel package in Mobian is still 0.15, and the fix is in the 0.16 release. Because of the bullseye upcoming release, matters are in some limbo.

Thanks for the update, Norm!


Hopefully, Zebulon will be able to similarly patch his postprocess.sh file and get his .jpg files back too.

-Cal

(04-28-2021, 04:40 PM)Zebulon Walton Wrote: Must have missed that, I'll make the change and see how it works, thanks!

I missed your post while I was replying to Norm.

Please let us know how it goes.

-Cal
  Reply
#25
(04-28-2021, 04:44 PM)calinb Wrote: Thanks for the update, Norm!


Hopefully, Zebulon will be able to similarly patch his postprocess.sh file and get his .jpg files back too.

-Cal



I missed your post while I was replying to Norm.

Please let us know how it goes.

-Cal


Success, the jpg is now being produced. (Takes a little while for processing, nature of the beast.) The dng version of the photo is also being saved. Is there a shell script mod to have those automagically deleted?
  Reply
#26
(04-28-2021, 05:21 PM)Zebulon Walto Wrote: Success, the jpg is now being produced. (Takes a little while for processing, nature of the beast.) The dng version of the photo is also being saved. Is there a shell script mod to have those automagically deleted?

I can't look at the script right now, but IIRC, I there might be a problem with the simple postprocess.sh script in targeting just the latest .dng files for deletion. However, I think I quickly came to the conclusion that it would be possible and simple to just "rm ~/Pictures/*.dng" to nuke all of them after each invocation of the script--or maybe mv them to Trash.
  Reply
#27
(04-28-2021, 05:50 PM)calinb Wrote: I can't look at the script right now, but IIRC, I there might be a problem with the simple postprocess.sh script in targeting just the latest .dng files for deletion. However, I think I quickly came to the conclusion that it would be possible and simple to just "rm ~/Pictures/*.dng" to nuke all of them after each invocation of the script--or maybe mv them to Trash.

Of course, simple solution!
  Reply
#28
I added:
Code:
rm "$TARGET_NAME.dng"
as the last line of postprocess.sh.

----------------------------------------

Thanks for all the debugging this was a super helpful thread!

Cheers!
  Reply
#29
(04-28-2021, 04:29 PM)NormandC Wrote: ...

(04-28-2021, 04:16 PM)calinb Wrote: I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).

You did, and I applied your fix a couple weeks ago. Thanks for that!

- Norm

+1 thanks to @calinb 

As identified the "silver bullet" in megapixels is the post-processing script.
  • The basic camera output is a raw .dng format
  • It is converted to .TIFF if dcraw is installed
  • The .TIFF is converted to .jpg if imagemagick is installed
  • The .jpg file gets some exif data if libimage-exiftool-perl is installed (in Mobian - Arch has a different name for it)
I don't have a note on what any default Mobian config has but for sure I always ensure I have dcraw, imagemagick and libimage-exiftool-perl packages installed each time I rebuild my Mobian system.

The other nugget for postprocess.sh is if you put a version in ~/.config/megapixels/ then that is used in preference to the default in /usr/share/megapixels/ . Which will survive any upgrades and can do stuff the way you prefer. So my version has 3 mods:
  1. the @calinb / v0.16 patch
  2. I have commented out the initial copy of the .dng file to /Pictures as I dont want to preserve any raw images
  3. I have added an exiftool line (and suggested it to Martijn) to set the DateTimeOriginal tag. (Because the package I use for my media library, fotoxx, can sort on that tag!)
My version
Code:
$ cat .config/megapixels/postprocess.sh
#!/bin/sh

# The post-processing script gets called after taking a burst of
# pictures into a temporary directory. The first argument is the
# directory containing the raw files in the burst. The contents
# are 1.dng, 2.dng.... up to the number of photos in the burst.
#
# The second argument is the filename for the final photo without
# the extension, like "/home/user/Pictures/IMG202104031234"
#
# The post-processing script is responsible for cleaning up
# temporary directory for the burst.

if [ "$#" -ne 2 ]; then
    echo "Usage: $0 [burst-dir] [target-name]"
    exit 2
fi

BURST_DIR="$1"
TARGET_NAME="$2"

MAIN_PICTURE="$BURST_DIR"/1

# Copy the first frame of the burst as the raw photo
# cp "$BURST_DIR"/1.dng "$TARGET_NAME.dng"

# Create a .jpg if raw processing tools are installed
DCRAW=""
TIFF_EXT="dng.tiff"
if command -v "dcraw_emu" > /dev/null
then
    DCRAW=dcraw_emu
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif [ -x "/usr/lib/libraw/dcraw_emu" ]; then
    DCRAW=/usr/lib/libraw/dcraw_emu
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif command -v "dcraw" > /dev/null
then
    DCRAW=dcraw
    TIFF_EXT="tiff"
    set --
fi

CONVERT=""
if command -v "convert" > /dev/null
then
    CONVERT="convert"
    # -fbdd 1    Raw denoising with FBDD
    set -- -fbdd 1
elif command -v "gm" > /dev/null
then
    CONVERT="gm"
fi


if [ -n "$DCRAW" ]; then
    # +M        use embedded color matrix
    # -H 4        Recover highlights by rebuilding them
    # -o 1        Output in sRGB colorspace
    # -q 3        Debayer with AHD algorithm
    # -T        Output TIFF
    $DCRAW +M -H 4 -o 1 -q 3 -T "$@" "$MAIN_PICTURE.dng"

    # If imagemagick is available, convert the tiff to jpeg and apply slight sharpening
    if [ -n "$CONVERT" ];
    then
        if [ "$CONVERT" = "convert" ]; then
            convert "$MAIN_PICTURE.$TIFF_EXT" -sharpen 0x1.0 "$TARGET_NAME.jpg"
        else
            gm convert "$MAIN_PICTURE.$TIFF_EXT" -sharpen 0x1.0 "$TARGET_NAME.jpg"
        fi

        # If exiftool is installed copy the exif data over from the tiff to the jpeg
        # since imagemagick is stupid
        if command -v exiftool > /dev/null
        then
            exiftool -tagsFromfile "$MAIN_PICTURE.$TIFF_EXT" \
                 -software="Megapixels" \
                 -overwrite_original "$TARGET_NAME.jpg"
            exiftool -tagsFromFile @ "-ModifyDate>DateTimeOriginal" -overwrite_original "$TARGET_NAME.jpg"
        fi

        echo "$TARGET_NAME.jpg"
    else
        cp "$MAIN_PICTURE.$TIFF_EXT" "$TARGET_NAME.tiff"

        echo "$TARGET_NAME.tiff"
    fi
fi

# Clean up the temp dir containing the burst
rm -rf "$BURST_DIR"
  • ROCKPro64 v2.1 2GB, 16Gb eMMC for rootfs, SX8200Pro 512GB NVMe for /home, HDMI video & sound, Bluetooth keyboard & mouse. Started Bionic minimal - now "hirsute", Openbox desktop for general purpose daily PC.
  • PinePhone BraveHeart now v1.2b 3/32Gb daily driver, dual boot via p-boot with Mobian/f2fs/Phosh on eMMC, Arch/ext4/Phosh on SDcard
  • PinePhone v1.2a 2G/16Gb that needs USB board replaced
  Reply
#30
Quote:@dukla2000:
The other nugget for postprocess.sh is if you put a version in ~/.config/megapixels/ then that is used in preference to the default in /usr/share/megapixels/ . Which will survive any upgrades and can do stuff the way you prefer. So my version has 3 mods:

    the @calinb / v0.16 patch
    I have commented out the initial copy of the .dng file to /Pictures as I dont want to preserve any raw images
    I have added an exiftool line (and suggested it to Martijn) to set the DateTimeOriginal tag. (Because the package I use for my media library, fotoxx, can sort on that tag!)

Neat, I copied the modified script to ~/.config/megapixels to play around with. Full processing of a photo with everything enabled takes about 45-50 seconds. By eliminating the exiftool step (I don't care about metadata) that gets cut down to about 30 seconds. Now I'm admittedly not familiar with the image processing tools, but if I understand correctly the script uses tiff format as an intermediate step to get to jpg (dng-->tiff-->jpg). Tiff format as far as I know is not widely used for photos and seems kind of a waste, would it be feasible to convert directly from dng to jpg to save processing time?

Another thing I'm noticing is the jpegs being produced now are much larger than when I last used the camera months ago, my photos from back then are under a megabyte, new ones are 3-4 megabytes. Unless I reduced the size of the old photos and forgot (which is possible) something changed.
  Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Axolotl on PinePhone / Mobian arno_nuehm 94 22,659 06-23-2021, 08:13 AM
Last Post: dante404
  Pinephone in Canada using Shaw mobile provider RRman 0 77 06-19-2021, 06:35 PM
Last Post: RRman
  Reboot Modem on Pinephone in Mobian eggmatters 5 560 06-17-2021, 02:08 PM
Last Post: eggmatters
  Mobian not updating via Software app FOSSagent0 2 256 06-07-2021, 09:22 PM
Last Post: Zebulon Walton
  mobile data on mobian not working synaption 1 192 06-02-2021, 03:16 PM
Last Post: Aberts10
  How do you copy and paste into mobian terminal? Do you ssh instead? temp0rary 6 342 06-01-2021, 12:16 PM
Last Post: temp0rary
  Turn by Turn alternative GPS app in Mobian livanti21 27 12,561 05-30-2021, 03:17 AM
Last Post: john_doe
  Pinephone ”reboot” or logging out by itself Wahtone 0 154 05-28-2021, 09:32 AM
Last Post: Wahtone
  Can't update OS Mobian Phosh Gon 5 532 05-23-2021, 10:52 PM
Last Post: pajux
  Mobian: 2 week as main phone steves 10 1,575 05-18-2021, 11:55 AM
Last Post: vortex

Forum Jump:


Users browsing this thread: 1 Guest(s)