Hello Cal,
(04-28-2021, 04:16 PM)calinb Wrote: I guess it still hasn't rolled out in a Mobian update / upgrade yet?
Unfortunately not, the megapixel package in Mobian is still 0.15, and the fix is in the 0.16 release. Because of the bullseye upcoming release, matters are in some limbo.
(04-28-2021, 04:16 PM)calinb Wrote: I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).
You did, and I applied your fix a couple weeks ago. Thanks for that!
- Norm
04-28-2021, 04:40 PM
(This post was last modified: 04-28-2021, 04:41 PM by Zebulon Walton.)
(04-28-2021, 04:16 PM)calinb Wrote: Zebulon,
I found this bug (non-POSIX syntax in the /usr/share/megapixels/postprocess.sh). Then I patched my script and reported it to Martijnj. He promptly squashed the tiny (but deadly bug) in his source. I guess it still hasn't rolled out in a Mobian update / upgrade yet?
All you need to do is change == to = to make postprocess.sh work, like this:
Code: - if [ "$CONVERT" == "convert" ]; then
+ if [ "$CONVERT" = "convert" ]; then
As documented here:
https://git.sr.ht/~martijnbraam/megapixe...process.sh
I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).
I get .dng and .jpg now so I need to double check that you have the same helper aps that I'm using, which I'll do later today.
-Cal
Must have missed that, I'll make the change and see how it works, thanks!
04-28-2021, 04:44 PM
(This post was last modified: 04-28-2021, 04:46 PM by calinb.)
(04-28-2021, 04:29 PM)NormandC Wrote: Hello Cal,
(04-28-2021, 04:16 PM)calinb Wrote: I guess it still hasn't rolled out in a Mobian update / upgrade yet?
Unfortunately not, the megapixel package in Mobian is still 0.15, and the fix is in the 0.16 release. Because of the bullseye upcoming release, matters are in some limbo.
Thanks for the update, Norm!
Hopefully, Zebulon will be able to similarly patch his postprocess.sh file and get his .jpg files back too.
-Cal
(04-28-2021, 04:40 PM)Zebulon Walton Wrote: Must have missed that, I'll make the change and see how it works, thanks!
I missed your post while I was replying to Norm.
Please let us know how it goes.
-Cal
04-28-2021, 05:21 PM
(This post was last modified: 04-28-2021, 05:21 PM by Zebulon Walton.)
(04-28-2021, 04:44 PM)calinb Wrote: Thanks for the update, Norm!
Hopefully, Zebulon will be able to similarly patch his postprocess.sh file and get his .jpg files back too.
-Cal
I missed your post while I was replying to Norm.
Please let us know how it goes.
-Cal
Success, the jpg is now being produced. (Takes a little while for processing, nature of the beast.) The dng version of the photo is also being saved. Is there a shell script mod to have those automagically deleted?
(04-28-2021, 05:21 PM)Zebulon Walto Wrote: Success, the jpg is now being produced. (Takes a little while for processing, nature of the beast.) The dng version of the photo is also being saved. Is there a shell script mod to have those automagically deleted?
I can't look at the script right now, but IIRC, I there might be a problem with the simple postprocess.sh script in targeting just the latest .dng files for deletion. However, I think I quickly came to the conclusion that it would be possible and simple to just "rm ~/Pictures/*.dng" to nuke all of them after each invocation of the script--or maybe mv them to Trash.
04-28-2021, 06:30 PM
(This post was last modified: 04-28-2021, 06:30 PM by Zebulon Walton.)
(04-28-2021, 05:50 PM)calinb Wrote: I can't look at the script right now, but IIRC, I there might be a problem with the simple postprocess.sh script in targeting just the latest .dng files for deletion. However, I think I quickly came to the conclusion that it would be possible and simple to just "rm ~/Pictures/*.dng" to nuke all of them after each invocation of the script--or maybe mv them to Trash.
Of course, simple solution!
I added:
Code: rm "$TARGET_NAME.dng"
as the last line of postprocess.sh.
----------------------------------------
Thanks for all the debugging this was a super helpful thread!
Cheers!
(04-28-2021, 04:29 PM)NormandC Wrote: ...
(04-28-2021, 04:16 PM)calinb Wrote: I thought I described my fix in my last posts in this thread (after mucking my way through the debug effort).
You did, and I applied your fix a couple weeks ago. Thanks for that!
- Norm
+1 thanks to @ calinb
As identified the "silver bullet" in megapixels is the post-processing script. - The basic camera output is a raw .dng format
- It is converted to .TIFF if dcraw is installed
- The .TIFF is converted to .jpg if imagemagick is installed
- The .jpg file gets some exif data if libimage-exiftool-perl is installed (in Mobian - Arch has a different name for it)
I don't have a note on what any default Mobian config has but for sure I always ensure I have dcraw, imagemagick and libimage-exiftool-perl packages installed each time I rebuild my Mobian system.
The other nugget for postprocess.sh is if you put a version in ~/.config/megapixels/ then that is used in preference to the default in /usr/share/megapixels/ . Which will survive any upgrades and can do stuff the way you prefer. So my version has 3 mods:
- the @calinb / v0.16 patch
- I have commented out the initial copy of the .dng file to /Pictures as I dont want to preserve any raw images
- I have added an exiftool line (and suggested it to Martijn) to set the DateTimeOriginal tag. (Because the package I use for my media library, fotoxx, can sort on that tag!)
My version
Code: $ cat .config/megapixels/postprocess.sh
#!/bin/sh
# The post-processing script gets called after taking a burst of
# pictures into a temporary directory. The first argument is the
# directory containing the raw files in the burst. The contents
# are 1.dng, 2.dng.... up to the number of photos in the burst.
#
# The second argument is the filename for the final photo without
# the extension, like "/home/user/Pictures/IMG202104031234"
#
# The post-processing script is responsible for cleaning up
# temporary directory for the burst.
if [ "$#" -ne 2 ]; then
echo "Usage: $0 [burst-dir] [target-name]"
exit 2
fi
BURST_DIR="$1"
TARGET_NAME="$2"
MAIN_PICTURE="$BURST_DIR"/1
# Copy the first frame of the burst as the raw photo
# cp "$BURST_DIR"/1.dng "$TARGET_NAME.dng"
# Create a .jpg if raw processing tools are installed
DCRAW=""
TIFF_EXT="dng.tiff"
if command -v "dcraw_emu" > /dev/null
then
DCRAW=dcraw_emu
# -fbdd 1 Raw denoising with FBDD
set -- -fbdd 1
elif [ -x "/usr/lib/libraw/dcraw_emu" ]; then
DCRAW=/usr/lib/libraw/dcraw_emu
# -fbdd 1 Raw denoising with FBDD
set -- -fbdd 1
elif command -v "dcraw" > /dev/null
then
DCRAW=dcraw
TIFF_EXT="tiff"
set --
fi
CONVERT=""
if command -v "convert" > /dev/null
then
CONVERT="convert"
# -fbdd 1 Raw denoising with FBDD
set -- -fbdd 1
elif command -v "gm" > /dev/null
then
CONVERT="gm"
fi
if [ -n "$DCRAW" ]; then
# +M use embedded color matrix
# -H 4 Recover highlights by rebuilding them
# -o 1 Output in sRGB colorspace
# -q 3 Debayer with AHD algorithm
# -T Output TIFF
$DCRAW +M -H 4 -o 1 -q 3 -T "$@" "$MAIN_PICTURE.dng"
# If imagemagick is available, convert the tiff to jpeg and apply slight sharpening
if [ -n "$CONVERT" ];
then
if [ "$CONVERT" = "convert" ]; then
convert "$MAIN_PICTURE.$TIFF_EXT" -sharpen 0x1.0 "$TARGET_NAME.jpg"
else
gm convert "$MAIN_PICTURE.$TIFF_EXT" -sharpen 0x1.0 "$TARGET_NAME.jpg"
fi
# If exiftool is installed copy the exif data over from the tiff to the jpeg
# since imagemagick is stupid
if command -v exiftool > /dev/null
then
exiftool -tagsFromfile "$MAIN_PICTURE.$TIFF_EXT" \
-software="Megapixels" \
-overwrite_original "$TARGET_NAME.jpg"
exiftool -tagsFromFile @ "-ModifyDate>DateTimeOriginal" -overwrite_original "$TARGET_NAME.jpg"
fi
echo "$TARGET_NAME.jpg"
else
cp "$MAIN_PICTURE.$TIFF_EXT" "$TARGET_NAME.tiff"
echo "$TARGET_NAME.tiff"
fi
fi
# Clean up the temp dir containing the burst
rm -rf "$BURST_DIR"
- ROCKPro64 v2.1 2GB, 16Gb eMMC for rootfs, SX8200Pro 512GB NVMe for /home, HDMI video & sound, Bluetooth keyboard & mouse. Arch (6.2 kernel, Openbox desktop) for general purpose daily PC.
- PinePhone Pro Explorer Edition, daily driver, rk2aw & U-boot on SPI, Arch/SXMO & Arch/phosh on eMMC
- PinePhone BraveHeart now v1.2b 3/32Gb, Tow-boot with Arch/SXMO on eMMC
04-29-2021, 07:09 AM
(This post was last modified: 04-29-2021, 07:37 AM by Zebulon Walton.)
Quote:@dukla2000:
The other nugget for postprocess.sh is if you put a version in ~/.config/megapixels/ then that is used in preference to the default in /usr/share/megapixels/ . Which will survive any upgrades and can do stuff the way you prefer. So my version has 3 mods:
the @calinb / v0.16 patch
I have commented out the initial copy of the .dng file to /Pictures as I dont want to preserve any raw images
I have added an exiftool line (and suggested it to Martijn) to set the DateTimeOriginal tag. (Because the package I use for my media library, fotoxx, can sort on that tag!)
Neat, I copied the modified script to ~/.config/megapixels to play around with. Full processing of a photo with everything enabled takes about 45-50 seconds. By eliminating the exiftool step (I don't care about metadata) that gets cut down to about 30 seconds. Now I'm admittedly not familiar with the image processing tools, but if I understand correctly the script uses tiff format as an intermediate step to get to jpg (dng-->tiff-->jpg). Tiff format as far as I know is not widely used for photos and seems kind of a waste, would it be feasible to convert directly from dng to jpg to save processing time?
Another thing I'm noticing is the jpegs being produced now are much larger than when I last used the camera months ago, my photos from back then are under a megabyte, new ones are 3-4 megabytes. Unless I reduced the size of the old photos and forgot (which is possible) something changed.
(04-29-2021, 07:09 AM)Zebulon Walton Wrote: ... Now I'm admittedly not familiar with the image processing tools, but if I understand correctly the script uses tiff format as an intermediate step to get to jpg (dng-->tiff-->jpg). Tiff format as far as I know is not widely used for photos and seems kind of a waste, would it be feasible to convert directly from dng to jpg to save processing time?
...
Yes but ...!
Technically imagemagick could do a direct conversion .dng -> .jpg. The "but" is that dcraw is doing more than just a format conversion, it is using some magic options (chosen by Martijn) to try make the picture look better as per the comments in the postprocess script. dcraw cannot output .jpg though. But again the megapixels choice to use imagemagick to do the conversion to .jpg also allows the chance to get imagemagick to try sharpen things.
Or you could cut out most of the (megapixels) postprocessing and use a package like RawTherapee to process your images.
The fundamental limitation though is the inexpensive sensors chosen by Pine64 are pretty much point and click cheapos and so no amount of postprocessing can turn me into David Bailey! Then again, even with slightly expensive kit I struggle to get stuff in focus so having the PinePhone for quick snaps is fine by me.
- ROCKPro64 v2.1 2GB, 16Gb eMMC for rootfs, SX8200Pro 512GB NVMe for /home, HDMI video & sound, Bluetooth keyboard & mouse. Arch (6.2 kernel, Openbox desktop) for general purpose daily PC.
- PinePhone Pro Explorer Edition, daily driver, rk2aw & U-boot on SPI, Arch/SXMO & Arch/phosh on eMMC
- PinePhone BraveHeart now v1.2b 3/32Gb, Tow-boot with Arch/SXMO on eMMC
|