Linux images are impossible to download.

Linux images are impossible to download. - Printable Version

+- PINE64 (https://forum.pine64.org)
+-- Forum: ROCKPRO64 (https://forum.pine64.org/forumdisplay.php?fid=98)
+--- Forum: Linux on RockPro64 (https://forum.pine64.org/forumdisplay.php?fid=101)
+--- Thread: Linux images are impossible to download. (/showthread.php?tid=6132)

Pages: 1 2

Linux images are impossible to download. - superpat - 06-02-2018

Hi,

Today, Saturday morning, I cannot successfully download any images from the github ayufan directory.

Using LInux mint, Firefox with S3 download manager addon.

Download starts, runs for short time, then hangs. I can resume it several times, maybe getting to 40-50% then down ceases and is CANCELLED by external event, not me!

This forces complete restart. I have run through my IP's data allowance several times over trying to get one iso complete!

PLEASE COULD PINE PROVIDE AN ALTERNATE DOWNLOAD SITE THAT IS RELIABLE??

regards

Patrick

RE: Linux images are impossible to download. - Luke - 06-02-2018

(06-02-2018, 05:07 AM)superpat Wrote: Hi,

Today, Saturday morning, I cannot successfully download any images from the github ayufan directory.

Using LInux mint, Firefox with S3 download manager addon.

Download starts, runs for short time, then hangs. I can resume it several times, maybe getting to 40-50% then down ceases and is CANCELLED by external event, not me!

This forces complete restart. I have run through my IP's data allowance several times over trying to get one iso complete!

PLEASE COULD PINE PROVIDE AN ALTERNATE DOWNLOAD SITE THAT IS RELIABLE??

regards

Patrick

I just fetched 3 images from ayufan's git just fine the its something wrong on your end. Also, what does PINE64 have to do with it ? : Smile

Once there is a release (stable image) it'll get added into the wiki and installer. Until then you can test pre-releases as they get pushed to git.

RE: Linux images are impossible to download. - superpat - 06-02-2018

(06-02-2018, 08:59 AM)Luke Wrote:
(06-02-2018, 05:07 AM)superpat Wrote: Hi,

Today, Saturday morning, I cannot successfully download any images from the github ayufan directory.

Using LInux mint, Firefox with S3 download manager addon.

Download starts, runs for short time, then hangs. I can resume it several times, maybe getting to 40-50% then down ceases and is CANCELLED by external event, not me!

This forces complete restart. I have run through my IP's data allowance several times over trying to get one iso complete!

PLEASE COULD PINE PROVIDE AN ALTERNATE DOWNLOAD SITE THAT IS RELIABLE??

regards

Patrick

I just fetched 3 images from ayufan's git just fine the its something wrong on your end. Also, what does PINE64 have to do with it ? : Once there is a release (stable image) it'll get added into the wiki and installer. Until then you can test pre-releases as they get pushed to git.

Hi Luke,

You cannot escape your, (Pine64)'s responsibility to provide a reliable download source, Git hub is the PITS, Google for git hub and failing downloads and you will find numerous people complaining of the same problem from various locations. It does appear to be a geographic problem, which I in the UK am suffering from suffering badly.
I can download an Android iso from Pine64 with no drop outs, but git hub is impossible.

If you look at the retro pie forum, they are suffering the same git hub problem and have set up a torrent to ease their situation.

I can appreciate that eventually you will move the Linux isos to a Pine64 site for downloading, which would cure the problem but when?

I purchased the Rockpro in good faith, anticipating that I would have access to the iso files as required during development but this is proving not to be the case. I do not want to have to wait for a year until you have a production release before I can use my Rockpro.

Here is the first page of a search for "Git hub + download failures"

Quote:Can't finish download of any big file · Issue #5390 · nextcloud ... - GitHub
https://github.com/nextcloud/server/issues/5390
13 Jun 2017 - Previously, right after the download failed, I saw an error in logs, identical to the one described in issue #1166, but after I applied the patch ...
Failed: Network Error - when downloading a file in chrome ... - GitHub
https://github.com/nghttp2/nghttp2/issues/678
6 Sep 2016 - But when chrome connects to nghttpx HTTP/2 proxy port ( by SwitchyOmega ), which backend is squid http proxy port, the download fails.
lfs files failing to download · Issue #904 · git-lfs/git-lfs · GitHub
https://github.com/git-lfs/git-lfs/issues/904
18 Dec 2015 - I'm having an issue where git-lfs pull is failing to download all of the files it ... The failures only seem to happen when we add new files to git-lfs.
Ability to resume failed/interrupted downloads in progress - GitHub
https://github.com/itchio/itch.io/issues/353
20 Feb 2016 - GitHub is where people build software. More than 27 million people use GitHub to discover, fork, and contribute to over 80 million projects.
GitHub - rust-lang-nursery/failure: Error management
https://github.com/rust-lang-nursery/failure
Error management https://boats.gitlab.io/failure · rust error- ... Find file. Clone or download ... failure is designed to make it easier to manage errors in Rust.
Downloads failing consistently · Issue #167 · dscottbuch/cTiVo · GitHub
https://github.com/dscottbuch/cTiVo/issues/167
15 Apr 2016 - I can see the list of items in my Tivo, but every download fails with Tivo Show Failed; retrying. Looking for any insight in how to debug and fix.
Download failures · Issue #193 · opengapps/opengapps · GitHub
https://github.com/opengapps/opengapps/issues/193
9 Dec 2015 - GitHub is where people build software. More than 27 million people use GitHub to discover, fork, and contribute to over 80 million projects.
Intermittent failures downloading large files · Issue #7 ... - GitHub
https://github.com/JFrogDev/artifactory-cli-go/issues/7
Intermittent failures downloading large files #7. Closed. ssssam opened this Issue on Dec 8, 2015 · 6 comments ...
Download failure rate >50%

These are not new reports, but it is obvious they still have problems with large files!

Would it not be possible for ayufan to upload his pre-release iso's to a file storage/ download site as well as to git hub, I am sure it would not break pine 64's bank to pay for a few gigs of storage space.

cheers

Patrick

(06-02-2018, 08:59 AM)Luke Wrote:
(06-02-2018, 05:07 AM)superpat Wrote: Hi,

Today, Saturday morning, I cannot successfully download any images from the github ayufan directory.

Using LInux mint, Firefox with S3 download manager addon.

Download starts, runs for short time, then hangs. I can resume it several times, maybe getting to 40-50% then down ceases and is CANCELLED by external event, not me!

This forces complete restart. I have run through my IP's data allowance several times over trying to get one iso complete!

PLEASE COULD PINE PROVIDE AN ALTERNATE DOWNLOAD SITE THAT IS RELIABLE??

regards

Patrick

I just fetched 3 images from ayufan's git just fine the its something wrong on your end. Also, what does PINE64 have to do with it ? : Once there is a release (stable image) it'll get added into the wiki and installer. Until then you can test pre-releases as they get pushed to git.

Quote:Can't finish download of any big file · Issue #5390 · nextcloud ... - GitHub
https://github.com/nextcloud/server/issues/5390
13 Jun 2017 - Previously, right after the download failed, I saw an error in logs, identical to the one described in issue #1166, but after I applied the patch ...
Failed: Network Error - when downloading a file in chrome ... - GitHub
https://github.com/nghttp2/nghttp2/issues/678
6 Sep 2016 - But when chrome connects to nghttpx HTTP/2 proxy port ( by SwitchyOmega ), which backend is squid http proxy port, the download fails.
lfs files failing to download · Issue #904 · git-lfs/git-lfs · GitHub
https://github.com/git-lfs/git-lfs/issues/904
18 Dec 2015 - I'm having an issue where git-lfs pull is failing to download all of the files it ... The failures only seem to happen when we add new files to git-lfs.
Ability to resume failed/interrupted downloads in progress - GitHub
https://github.com/itchio/itch.io/issues/353
20 Feb 2016 - GitHub is where people build software. More than 27 million people use GitHub to discover, fork, and contribute to over 80 million projects.
GitHub - rust-lang-nursery/failure: Error management
https://github.com/rust-lang-nursery/failure
Error management https://boats.gitlab.io/failure · rust error- ... Find file. Clone or download ... failure is designed to make it easier to manage errors in Rust.
Downloads failing consistently · Issue #167 · dscottbuch/cTiVo · GitHub
https://github.com/dscottbuch/cTiVo/issues/167
15 Apr 2016 - I can see the list of items in my Tivo, but every download fails with Tivo Show Failed; retrying. Looking for any insight in how to debug and fix.
Download failures · Issue #193 · opengapps/opengapps · GitHub
https://github.com/opengapps/opengapps/issues/193
9 Dec 2015 - GitHub is where people build software. More than 27 million people use GitHub to discover, fork, and contribute to over 80 million projects.
Intermittent failures downloading large files · Issue #7 ... - GitHub
https://github.com/JFrogDev/artifactory-cli-go/issues/7
Intermittent failures downloading large files #7. Closed. ssssam opened this Issue on Dec 8, 2015 · 6 comments ...
Download failure rate >50%

RE: Linux images are impossible to download. - Luke - 06-02-2018

Quote:(Pine64)'s responsibility to provide a reliable download source

There is a very reliable server for stable and end-user ready OS images (labeled release). Pre-release images, some of which do not work, shouldn't be distributed to end-users for obvious reasons.
Also, just to make it clear, all images from ayufan are community builds and at the time of writing pre-release/ alpha images.

Quote:It does appear to be a geographic problem, which I in the UK am suffering from suffering badly

Strange, I live in London and never had any problems. Same goes for other people in the community on the isles.

Quote:I can appreciate that eventually you will move the Linux isos to a Pine64 site for downloading, which would cure the problem but when?

When there is a stable release image, so we don't have a bunch of people whining about something not working.

Quote:[I was] anticipating that I would have access to the iso files as required during development but this is proving not to be the case. I do not want to have to wait for a year until you have a production release

Two things. As I mentioned, hardly PINE64's fault that you cant download the images from GIT. Second, I think that you'll see a stable release within a month or so, not a year. Please do appreciate that the board has been out just for 2 weeks (and the devs had literally that long time to get it working). Images are hardly production-ready.

Quote:[could] ayufan upload his pre-release iso's to aile storage/ download site as well as to git hub, I am sure it would not break pine 64's bank to pay for a few gigs

You'd have to ask ayufan (chat icon at the very top of site) but I doubt it. Also, nothing to do with server space or money, as explained earlier.

Hope this clears things up. Regardless, I am happy to upload an image for you to google drive for testing - which one do you need ? Again, do keep in mind these are pre-release images and may not even boot.

Linux images are impossible to download. - jsfrederick - 06-02-2018

[/quote]
Hi Luke,

You cannot escape your, (Pine64)'s responsibility to provide a reliable download source, Git hub is the PITS, Google for git hub and failing downloads and you will find numerous people complaining of the same problem from various locations. It does appear to be a geographic problem, which I in the UK am suffering from suffering badly.

Patrick
[/quote]

Pine64 has absolutely NO responsibility to provide Ayufan's images to you. Ayufan builds those images outside of Pin64's control or approval. If you have an issue with them, contact Ayufan or GitHub.

Others, including myself, are not having any issues so it IS something on your side. You might want to contact your internet provider and see if they can assist you.

Sent from my iPad using Tapatalk

RE: Linux images are impossible to download. - pfeerick - 06-02-2018

Not to add fuel to the fire, but it is not necessarily his internet connection. The OP is correct in indicating that large file downloads (specifically releases) are occasionally flakey. I (in Australia, not the UK) only have issues with downloading from github - nowehere else, but it is intermittent. I offset this issue by using aria2, usually with 4 connections (-x4) and when I got the 403 errors from github I just run the download again to let aria2 resume and finish. Often in my case it fails in the last 1-2%, so it only takes a coulpe of seconds to finish then.

As far as the bunch of links provided, I would question the relevance of some of them, as git-lfs and other git releated issues are nothing to do with the releases hosting provided via Amazon Web Services.

However, I strongly agree with Luke. Images which are considered stable and ready for use are also hosted by pine64 themselves. The 'pre-release' images from ayufan can be considered alpha/development/testing releases, and the 'release' images from ayufan can be considered beta images.

If your problem is accessing the rockpro64 images, I would be willing upload a select few to my VPS over in Amsterdam and give links for HTTP download and might try seeing if I can get torrent hosting going. Tell me which of the images you want (strech, bionic, minimal, mate, omv, etc) and I'll download and mirror them. I'll only do a few as whilst network bandwidth isn't an issue (unmetered 200Mbit), storage is Wink

RE: Linux images are impossible to download. - elatllat - 06-03-2018

If anyone has trouble obtaining an image post a link to the one you want and I'll torrent or mirror it.

RE: Linux images are impossible to download. - Tigger - 06-04-2018

(06-02-2018, 10:01 AM)superpat Wrote: .......... Git hub is the PITS, Google for git hub and failing downloads and you will find numerous people complaining of the same problem from various locations.

Be fair to Github. It was never intended for, and is not suited for (large) binaries in a repo.

Binaries bloat the repo (each one is kept in the repo's history) because binaries can't be diffed by git. And each time that the binary is updated or a new one added, the repo's size is increased by that binary file's size. Code/Text files just store the diff with each update. In short the repo gets larger and slower.

And Releases at Github were meant for versioned releases of source code and, optionally, compiled programs. Not for binaries like ISOs : https://help.github.com/articles/creating-releases/.

The fact that repo owners use the Releases section of their repo(s) to make large binaries like ISOs available, is only indicative of how lenient Github is towards this. They could easily put a limit on the size of binaries in the Releases section and in the actual repo itself.

RE: Linux images are impossible to download. - ayufan - 07-04-2018

Tigger. They put. It is 2GB per-file, an unlimited amount of files. So this is a conscious decision of them to give 2GB of per-file storage for releases.

And. I use GitHub Releases, not Git. Git does not hold binaries, only sources.

RE: Linux images are impossible to download. - Tigger - 07-04-2018

(07-04-2018, 02:29 PM)ayufan Wrote: Tigger. They put. It is 2GB per-file, an unlimited amount of files. So this is a conscious decision of them to give 2GB of per-file storage for releases.

And. I use GitHub Releases, not Git. Git does not hold binaries, only sources.

I didn't say anything good nor bad about developers releasing binaries for download at Github. I was rooting for Github. Because people who only download from Github, don't realise what it's for. It's a site that I use every day. Even after work.

About the 2GB limit on binary size for Releases : there is also no bandwidth limit. They just force you to split up the binary when it's larger than 2GB (as you wrote). Something HJ-Split can easily do (split before upload, and recombine after download). Just chop a 10 GB binary monster file in 5 x 2GB chunks. So theoretically there is no limit for Releases.

https://help.github.com/articles/distributing-large-binaries/ Wrote:We don't limit the total size of your binary release files, nor the bandwidth used to deliver them. However, each individual file must be under 2 GB in size.

If only a repo's recommended max size wasn't 1 GB. Github will trigger a polite email when a repo goes over 1GB.

https://help.github.com/articles/what-is-my-disk-quota/ Wrote:GitHub doesn't have any set disk quotas. We try to provide abundant storage for all Git repositories, within reason. Keeping repositories small ensures that our servers are fast and downloads are quick for our users.
..........
We recommend repositories be kept under 1GB each. This limit is easy to stay within if large files are kept out of the repository. If your repository exceeds 1GB, you might receive a polite email from GitHub Support requesting that you reduce the size of the repository to bring it back down.

There is a (git push) max file size limit of 100MB for repos (and a warning when pushing files from 50+ MB - 100MB).
But, Github does mention Git LFS (Large File Storage) on the page of the following quote though. In the blue box. So indirectly there is also no file size limit for repos. When Git LFS and HJ-Split would be used for GB sized files.

https://help.github.com/articles/working-with-large-files/ Wrote:Conditions for large files
GitHub will warn you when pushing files larger than 50 MB. You will not be allowed to push files larger than 100 MB.

There is also a git-fit repo. But I don't know what its status is now.

And, I did describe the Repos (Git) and Releases separate, deliberately. EDIT: Also, it is possible to push binaries to a Git repo. Not just source code.

Maybe when the takeover by Microsoft is fully completed, the infra will be better suited to binary downloads (due to Microsoft's cloud). And then people can download without having to go on the net to write stuff like "Github is the PITS" over failed downloads.