The lack of resources about Pine64 on search engines was suddenly explained when I found out that both the forum and the wiki had their "
robots.txt" files configured to forbid indexing:
Code:
User-agent: *
Disallow: /
Can the person responsible for the domains please enable indexing of the contents and make our user lives easier?
(06-09-2016, 09:00 PM)lvella Wrote: [ -> ]The lack of resources about Pine64 on search engines was suddenly explained when I found out that both the forum and the wiki had their "robots.txt" files configured to forbid indexing:
Code:
User-agent: *
Disallow: /
Can the person responsible for the domains please enable indexing of the contents and make our user lives easier?
We purposely disabled this due to it will generate a lot of soliciting traffic and slow down forum.
(06-10-2016, 01:38 AM)tllim Wrote: [ -> ] (06-09-2016, 09:00 PM)lvella Wrote: [ -> ]The lack of resources about Pine64 on search engines was suddenly explained when I found out that both the forum and the wiki had their "robots.txt" files configured to forbid indexing:
Code:
User-agent: *
Disallow: /
Can the person responsible for the domains please enable indexing of the contents and make our user lives easier?
We purposely disabled this due to it will generate a lot of soliciting traffic and slow down forum.
The forum allows only one search per 30 seconds, but Google gives all of us for free multiple searches per second. I don't know what is the burden of the bots on the servers, but looking for where I stand, it seems hardly worthwhile to forbid them, given the benefits (and that only good bots honors robots.txt).
(06-10-2016, 11:23 PM)lvella Wrote: [ -> ] (06-10-2016, 01:38 AM)tllim Wrote: [ -> ] (06-09-2016, 09:00 PM)lvella Wrote: [ -> ]The lack of resources about Pine64 on search engines was suddenly explained when I found out that both the forum and the wiki had their "robots.txt" files configured to forbid indexing:
Code:
User-agent: *
Disallow: /
Can the person responsible for the domains please enable indexing of the contents and make our user lives easier?
We purposely disabled this due to it will generate a lot of soliciting traffic and slow down forum.
The forum allows only one search per 30 seconds, but Google gives all of us for free multiple searches per second. I don't know what is the burden of the bots on the servers, but looking for where I stand, it seems hardly worthwhile to forbid them, given the benefits (and that only good bots honors robots.txt).
I will explore with IT manager again, he against the idea on opening.