FAQ   Search   Memberlist  
Profile    Log in to check your private messages    Register    Log in
rsyncing blag repo (mirror)
Goto page 1, 2  Next
 
Post new topic   Reply to topic    BLAG Forum Index -> open discussion
View previous topic :: View next topic  
Author Message
john maclean
PostPosted: Thu Feb 28, 2008 11:28 am    Post subject: rsyncing blag repo (mirror) Reply with quote

In an attempt to mirror the blag 70k repo, I've found that I keep getting time outs;
Code:
rsync --timeout=600 --archive  rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag/
rsync: opendir "/70000/en/os/i386/SRPMS.errata/.~tmp~" (in blag) failed: Permission denied (13)
rsync: opendir "/70000/en/os/i386/SRPMS.TESTING/.~tmp~" (in blag) failed: Permission denied (13)
rsync: opendir "/70000/en/os/i386/SRPMS.TESTING/repodata/.~tmp~" (in blag) failed: Permission denied (13)
rsync: opendir "/70000/en/os/i386/BLAG/RPMS.errata/.~tmp~" (in blag) failed: Permission denied (13)
rsync: opendir "/70000/en/os/i386/BLAG/RPMS.TESTING/.~tmp~" (in blag) failed: Permission denied (13)
io timeout after 600 seconds -- exiting
rsync error: timeout in data send/receive (code 30) at io.c(165) [receiver=2.6.9]
rsync: connection unexpectedly closed (1581858 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(453) [generator=2.6.9]


It's a 100mbps connection, up and down. Any way to resolve this? Note that I've also tried the --bwlimit=X and --compress options


_________________
BLAG 'em up!
jebba
PostPosted: Fri Feb 29, 2008 8:45 pm    Post subject: Reply with quote

Uh, related to the power cut over I've upgraded that box (kernel & rsync server), let me know if you still see the issue. thx
john maclean
PostPosted: Sat Mar 01, 2008 11:07 am    Post subject: Reply with quote

http://ks355011.kimsufi.com/

I want some testers. please run this command and post the results here
Code:
 time wget http://ks355011.kimsufi.com/70000/en/iso/BLAG-70000.iso


Tell me your location, type of connection, speed and total time for the download. This is a mirror in testing! Do not add this to your repo list! There are still a few things to iron out, like stats and keep alives...


_________________
BLAG 'em up!
john maclean
PostPosted: Sat Mar 01, 2008 11:27 am    Post subject: Reply with quote

Getting an average of 240K/sec download from London.

_________________
BLAG 'em up!
john maclean
PostPosted: Sat Mar 01, 2008 1:59 pm    Post subject: Reply with quote

Code:
time wget http://ks355011.kimsufi.com/70000/en/iso/BLAG-70000.iso
--11:21:23--  http://ks355011.kimsufi.com/70000/en/iso/BLAG-70000.iso
           => `BLAG-70000.iso'
Resolving ks355011.kimsufi.com... 91.121.115.84
Connecting to ks355011.kimsufi.com|91.121.115.84|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 722,649,088 (689M) [application/octet-stream]

100%[====================================>] 722,649,088  207.09K/s    ETA 00:00

12:17:38 (209.42 KB/s) - `BLAG-70000.iso' saved [722649088/722649088]


real    56m15.637s
user    0m2.201s
sys     0m12.820s


_________________
BLAG 'em up!
platypuss72
PostPosted: Sat Mar 01, 2008 2:16 pm    Post subject: speedtest Reply with quote

i am in hucknall (nottingham uk) , on virginmedia cable 2mb base package, running blag 70000 with kde 3.5 , on athalon 1.8, 512mb ram , 2 hard drive (40gb+ 120gb)

the wget test was

--11:52:07-- http://ks355011.kimsufi.com/70000/en/iso/BLAG-70000.iso
=> `BLAG-70000.iso.1'
Resolving ks355011.kimsufi.com... 91.121.115.84
Connecting to ks355011.kimsufi.com|91.121.115.84|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 722,649,088 (689M) [application/octet-stream]

100%[====================================>] 722,649,088 250.55K/s ETA 00:00

13:30:59 (118.99 KB/s) - `BLAG-70000.iso.1' saved [722649088/722649088]


real 98m52.282s
user 0m12.048s
sys 0m47.015s

john maclean
PostPosted: Sat Mar 01, 2008 2:46 pm    Post subject: Reply with quote

OK thanks. Got three results so far. If you don't want to use the wget command you'll still be able to get the iso image with a browser, (even if you use IE), however you will not get any details of the download speeds.

_________________
BLAG 'em up!
extraspecialbitter
PostPosted: Sat Mar 01, 2008 3:42 pm    Post subject: Reply with quote

Pretty speedy (>460K/sec) from Cochituate, MA (USA) via Comcast Cable broadband and a Netgear FVS318 router. The whole download took just over 25 minutes. Where is the mirror located?

Code:
pablo@brixton=> time wget http://ks355011.kimsufi.com/70000/en/iso/BLAG-70000.iso
--10:09:43--  http://ks355011.kimsufi.com/70000/en/iso/BLAG-70000.iso
           => `BLAG-70000.iso'
Resolving ks355011.kimsufi.com... 91.121.115.84
Connecting to ks355011.kimsufi.com|91.121.115.84|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 722,649,088 (689M) [application/octet-stream]

100%[====================================>] 722,649,088  509.31K/s    ETA 00:00

10:35:10 (462.21 KB/s) - `BLAG-70000.iso' saved [722649088/722649088]


real    25m27.352s
user    0m2.967s
sys     0m16.680s

john maclean
PostPosted: Sat Mar 01, 2008 4:11 pm    Post subject: Reply with quote

Thanks for these results. Note that the URL points to http://ks355011.kimsufi.com/ . You can search the whole dir with a browser as there is no ftp set up just yet. I may move it to something more descriptive like http://ks355011.kimsufi.com/distros/blag if it makes any difference.

Once the speeds are confirmed to be "decent" I'll start running a cronjob, rsync'ing the 70k repository twice a day.


_________________
BLAG 'em up!
john maclean
PostPosted: Wed Mar 05, 2008 9:52 pm    Post subject: Reply with quote

Any more offers of results for connections and download speeds?

_________________
BLAG 'em up!
Crwyddwr
PostPosted: Thu Mar 06, 2008 4:53 pm    Post subject: Reply with quote

Cardiff, Wales on an ADSL connection

Code:
 time wget http://ks355011.kimsufi.com/70000/en/iso/BLAG-70000.iso
--14:56:31--  http://ks355011.kimsufi.com/70000/en/iso/BLAG-70000.iso
           => `BLAG-70000.iso'
Resolving ks355011.kimsufi.com... 91.121.115.84
Connecting to ks355011.kimsufi.com|91.121.115.84|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 722,649,088 (689M) [application/octet-stream]

100%[====================================>] 722,649,088  211.07K/s    ETA 00:00

15:42:15 (257.62 KB/s) - `BLAG-70000.iso' saved [722649088/722649088]


real    45m44.520s
user    0m2.940s
sys     0m17.633s

stevo32
PostPosted: Fri Mar 07, 2008 3:33 am    Post subject: Reply with quote

I get an average of 100-110kbps from Hamilton, ON on Bell Sympatico (connection speed is 175kbps).

_________________
E-mail me at s.clement@localhost (replace localhost with sympatico.ca) or stevo32@localhost (replace localhost with blagblagblag.org).
john maclean
PostPosted: Fri Mar 07, 2008 12:17 pm    Post subject: Reply with quote

You can get some stats on the box here http://ks355011.kimsufi.com/munin/localdomain/index.html . I've just started a cronjob to keep the 70k mirror up to date twice a day.

_________________
BLAG 'em up!
jebba
PostPosted: Fri Mar 07, 2008 2:51 pm    Post subject: Reply with quote

Can you post your cron script here?
john maclean
PostPosted: Fri Mar 07, 2008 10:45 pm    Post subject: Reply with quote

Code:
 ssh root@ks355011.kimsufi.com crontab -l                                                                                                                 
# m h  dom mon dow   command                                                                                                                                             
                                                                                                                                                                         
0 0,12 * * *  root rsync --timeout=600 --archive  rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag/ --verbose


_________________
BLAG 'em up!
Display posts from previous:   
Post new topic   Reply to topic    BLAG Forum Index -> open discussion Goto page 1, 2  Next
Page 1 of 2

Protected by Anti-Spam ACP