| View previous topic :: View next topic |
| Author |
Message |
benj
|
 Posted: Fri Jun 20, 2008 1:59 pm Post subject: |
 |
|
Ok, I think I've got everything I need to open tickets with Level3 and Cogent. I'll keep everyone updated with what we hear from them, and will let you know if we need anything further.
Thanks!
|
|
|
 |
jebba
|
| Posted: Fri Jun 20, 2008 2:38 pm Post subject: |
|
|
Cool, thanks. keep us posted. :)
|
|
|
|
benj
|
| Posted: Thu Jun 26, 2008 7:55 pm Post subject: |
|
|
Hello all,
We're still hitting our heads against the wall on this one. jebba has reported 9Mbps rsyncs from cascompany and 400Kbps using wget FTP.
If at all possible, could we get some more rsync examples?
I'm not overly familiar with rsync, can someone explain how it works, or why speeds might be higher using rsync as opposed to ftp / wget?
Again, all of us here at FRII NOC really do appreciate your help in finding a solution to this issue. I know testing can be time-consuming, but you all are really the ones needed for tests.
Thanks again.
|
|
|
|
john maclean
|
| Posted: Thu Jun 26, 2008 8:11 pm Post subject: |
|
|
| Code: | crontab -l
# m h dom mon dow command
0 0,6,12 * * * rsync --timeout=600 --archive rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag/ --verbose
|
I use the above to mirror the BLAG 70k repository. Will run this from scratch and post a time here....
|
_________________
BLAG 'em up! |
|
|
benj
|
| Posted: Thu Jun 26, 2008 8:30 pm Post subject: |
|
|
| john maclean wrote: | | Code: | crontab -l
# m h dom mon dow command
0 0,6,12 * * * rsync --timeout=600 --archive rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag/ --verbose
|
I use the above to mirror the BLAG 70k repository. Will run this from scratch and post a time here.... |
Cool, I was wondering how to run rsync. I'll try it from here, a colocate in Texas, and a box at home.
Thanks!
|
|
|
|
john maclean
|
| Posted: Mon Jun 30, 2008 9:53 am Post subject: |
|
|
If any one runs ruby I have some scripts that you can run to download stuff from frii and blagblagblag.org and publish the results.
|
_________________
BLAG 'em up! |
|
|
jebba
|
| Posted: Mon Jun 30, 2008 5:49 pm Post subject: |
|
|
| john maclean wrote: | | If any one runs ruby I have some scripts that you can run to download stuff from frii and blagblagblag.org and publish the results. |
Can you give us a URL to them or something?
|
|
|
|
john maclean
|
| Posted: Tue Jul 01, 2008 8:48 am Post subject: |
|
|
| Code: |
#!/usr/bin/env ruby -d
# -*- mode: ruby -*-
# we assume that this library exists
require 'net/ftp'
class FtpServer
def initialize(server, user, passwd)
@server = server
@user = user
@passwd = passwd
@data = Hash.new # empty hsh to store detils about the server/session
end # def initialize
def xlogin
Net::FTP::open(@server) do |x|
puts "new ftp session initiated for #{@user}@#{@server} at #{Time.now} from host: #{ENV['USERNAME']}@#{ENV['HOSTNAME']}"
x.passive = true
p x.hash
x.login(@user, @passwd)
case @server
when "ftp.blagblagblag.org"
@data['starttime'] = Time.now
x.chdir("/pub/BLAG/linux/90000/en/iso/")
p x.pwd
p "grabbing BLAG-89999.22222-i386-netinst.iso"
@data['starttime'] = Time.now
x.hash
x.get("BLAG-89999.22222-i386-netinst.iso", "BLAG-89999.22222-i386-netinst.iso")
@data['totaltime'] = sprintf("+%f3", (Time.now - @data['starttime'] ))
when "ftp.frii.com"
x.chdir("pub/support/outgoing/")
p x.pwd
@data['starttime'] = Time.now
p "grabbing 1meg.doc"
x.get('1meg.doc', '10meg.doc', 1024)
@data['totaltime'] = sprintf("+%f3", (Time.now - @data['starttime'] ))
end # case @server
end # Net::FTP::open(@server) do |x|
end # def xlogin
end # class FtpServer
#define hosts here . we dont have to give a username an passwd for anon but we do
# these hosts are not used - they use http instead
#serverpronto = FtpServer.new("serverpronto.blagblagblag.org","anonymous", "jayeola@gail.com")
#singlehop = FtpServer.new("singlehop.blagblagblag.org","anonymous", "jayeola@gail.com")
#colostore = FtpServer.new("colostore.blagblagblag.org","anonymous", "jayeola@gail.com")
blag = FtpServer.new("ftp.blagblagblag.org", "anonymous", "jayeola@gmail.com")
frii = FtpServer.new("ftp.frii.com", "anonymous", "foo@com.org")
ftplist = [blag, frii]
ftplist.each do |xy|
p xy.xlogin
p xy.inspect
end
|
Example of results
| Code: |
load 'blagmirrors.rb'
new ftp session initiated for anonymous@ftp.blagblagblag.org at Tue Jul 01 09:07:13 +0100 2008 from host: jayeola@acid
-605353108
"/pub/BLAG/linux/90000/en/iso"
"grabbing BLAG-89999.22222-i386-netinst.iso"
"+1064.0495073"
"#<FtpServer>Tue Jul 01 09:07:14 +0100 2008, \"totaltime\"=>\"+1064.0495073\"}, @passwd=\"jayeola@gmail.com\", @user=\"anonymous\">"
new ftp session initiated for anonymous@ftp.frii.com at Tue Jul 01 09:24:58 +0100 2008 from host: jayeola@acid
-605123968
"/pub/support/outgoing"
"grabbing 1meg.doc"
"+6.7065843"
"#<FtpServer>Tue Jul 01 09:24:59 +0100 2008, \"totaltime\"=>\"+6.7065843\"}, @passwd=\"foo@com.org\", @user=\"anonymous\">"
true
|
Run as ruby -W /path/to/blagmirrors.rb. You don't have to make it executable
|
_________________
BLAG 'em up! |
|
|
jebba
|
| Posted: Tue Jul 01, 2008 2:34 pm Post subject: |
|
|
Well that looks nice, but i don't see where it outputs like wget how many kilobytes/sec.
|
|
|
|
benj
|
| Posted: Wed Jul 02, 2008 2:37 pm Post subject: |
|
|
Good morning, all,
Interesting results from the colo box in Texas (100Mbps)
The link was completely saturated while running the rsync:
This isn't my colocate, it belongs to a friend of mine. He also posted a slurm chart (I don't know anything about this, but some of you might)
Also, I thought this was interesting - when my buddy was running the rsync, it made a dent in our upstream aggregate:
I'll have some more updates later on.
|
|
|
|
jebba
|
| Posted: Wed Jul 02, 2008 9:00 pm Post subject: |
|
|
Ah cool, thx for the pretty graphs. :)
|
|
|
|
john maclean
|
| Posted: Mon Jul 14, 2008 3:57 am Post subject: |
|
|
rm if this is not helpful;
| Code: | time rsync --timeout=600 --archive rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag --verbose --verbose
sent 515016 bytes received 25007314154 bytes 4299463.45 bytes/sec
total size is 76070244768 speedup is 3.04
rsync error: some files could not be transferred (code 23) at main.c(1385) [generator=2.6.9]
real 96m56.156s
user 3m37.094s
sys 5m8.555s
|
On the French box
|
_________________
BLAG 'em up! |
|
|
|
|
|
