FAQ   Search   Memberlist  
Profile    Log in to check your private messages    Register    Log in
traceroute blagblagblag.org
Goto page Previous  1, 2, 3, 4, 5
 
Post new topic   Reply to topic    BLAG Forum Index -> open discussion
View previous topic :: View next topic  
Author Message
benj
PostPosted: Fri Jun 20, 2008 1:59 pm    Post subject: Reply with quote

Ok, I think I've got everything I need to open tickets with Level3 and Cogent. I'll keep everyone updated with what we hear from them, and will let you know if we need anything further.

Thanks!

jebba
PostPosted: Fri Jun 20, 2008 2:38 pm    Post subject: Reply with quote

Cool, thanks. keep us posted. :)
benj
PostPosted: Thu Jun 26, 2008 7:55 pm    Post subject: Reply with quote

Hello all,

We're still hitting our heads against the wall on this one. jebba has reported 9Mbps rsyncs from cascompany and 400Kbps using wget FTP.

If at all possible, could we get some more rsync examples?

I'm not overly familiar with rsync, can someone explain how it works, or why speeds might be higher using rsync as opposed to ftp / wget?

Again, all of us here at FRII NOC really do appreciate your help in finding a solution to this issue. I know testing can be time-consuming, but you all are really the ones needed for tests.

Thanks again.

john maclean
PostPosted: Thu Jun 26, 2008 8:11 pm    Post subject: Reply with quote

Code:
crontab -l   
# m h  dom mon dow   command
0 0,6,12 * * *  rsync --timeout=600 --archive  rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag/ --verbose


I use the above to mirror the BLAG 70k repository. Will run this from scratch and post a time here....


_________________
BLAG 'em up!
benj
PostPosted: Thu Jun 26, 2008 8:30 pm    Post subject: Reply with quote

john maclean wrote:
Code:
crontab -l   
# m h  dom mon dow   command
0 0,6,12 * * *  rsync --timeout=600 --archive  rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag/ --verbose


I use the above to mirror the BLAG 70k repository. Will run this from scratch and post a time here....


Cool, I was wondering how to run rsync. I'll try it from here, a colocate in Texas, and a box at home.

Thanks!

john maclean
PostPosted: Mon Jun 30, 2008 9:53 am    Post subject: Reply with quote

If any one runs ruby I have some scripts that you can run to download stuff from frii and blagblagblag.org and publish the results.

_________________
BLAG 'em up!
jebba
PostPosted: Mon Jun 30, 2008 5:49 pm    Post subject: Reply with quote

john maclean wrote:
If any one runs ruby I have some scripts that you can run to download stuff from frii and blagblagblag.org and publish the results.


Can you give us a URL to them or something?

john maclean
PostPosted: Tue Jul 01, 2008 8:48 am    Post subject: Reply with quote

Code:

#!/usr/bin/env ruby -d
# -*- mode: ruby -*-

# we assume that this library exists
require 'net/ftp'

class FtpServer


  def initialize(server, user, passwd)
    @server = server
    @user = user
    @passwd = passwd
    @data = Hash.new # empty hsh to store detils about the server/session
  end #   def initialize


  def xlogin

    Net::FTP::open(@server) do |x|
      puts "new ftp session initiated for #{@user}@#{@server} at #{Time.now} from host: #{ENV['USERNAME']}@#{ENV['HOSTNAME']}"
      x.passive = true
      p x.hash
      x.login(@user, @passwd)


      case @server

      when "ftp.blagblagblag.org"
        @data['starttime'] = Time.now       
        x.chdir("/pub/BLAG/linux/90000/en/iso/")
        p x.pwd
        p "grabbing BLAG-89999.22222-i386-netinst.iso"
        @data['starttime'] = Time.now
        x.hash
        x.get("BLAG-89999.22222-i386-netinst.iso", "BLAG-89999.22222-i386-netinst.iso")
        @data['totaltime'] =  sprintf("+%f3", (Time.now - @data['starttime']  ))

       
      when  "ftp.frii.com"
        x.chdir("pub/support/outgoing/")
        p x.pwd

        @data['starttime'] = Time.now
        p "grabbing 1meg.doc"
        x.get('1meg.doc', '10meg.doc', 1024)
        @data['totaltime'] =  sprintf("+%f3", (Time.now - @data['starttime']  ))

      end #     case @server
    end # Net::FTP::open(@server) do |x|
  end #     def xlogin
end # class FtpServer


#define hosts here . we dont have to give a username an passwd for anon but we do
# these hosts are not used - they use http instead

#serverpronto  = FtpServer.new("serverpronto.blagblagblag.org","anonymous", "jayeola@gail.com")
#singlehop  = FtpServer.new("singlehop.blagblagblag.org","anonymous", "jayeola@gail.com")
#colostore  = FtpServer.new("colostore.blagblagblag.org","anonymous", "jayeola@gail.com")

blag = FtpServer.new("ftp.blagblagblag.org", "anonymous", "jayeola@gmail.com")
frii = FtpServer.new("ftp.frii.com",  "anonymous", "foo@com.org")

ftplist = [blag, frii]
ftplist.each do |xy|

  p xy.xlogin
  p xy.inspect
end


Example of results
Code:

load 'blagmirrors.rb'
new ftp session initiated for anonymous@ftp.blagblagblag.org at Tue Jul 01 09:07:13 +0100 2008 from host: jayeola@acid
-605353108
"/pub/BLAG/linux/90000/en/iso"
"grabbing BLAG-89999.22222-i386-netinst.iso"
"+1064.0495073"
"#<FtpServer>Tue Jul 01 09:07:14 +0100 2008, \"totaltime\"=>\"+1064.0495073\"}, @passwd=\"jayeola@gmail.com\", @user=\"anonymous\">"
new ftp session initiated for anonymous@ftp.frii.com at Tue Jul 01 09:24:58 +0100 2008 from host: jayeola@acid
-605123968
"/pub/support/outgoing"
"grabbing 1meg.doc"
"+6.7065843"
"#<FtpServer>Tue Jul 01 09:24:59 +0100 2008, \"totaltime\"=>\"+6.7065843\"}, @passwd=\"foo@com.org\", @user=\"anonymous\">"
true


Run as ruby -W /path/to/blagmirrors.rb. You don't have to make it executable


_________________
BLAG 'em up!
jebba
PostPosted: Tue Jul 01, 2008 2:34 pm    Post subject: Reply with quote

Well that looks nice, but i don't see where it outputs like wget how many kilobytes/sec.
benj
PostPosted: Wed Jul 02, 2008 2:37 pm    Post subject: Reply with quote

Good morning, all,

Interesting results from the colo box in Texas (100Mbps)

The link was completely saturated while running the rsync:

gordonrsync

This isn't my colocate, it belongs to a friend of mine. He also posted a slurm chart (I don't know anything about this, but some of you might)

slurm

Also, I thought this was interesting - when my buddy was running the rsync, it made a dent in our upstream aggregate:

jebba_out_rsync_logie

I'll have some more updates later on.

jebba
PostPosted: Wed Jul 02, 2008 9:00 pm    Post subject: Reply with quote

Ah cool, thx for the pretty graphs. :)
john maclean
PostPosted: Mon Jul 14, 2008 3:57 am    Post subject: Reply with quote

rm if this is not helpful;
Code:
time rsync --timeout=600 --archive  rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag --verbose  --verbose
sent 515016 bytes  received 25007314154 bytes  4299463.45 bytes/sec
total size is 76070244768  speedup is 3.04
rsync error: some files could not be transferred (code 23) at main.c(1385) [generator=2.6.9]
real    96m56.156s
user    3m37.094s
sys     5m8.555s

On the French box


_________________
BLAG 'em up!
Display posts from previous:   
Post new topic   Reply to topic    BLAG Forum Index -> open discussion Goto page Previous  1, 2, 3, 4, 5
Page 5 of 5

Protected by Anti-Spam ACP