BLAG

BLAG Forums
It is currently Mon Dec 22, 2014 2:58 pm

All times are UTC




Post new topic Reply to topic  [ 72 posts ]  Go to page Previous  1, 2, 3, 4, 5
Author Message
 Post subject:
PostPosted: Fri Jun 20, 2008 1:59 pm 
Offline

Joined: Sun Jun 15, 2008 2:21 pm
Posts: 5
Ok, I think I've got everything I need to open tickets with Level3 and Cogent. I'll keep everyone updated with what we hear from them, and will let you know if we need anything further.

Thanks!


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jun 20, 2008 2:38 pm 
Offline
Site Admin

Joined: Sun Mar 14, 2004 3:17 pm
Posts: 4492
Location: Loveland, Colorado, USA
Cool, thanks. keep us posted. :)


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 26, 2008 7:55 pm 
Offline

Joined: Sun Jun 15, 2008 2:21 pm
Posts: 5
Hello all,

We're still hitting our heads against the wall on this one. jebba has reported 9Mbps rsyncs from cascompany and 400Kbps using wget FTP.

If at all possible, could we get some more rsync examples?

I'm not overly familiar with rsync, can someone explain how it works, or why speeds might be higher using rsync as opposed to ftp / wget?

Again, all of us here at FRII NOC really do appreciate your help in finding a solution to this issue. I know testing can be time-consuming, but you all are really the ones needed for tests.

Thanks again.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 26, 2008 8:11 pm 
Offline
Site Admin

Joined: Wed Mar 17, 2004 6:17 pm
Posts: 1340
Location: London, UK
Code:
crontab -l   
# m h  dom mon dow   command
0 0,6,12 * * *  rsync --timeout=600 --archive  rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag/ --verbose


I use the above to mirror the BLAG 70k repository. Will run this from scratch and post a time here....

_________________
BLAG 'em up!


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jun 26, 2008 8:30 pm 
Offline

Joined: Sun Jun 15, 2008 2:21 pm
Posts: 5
john maclean wrote:
Code:
crontab -l   
# m h  dom mon dow   command
0 0,6,12 * * *  rsync --timeout=600 --archive  rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag/ --verbose


I use the above to mirror the BLAG 70k repository. Will run this from scratch and post a time here....


Cool, I was wondering how to run rsync. I'll try it from here, a colocate in Texas, and a box at home.

Thanks!


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 30, 2008 9:53 am 
Offline
Site Admin

Joined: Wed Mar 17, 2004 6:17 pm
Posts: 1340
Location: London, UK
If any one runs ruby I have some scripts that you can run to download stuff from frii and blagblagblag.org and publish the results.

_________________
BLAG 'em up!


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jun 30, 2008 5:49 pm 
Offline
Site Admin

Joined: Sun Mar 14, 2004 3:17 pm
Posts: 4492
Location: Loveland, Colorado, USA
john maclean wrote:
If any one runs ruby I have some scripts that you can run to download stuff from frii and blagblagblag.org and publish the results.


Can you give us a URL to them or something?


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 01, 2008 8:48 am 
Offline
Site Admin

Joined: Wed Mar 17, 2004 6:17 pm
Posts: 1340
Location: London, UK
Code:
#!/usr/bin/env ruby -d
# -*- mode: ruby -*-

# we assume that this library exists
require 'net/ftp'

class FtpServer


  def initialize(server, user, passwd)
    @server = server
    @user = user
    @passwd = passwd
    @data = Hash.new # empty hsh to store detils about the server/session
  end #   def initialize


  def xlogin

    Net::FTP::open(@server) do |x|
      puts "new ftp session initiated for #{@user}@#{@server} at #{Time.now} from host: #{ENV['USERNAME']}@#{ENV['HOSTNAME']}"
      x.passive = true
      p x.hash
      x.login(@user, @passwd)


      case @server

      when "ftp.blagblagblag.org"
        @data['starttime'] = Time.now       
        x.chdir("/pub/BLAG/linux/90000/en/iso/")
        p x.pwd
        p "grabbing BLAG-89999.22222-i386-netinst.iso"
        @data['starttime'] = Time.now
        x.hash
        x.get("BLAG-89999.22222-i386-netinst.iso", "BLAG-89999.22222-i386-netinst.iso")
        @data['totaltime'] =  sprintf("+%f3", (Time.now - @data['starttime']  ))

       
      when  "ftp.frii.com"
        x.chdir("pub/support/outgoing/")
        p x.pwd

        @data['starttime'] = Time.now
        p "grabbing 1meg.doc"
        x.get('1meg.doc', '10meg.doc', 1024)
        @data['totaltime'] =  sprintf("+%f3", (Time.now - @data['starttime']  ))

      end #     case @server
    end # Net::FTP::open(@server) do |x|
  end #     def xlogin
end # class FtpServer


#define hosts here . we dont have to give a username an passwd for anon but we do
# these hosts are not used - they use http instead

#serverpronto  = FtpServer.new("serverpronto.blagblagblag.org","anonymous", "jayeola@gail.com")
#singlehop  = FtpServer.new("singlehop.blagblagblag.org","anonymous", "jayeola@gail.com")
#colostore  = FtpServer.new("colostore.blagblagblag.org","anonymous", "jayeola@gail.com")

blag = FtpServer.new("ftp.blagblagblag.org", "anonymous", "jayeola@gmail.com")
frii = FtpServer.new("ftp.frii.com",  "anonymous", "foo@com.org")

ftplist = [blag, frii]
ftplist.each do |xy|

  p xy.xlogin
  p xy.inspect
end


Example of results
Code:
load 'blagmirrors.rb'
new ftp session initiated for anonymous@ftp.blagblagblag.org at Tue Jul 01 09:07:13 +0100 2008 from host: jayeola@acid
-605353108
"/pub/BLAG/linux/90000/en/iso"
"grabbing BLAG-89999.22222-i386-netinst.iso"
"+1064.0495073"
"#<FtpServer>Tue Jul 01 09:07:14 +0100 2008, \"totaltime\"=>\"+1064.0495073\"}, @passwd=\"jayeola@gmail.com\", @user=\"anonymous\">"
new ftp session initiated for anonymous@ftp.frii.com at Tue Jul 01 09:24:58 +0100 2008 from host: jayeola@acid
-605123968
"/pub/support/outgoing"
"grabbing 1meg.doc"
"+6.7065843"
"#<FtpServer>Tue Jul 01 09:24:59 +0100 2008, \"totaltime\"=>\"+6.7065843\"}, @passwd=\"foo@com.org\", @user=\"anonymous\">"
true


Run as ruby -W /path/to/blagmirrors.rb. You don't have to make it executable

_________________
BLAG 'em up!


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 01, 2008 2:34 pm 
Offline
Site Admin

Joined: Sun Mar 14, 2004 3:17 pm
Posts: 4492
Location: Loveland, Colorado, USA
Well that looks nice, but i don't see where it outputs like wget how many kilobytes/sec.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 02, 2008 2:37 pm 
Offline

Joined: Sun Jun 15, 2008 2:21 pm
Posts: 5
Good morning, all,

Interesting results from the colo box in Texas (100Mbps)

The link was completely saturated while running the rsync:

<img src="http://farm4.static.flickr.com/3030/2630483581_5561f6b256_o.png" width="853" height="288" alt="gordonrsync">

This isn't my colocate, it belongs to a friend of mine. He also posted a slurm chart (I don't know anything about this, but some of you might)

<img src="http://farm4.static.flickr.com/3280/2630483591_ae32e92c8d_o.png" width="546" height="326" alt="slurm">

Also, I thought this was interesting - when my buddy was running the rsync, it made a dent in our upstream aggregate:

<img src="http://farm4.static.flickr.com/3037/2630506703_aa1b38bf36_o.jpg" width="800" height="600" alt="jebba_out_rsync_logie">

I'll have some more updates later on.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jul 02, 2008 9:00 pm 
Offline
Site Admin

Joined: Sun Mar 14, 2004 3:17 pm
Posts: 4492
Location: Loveland, Colorado, USA
Ah cool, thx for the pretty graphs. :)


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jul 14, 2008 3:57 am 
Offline
Site Admin

Joined: Wed Mar 17, 2004 6:17 pm
Posts: 1340
Location: London, UK
rm if this is not helpful;
Code:
time rsync --timeout=600 --archive  rsync://rsync.blagblagblag.org/blag/70000 /home/jayeola/blag --verbose  --verbose
sent 515016 bytes  received 25007314154 bytes  4299463.45 bytes/sec
total size is 76070244768  speedup is 3.04
rsync error: some files could not be transferred (code 23) at main.c(1385) [generator=2.6.9]
real    96m56.156s
user    3m37.094s
sys     5m8.555s

On the French box

_________________
BLAG 'em up!


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 72 posts ]  Go to page Previous  1, 2, 3, 4, 5

All times are UTC


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group