1,574
edits
Changes
Jump to navigation
Jump to search
Line 3:
Line 3: − + Line 12:
Line 12: − + + + + + Line 26:
Line 30: − + + + − + − + − + + + + − + − + − + Line 98:
Line 107: − + + + − + Line 116:
Line 127: − + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
→Configuring for pull
To view the status of contribs.org mirrors, look at http://mirror.contribs.org/mirrors.
To view the status of contribs.org mirrors, look at http://mirror.contribs.org/mirrors.
===Accessing the contribs.org mirros===
===Accessing the contribs.org mirrors===
To access the contribs.org download mirrors, always use the URL http://mirror.contribs.org. The current releases can be found at http://mirror.contribs.org/smeserver/releases/, the contribs section can be found at: http://mirror.contribs.org/smeserver/contribs/
To access the contribs.org download mirrors, always use the URL http://mirror.contribs.org. The current releases can be found at http://mirror.contribs.org/smeserver/releases/, the contribs section can be found at: http://mirror.contribs.org/smeserver/contribs/
{{Tip box|If you use the mirrors.contribs.org URL you will automatically be redirected to a mirror that is current within the last 8 hours.}}
{{Tip box|If you use the mirrors.contribs.org URL you will automatically be redirected to a mirror that is current within the last 8 hours.}}
The obsolete directory accounts for 16Gb, and the testing directory which include the next version SME8 is 7Gb big.
The obsolete directory accounts for 16Gb, and the testing directory which include the next version SME8 is 7Gb big.
If you are short in space you can easily save 16Gb with --exclude="obsolete/" and 7Gb more with --exclude="testing/"
If you are short in space you can easily save 16Gb with --exclude=obsolete/ in the EXCLUDE="" tag in etc/ftpsync-smeserver.conf like this:
EXCLUDE="--exclude=obsolete/"
More information on the configuration file can be found at [[:Mirrors#Configuration options]].
====Bandwith====
====Bandwith====
====Preparing your system====
====Preparing your system====
<ol><li>Create a storage location for mirror<br/>
<ol><li>Create a storage location for mirror<br />
<pre>mkdir -p {/path/to/your/data/store/}</pre>
<pre>mkdir -p {/path/to/your/data/store/}</pre>
</li><li>Create a new user to perform sync. If you are running SME Server you can create the user through the server-manager panel.
</li><li>Create a new user to perform sync. If you are running SME Server you can create the user through the server-manager panel.
</li><li>Go to the new user's directory. On SME server this will be: <tt>/home/e-smith/files/users/{user}/</tt>
cd {/path/to/user/dir/}
</li><li>Now it is time to download the ftpsync script and all files it requires:
</li><li>Now it is time to download the ftpsync script and all files it requires:
wget http://contribs.org/ftpsync.tgz
wget http://wiki.contribs.org/files/ftpsync.tgz
</li><li>Extract the tarball in users directory
</li><li>Extract the tarball in users directory
tar zxof ftpsync.tgz
tar zxof ftpsync.tgz
</li><li>Change the ownership of the directories to the new user
</li><li>Change the ownership of the directories to the new user
chown -R {user} bin etc log .ssh {/path/to/your/data/store/}
chown -R {user} bin etc log locks .ssh {/path/to/your/data/store/}
</li><li>Now we have installed and set things up as is required but we will need to update the configuration file to point to the storage location of the data (TO) in the config file (etc/ftpsync.conf). Use your favorite text editor for it.</li></ol>
</li><li>Now we have installed and set things up as is required but we will need to update the configuration file to point to the storage location of the data (TO) in the config file (etc/ftpsync-smeserver.conf). Use your favorite text editor for it.</li>
TO="/path/to/your/data/store/"
<li>'''Only''' when you are setting your server up as a [[#Configuring for pull|pull-mirror]], set the ''RSYNC_HOST'' value in the config file (etc/ftpsync-smeserver.conf) to (this is the default setting - disable for push) :<pre>RSYNC_HOST=mirror.canada.pialasse.com</pre>This because of the recent unreliability of ibiblio.org (see: [[bugzilla:7360]]).</li></ol>
====Testing your setup====
====Testing your setup====
<ol><li>Now it is time to perform the initial sync (and test that script does what it needs to)
<ol><li>Now it is time to perform the initial sync (and test that script does what it needs to)
{{Note box| This may take a long time depending on the speed of your connection}}
<pre>su - {user} -s /bin/bash
<pre>su - {user} -s /bin/bash
~/bin/ftpsync</pre>
~/bin/ftpsync sync:archive:smeserver</pre>
</li><li>Now check heck the logs to see if there are any errors. Since the initial sync will take a lot of time you can best do this in a second terminal window:
</li><li>Now check the logs to see if there are any errors. Since the initial sync will take a lot of time you can best do this in a second terminal window:
cd ~/log
cd ~/log
cat rsync-ftpsync.error.0
cat rsync-ftpsync-smeserver.error.0
</li></ol>
</li></ol>
=====Configuring for pull=====
=====Configuring for pull=====
{{Note box|We prefer you configure your mirror to be setup as a push mirror, but if you can not do so or have other ways for not doing so you can also configure your mirror to pull.}}
{{Note box|We prefer you configure your mirror to be setup as a push mirror, but if you can not do so or have other reasons for not doing so you can also configure your mirror to pull.}}
{{Note box| See Bug list at bottom of page re error on ftpsync pull request when using this version of ftpsync, see Bugzilla 11754}}
Configuring for a pull based mirror is easy. Just schedule a cron job to run every 2 hours that does the exact same sync command you do to get the mirror in the first place, you can add a comment like in the example below:
Configuring for a pull based mirror is easy. Just schedule a cron job to run every 2 hours that does the exact same sync command you do to get the mirror in the first place, you can add a comment like in the example below:
1 */2 * * * {user} ~/bin/ftpsync
1 */2 * * * {user} ~/bin/ftpsync sync:archive:smeserver
====Advertising your mirror====
====Advertising your mirror====
===Configuration options===
===Configuration options===
The ftpsync configuration script has a number of options you can configure. You might have already seen some of them when you had to adjust the storage location in the configuration process.
The ftpsync configuration file can be found in ~/etc/ftpsync-smeserver.conf. The ftpsync configuration file has a number of options you can configure. You might have already seen some of them when you had to adjust the storage location in the configuration process.
The configuration file is well documented but we will discuss some of the features here.
The configuration file is well documented but we will discuss some of the features here.
{{Incomplete}}
{{Incomplete}}
===Configure your mirror as hub===
this is intented to do on an already synced mirror in push mode.
==== setting up rsync as server====
as root.
open /etc/xinetd.d/rsync and remove the disabled line
vim /etc/xinetd.d/rsync
create a rsyncd.conf file
vim /etc/rsyncd.conf
then add this
# rsync.conf
uid = {USEROWNING THE MIRROR FILES}
gid = {GROUP OWNING THE MIRROR FILES}
use chroot = yes
max connections = 5
timeout = 300
read only = true
strict modes = true
transfer logging = true
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz
[smeserver]
path = /home/e-smith/files/ibays/{IBAY}/html
comment = SME Server Mirror
hosts allow = {YOUR CLIENT IP}
install xinetd
yum install xinetd
configure as services
ln -s /etc/rc.d/init.d/e-smith-service /etc/rc7.d/S91xinetd
config set xinetd service status enabled
config set rsyncd service TCPPort 873 access public status enabled
signal-event remoteaccess-update
service xinetd start
configure hosts.allow
mkdir -p /etc/e-smith/templates-custom/etc/hosts.allow/
echo "rsync: ALL" > /etc/e-smith/templates-custom/etc/hosts.allow/rsync
signal-event remoteaccess-update
you can browse your server like this
rsync --list-only rsync://YourIP
you can download from your server like this
rsync -avz YourIP::smeserver YourLocalFolder
==== create a rsa file ====
as user responsible of ftpsync.
ssh-keygen -qt rsa -b 1024 -C "contribs-push@{MY MIRROR}" -f ~/.ssh/contribs-push\@{MY MIRROR}.rsa
then to protect the key and limit it to one usage and only one IP. You need to change {MY MIRROR} and {MY HUB IP }
echo "no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command=\"~/bin/ftpsync\",from=\"{MY HUB IP }\" "|cat - ~/.ssh/contribs-push\@{MY MIRROR}.rsa.pub > ~/.ssh/contribs-push.tmp && mv ~/.ssh/contribs-push.tmp ~/.ssh/contribs-push\@{MY MIRROR}.rsa.pub
finally protect your keys
chmod 0600 ~/.ssh/contribs-push\@{MY MIRROR}.rs*
never give your private key, only send your public (.pub) to the downstream mirror.
==== setting up ftpsync-smeserver.conf====
as user responsible of ftpsync.
this step will allow your ftpsync to know that he need to trigger another server
edit ~/etc/ftpsync-smeserver.conf
vim ~/etc/ftpsync-smeserver.conf
under the line #HUB=false, insert
HUB=true
==== setting up runmirrors-smeserver.conf====
as user responsible of ftpsync.
vim ~/etc/runmirrors-smeserver.conf
then comment out the following line and adapt it to the path of you private key
KEYFILE=.ssh/contribs-push\@{MY MIRROR}.rsa
==== setting up runmirrors-smeserver.mirror====
as user responsible of ftpsync.
edit the file
vim ~/etc/runmirrors-smeserver.mirror
add the following at the end of the file
mhop {NAME FOR LOG PURPOSE} {YOUR CLIENT ADRESS} {YOUR CLIENT USER} -p {YOUR CLIENT SSH PORT}
===Configure your mirror under another hub===
follow : http://wiki.contribs.org/Mirrors#How_to_become_a_mirror_site.3F
except you will have to import your own public key (generated just above)
then you will have to tunes :
vim ~/etc/ftpsync-smeserver.conf
change RSYNC_HOST= to point to your hub mirror
RSYNC_HOST={HUB MIRROR ADRESSE}
tada! you are done!
=== Current SME Server contribs.org Mirror Tree ===
{| class="wikitable"
|+
| rowspan="11" |koozali.org mirrors
| rowspan="11" |mirror.canada.pialasse.com
Rsync, Push
|smeserver.bhs.mirrors.ovh.net
Rsync / 1GPs /pull %36h
|
|-
|distro.ibiblio.org
Rsync, Push
|ftp.iinet.net.au
mirror.internode.on.net
ftp.icm.edu.pl
|-
|mirror.pialasse.com
Push
|
|-
|sme-mirror.tw.co.nz
Pull
|
|-
|sme-mirror.firewall-services.com
Pull
|
|-
|ibsgaarden.dk
Push
|
|-
|mirrors.rbx.opencare.nl
Push
|
|-
|ftp.nluug.nl / ftp.vim.org
Pull
|
|-
|ftp.icm.edu.pl
Pull
|
|-
|mirrors.mab974.re
Pull
|
|-
|www.mirrorservice.org
Rsync, Pull
|
|}
====RSYNC access====
# ibiblio : rsync://distro.ibiblio.org/smeserver/releases/ ( rsync -avv --stats distro.ibiblio.org::smeserver/releases . )
# mirror.canada.pialasse.com: rsync -avv mirror.canada.pialasse.com::smeserver/releases .
# mirrorservice.org : rsync[http://rsync.mirrorservice.org/sites/mirror.contribs.org/smeserver/releases/ //rsync.mirrorservice.org/sites/mirror.contribs.org/smeserver/releases/] (rsync -avv --stats rsync.mirrorservice.org::mirror.contribs.org/smeserver/releases/ . )
====FTP access====
# ftp://distro.ibiblio.org/smeserver/releases/
=== Bugs ===
Please raise bugs under the SME-Contribs section in [http://bugs.contribs.org/enter_bug.cgi bugzilla]
and select the Mirrors component or use {{BugzillaFileBug|product=SME%20Contribs|component=Mirrors|title=this link}}
{{#bugzilla:columns=id,product,version,status,summary |sort=id |order=desc |component=Mirrors |noresultsmessage="No open bugs found."}}
[[Category:Howto]]
[[Category:Development Tools]]